Java Web services: Metro vs. Axis2 performance

See how Metro stacks up against Axis2 on performance, both with and without WS-Security

The Metro Web services stack provides the same functionality as the Axis2 stack but, aside from the optional use of JAXB and JAX-WS in Axis2, uses completely different implementations of the technologies involved. In this article, Dennis Sosnoski continues his Java Web services column series with a performance comparison between the Metro and Axis2 stacks, both with and without WS-Security.

Dennis Sosnoski, Architecture Consultant and Trainer, Sosnoski Software Solutions, Inc.

Author photoDennis Sosnoski is a consultant and trainer specializing in Java-based XML and Web services. His professional software development experience spans more than 30 years, with the last 10 focused on server-side XML and Java technologies. Dennis is the lead developer of the open source JiBX XML Data Binding framework and the associated JiBX/WS Web services framework, as well as a committer on the Apache Axis2 Web services framework. He was also one of the Expert Group members for the JAX-WS 2.0 and JAXB 2.0 specifications. The material for the Java Web Services series is based on Dennis' training classes.



19 January 2010

Also available in Chinese Japanese Portuguese

The Metro Web services stack is based on the reference implementations of the JAXB 2.x data-binding and JAX-WS 2.x Web services standards, but it uses additional components to provide features beyond the basic support defined by JAX-WS. WS-Security and other SOAP extension technologies are implemented by the Web Services Interoperability Technologies (WSIT) project, with the actual WS-Security processing implemented by another added component: the XML and WebServices Security Project (XWSS) (see Resources).

About this series

Web services are a crucial part of Java technology's role in enterprise computing. In this series of articles, XML and Web services consultant Dennis Sosnoski covers the major frameworks and technologies that are important to Java developers using Web services. Follow the series to stay informed of the latest developments in the field and to be aware of how you can use them to aid your programming projects.

Axis2 is based on completely different technologies, including the default Axis2 Data Binding (ADB) data-binding implementation, the Axis2 engine itself, and the Rampart module combined with the Web Services Security for Java (WSS4J) for WS-Security support. An earlier article in this series, "The high cost of (WS-)Security," showed the impact of WS-Security processing on performance in the Axis2 Web services stack.

"Introducing Metro" and "WS-Security with Metro" showed you how the two stacks differ in terms of installation, configuration, and actual usage. This article looks at the performance differences between the two, including the differences when WS-Security is used.

Looking at performance

Just like "The high cost of (WS-)Security," this article takes the approach of measuring the time required to execute a particular sequence of requests when both the client and the server are running on a single system. This approach does a great job of comparing Web services processing overhead, since the impact of network latencies and overhead are eliminated from timing results. Assuming the client code isn't significantly slower than the server, the figures are also good representations of actual server performance under load.

This article uses the same test application as the earlier article, a seismic data retrieval service. The service uses an actual database of more than 93,000 earthquakes that occurred worldwide over a period of years. Requests to the service specify a time range and a geographic coordinate range, and the service returns all earthquakes within the specified range. See "The high cost of (WS-)Security " for full details and a sample request-response message pair.

As in the previous article, two sets of request sequences were used for the performance tests. The first set used 1,000 requests, with query parameters adjusted to match a small portion of the entire earthquake database (returning only 816 matching earthquakes for the 1,000 requests). The second set used 100 requests, adjusted to match a larger portion of the database (returning 176,745 matching earthquakes for the 100 requests). Each request sequence was run multiple times in different security configurations, with only the best time for each configuration kept in the results.

The tests were run on a Mandriva 2009.1 64-bit Linux system with an Athlon X2 5400+ processor and 4 GB of RAM, using a Sun Java 1.6.0_13 32-bit JVM (which gave slightly better performance than the 64-bit JVM for a given heap size). The server code was run on Tomcat 6.0.20, configured to use 1024 MB of heap, with the client code using 512 MB of heap. The Web service stack versions were Metro 1.5 (which includes WSIT and XWSS) and Axis2 1.5.1 with a current build of the Rampart code (since there's still no release of Rampart matching the Axis2 1.5.x code). If you want to try the tests on your own hardware and JVM, download the code.

The earlier article looked only at Axis2 performance, and included plain text, SSL, and various WS-Security/WS-SecureConversation configurations. This one uses a more-limited set of configurations but directly compares the Axis2 and Metro performance for each configuration.


Performance without WS-Security

Figure 1 shows the measured test times for both Axis2 and Metro without any WS-Security usage. The chart shows that there's only a small difference between the two stacks. In the test with 1,000 requests and small responses, Metro ran half a second faster than Axis2. In the test with only 100 requests but larger responses, the two stacks ran equally fast (to within .1 seconds).

Figure 1. Test times without security
Bar graph of test times without security

These timing results show that (for the data used by the test application) Metro is probably slightly faster than Axis2 in per-request processing, but in actual data conversions the two run neck-and-neck (when using the default ADB data binding with Axis2 — other data bindings may give different results, and XMLBeans binding, in particular, is known to be considerably slower).


Performance with WS-Security

The next two figures show the relative test times for the following security configurations:

  • plain— No security (the same values as in Figure 1, but normalized to the Axis2 time)
  • username— WS-Security plain-text UsernameToken on requests
  • sign— WS-Security signing of body and headers, with timestamp
  • signencr— WS-Security signing of body and headers, with timestamp, and encryption of body

Both Figure 2 and Figure 3 show the measured times as multiples of the Axis2 plain time, to make it easy to compare the results. Figure 2 shows the times for 1,000 requests with small responses:

Figure 2. Small response test
Bar graph of small response test

Figure 3 shows the times for 100 requests with large responses:

Figure 3. Large response test
Bar graph of large response test

Metro is consistently much faster than Axis2 for the WS-Security configurations — more than twice as fast overall, and more than three times as fast in the case of the username and sign configurations for large messages. This is a surprising result for two Web services stacks at about the same level of maturity.

Rampart has rudimentary time logging built-in using the org.apache.rampart.TIME logger. By enabling this logger at DEBUG level, you can find out the amount of clock time required for various parts of the Rampart processing. Strangely, when I did this for the signatures example, I found that the Rampart processing times made up less than half of the total time taken for the test. This implies that the main performance problems with Axis2/Rampart WS-Security handling lie outside Rampart and the underlying WSS4J security implementation.

Rampart certainly has a lot of room for improvement. As mentioned in "The high cost of (WS-)Security," Rampart does build a complete in-memory model of the message any time WS-Security processing is involved. The overhead from building the in-memory model is the apparent cause of Axis2/Rampart's poor performance in the UsernameToken case. It's possible that the poor performance by Axis2/Rampart in other WS-Security scenarios also relates to this same type of conversion issue.


Wrapping up Metro

Metro can be awkward to configure for standalone use, especially given the limited documentation available (see Resources). Metro also only works with JAXB 2.x data binding and JAX-WS 2.x Web services configuration, as opposed to the wider range of data bindings and alternative configuration supported by Axis2. But Metro offers performance equal to Axis2 for plain-text message exchanges and much better performance than Axis2 when WS-Security is involved. If you're using WS-Security and concerned about performance, you should definitely look into using Metro for your application.

The next article moves on to looking at the CXF Web services stack, another Apache Foundation project. CXF uses some of the same underlying components as Axis2 but a very different style for configuring and deploying Web services. You'll see the basics of CXF usage and how it differs from Axis2 and Metro then.


Download

DescriptionNameSize
Sample code for this articlej-jws11.zip3.7MB

Resources

Learn

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Java technology on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Java technology, SOA and web services
ArticleID=462150
ArticleTitle=Java Web services: Metro vs. Axis2 performance
publish-date=01192010