Level: Introductory Frank Cohen (fcohen@pushtotest.com), CEO, PushToTest
01 Nov 2001 SOAP is a boon in the quest for interoperability between servers -- but it is still a challenge to deliver SOAP-based solutions which perform well in production environments. Frank Cohen offers tips on overcoming scalability problems, and introduces a free open-source utility to proof performance in SOAP-based Web services.
Visit most Web sites today and you are using more than one server software
application package. This is especially true for dynamic Web sites
offering personalized content. If you sign in to the human resources Web
pages on your company internal portal, will the same sign-in be valid when
you check manufacturing's shipment reports?
Behind the scenes, teams of engineers are doing their best to keep the
back-end systems interoperating. New hardware (routers, servers, and storage)
and revisions to server software (application servers, databases, and ERP
solutions) appear with alarming frequency. Software engineers are constantly
writing expensive and difficult-to-maintain custom software to keep the
systems sharing data.
SOAP framework
SOAP is the best hope yet to develop a lingua franca for
server-to-server communication. The benefits of being able to write server
applications that freely communicate with other servers, platforms and
hardware are many. Less time is spent on maintaining the code and the
programmer gets to concentrate on solving bigger problems. The operations
manager has a wider choice of platforms and hardware. And users benefit by
having more advanced applications available.
 | |
Tools using SOAP to enable interoperable software are inexpensive, freely
available, and widely supported. Your considerations for tools, hardware
and network equipment will greatly determine the performance and
scalability potential of your deployed SOAP-based Web services. With that
in mind, I would like to discuss a scalable framework for developing Web
services, strategies for avoiding performance problems, and offer an
open-source set of test objects and scripting language called Load that
can help with performance and scalability testing.
Framework for developing Web services
SOAP is a lightweight protocol intended to fit into your existing Web
application infrastructure. An emerging framework for developing scalable
SOAP-based Web services favors a Web architecture with many small servers
that are accessed through a load balancer, providing a front-end to a
powerful database server.
Figure 1: Framework for developing Web services
The framework for building SOAP-based Web Services in Java uses these
components (see Resources for links to the components listed here):
-
1. Apache SOAP
Based largely on IBM's SOAP contribution, Apache SOAP is an open-source
project delivering a full-featured SOAP implementation for Java. Apache
SOAP implements most of the SOAP v1.1 specification, supports SOAP
messages, server and client implementations, and comes with full source
code under an Apache-style license (which means you can change the code
and even deploy proprietary software products with your changes.)
-
2. JDOM
JDOM may become part of Java, as Sun accepted JDOM as JSR102 (see
Resources). Apache SOAP comes with the Xerces XML parser -- however, any
SAX-compliant XML parser can be used instead. Java developers will find
JDOM to be an easier, friendlier API to use to manipulate SOAP's XML
documents, and it also allows you to change the underlying XML parser
without recoding the SOAP application. This flexibility gives you many
choices when trying to solve scalability or performance problems in a
particular XML parser. JDOM is also distributed under an Apache-style
open-source license.
-
3. Load balancer with SSL support
The SOAP 1.1 protocol does not yet define encryption and authentication
methods. Until SOAP defines an authentication method, the framework
recommends you write your business logic into a servlet, then use the
underlying Web server's SSL support to make an HTTPS request to the Web
service. The load balancer's SSL support unecrypts the request and passes
it along to the Web service as an unencrypted SOAP call. This frees up
your Web service server from the computing overhead of SSL.
Figure 2: Points of failure
-
4. Load balancer with cookie-based session tracking
SOAP 1.1 does not yet define a session management mechanism. In a
load-balanced environment, some of your SOAP requests are bound to carry
stateful information. For example, communication with a Web service may
require multiple requests and responses in series (the load balancer must
have the option to bring your request to the same Web service server
during a session). Most load balancers today support cookie-based session
tracking.
This framework has many benefits. Java engineers find debugging less
complex because fewer threads are typically running at any time. Company
financial managers like the architecture because they can buy lots of
small, inexpensive servers and avoid the giant system purchases. Network
managers like the flexibility all those small servers give.
This leaves us with the question: Is SOAP ready for production
environments?
Design for performance and scalability
SOAP is still a very new and untested system. Inside SOAP are many places
to harbor performance and scalability problems. Determining
production-worthiness requires both unit- and system-level testing.
The SOAP protocol uses a multistep process to complete a communication
transaction. The SOAP request begins with the business logic of your
application learning the method and parameter to call from a Web Services
Description Language (WSDL) document.
Figure 3: Designing for performance and scalability
As an example, Listing 1 is part of the WSDL for a publicly available Web
service that returns the current weather for a US postal zip code (see
Resources for a link to the full WSDL).
Listing 1: Web service returns the current weather
<message name = "getTempRequest">
<part name = "zipcode" type = "xsd:string"/>
</message>
<message name = "getTempResponse">
<part name = "return" type = "xsd:float"/>
</message>
|
The weather service requires you to call the getTempRequest method by passing in a zipcode value as a string and receiving the
temperature as a floating-point value in the response.
Since the WSDL rarely changes, many developers embed the WSDL definition
into their code to avoid the overhead of getting the WSDL every time. While
this will improve performance it also becomes a maintenance headache when
the WSDL changes over time.
The better way to avoid maintenance problems is to cache the WSDL in the
centralized database and then periodically check the timestamp/version
number of the WSDL to see if a newer one is available.
Another way to improve performance is to turn XML validation off. In this
case, your application should do light validation of the response results.
For example, this WSDL in Listing 2 defines the schema for the response.
Listing 2: Light XML validation
<element name="zipcode" type="int"/>
<element name="temperature" type="float"/>
<element name="remarks" type="string"/>
|
The result of a call to this service looks like Listing 3.
Listing 3: The result
<zipcode>95008</zipcode>
<temperature>65 F</temperature>
<remarks>Storm warning</remarks>
|
This response should throw an exception because the temperature value is
not a float type -- it is actually a string. Validating the response
yourself will normally be much faster than depending on the DTD or XML
schema code to validate the response.
Parameter types in SOAP present a possible scalability problem. SOAP
defines simple data types: String, Int, Float, and NegativeInteger. The
WSDL may include non-trivial new data types. For example, imagine the
temperature Web service also retrieved maps. The schema for the call might
look like Listing 4.
Listing 4: Retrieving a map
<message name = "getTempRequest">
<part name = "zipcode" type = "xsd:string"/>
</message>
<message name = "getTempResponse">
<part name = "return" type = "xsd:float"/>
<part name = "map" type = "xsd:
http://www.pushtotest.com/wsdl/mapformat"/>
</message>
|
While reading the response, a validating XML parser will contact the
pushtotest.com host to get the XML Schema definition for the mapformat. The overhead of this request can make a
system unscalable if the validating parser does not cache the schema
definitions.
A general performance rule is to stay with the simple SOAP data types
unless there is a compelling need to use another data type. Each new data
type introduces a serializer to convert from the XML value into a Java
value and back again. The serializer may cause performance problems or just
be buggy. For example, the Apache and Microsoft SOAP implementations both
include a BigDecimal data type. However, they are not compatible. Glue is a
commercial product that maps between the platform differences of XML. See
Resources for links to Glue and related pages.
While SOAP was designed to work within existing Web application
environments, the protocol may introduce firewall and routing problems.
Unlike a normal Web server using HTTP, all SOAP messages are the
equivalent of HTTP form submits. The calls move much more data than the
average HTTP GET or POST call, and network performance is bound to be
impacted.
Special testing of the firewall and routing equipment should be
undertaken. For example, you should check your firewall's security policy
to make certain it doesn't monitor SOAP-requests as Web traffic. If it
does, you may find the firewall shunting away traffic that looks like a
Denial of Service (DoS) attack.
The early Web services are very straightforward -- you make a SOAP call
and get a response. More advanced SOAP applications make series of get and
response calls until a transaction is finished. Transactional SOAP calls
need to identify -- and cache the state of -- sessions. Caching mechanisms
for SOAP transactions are potential problem spots for scalability.
Now that I have covered the places to look for scalability and
performance problems, I can turn my attention to actually testing
SOAP-based Web services for scalability and performance.
Test strategies using Load
Moving a SOAP-based Web service into a production environment requires
assurances of high availability and consistently good performance. Here is
a checklist Java developers should keep in mind when planning to test a Web
service:
-
Stateful testing
When you use SOAP to set a server value, does the server respond
correctly later on?
-
Privilege testing
What happens when the everyday user tries to access a control that is
authorized only for administrators?
-
Speed testing
Is the Web service taking too long to respond?
-
Boundary timing testing
What happens when your Web service request times-out, or takes a really
long time to respond?
-
Regression testing
Did a new build break an existing Web service function?
These are fairly common tests for any software application. Since this is a
Web service, though, the testing arena expands into a matrix, as the
following table of a Web service test suite describes:
| Web Service Test Suite | 1 | 50 | 500 | | Stateful testing | yes | yes | no | | Privilege testing | yes | no | no | | Speed testing | no | no | no | | Boundary timing testing | no | no | no | | Regression | no | no | no |
Even if your company is sloppy at testing and doesn't invest in test
software, in the past you could test a Web application yourself using a
Web browser. Not so with a SOAP-based Web service. Manually reading the
XML documents emitted during a SOAP transaction becomes time consuming
very rapidly. Developing and using automated test suites is a must.
A free open-source utility, Load, is available at my company site,
Pushtotest, as well as at the Open source zone on IBM developerWorks (see
Resources for links). Load is available to write test automation suites
for performance and scalability testing of SOAP-based Web services. It
offers a scripting language to operate a library of test objects, and
enables you to develop intelligent scripts that drive a Web service.
Multiple concurrent running copies of the script show whether the Web
service performs under simulated production situations.
Let's see how Load would be used to call the weather temperature service
discussed previously. Listing 5 is the script in its entirety.
Listing 5: Testing weather-reporting Web service with Load
<load>
<script>
<!-- Tells where to find the weather service -->
<variable name="serviceurl"
value="http://services.xmethods.net/soap/servlet/rpcrouter"/>
<!-- Establishes a variable to hold the zip code -->
<variable name="thezip" value="95008" />
<!-- Identify the source of the delimited data -->
<soapsource name="temperature" target="urn:xmethods-Temperature"
method="getTemp" url="$serviceurl;"/>
<soapsource name="temperature" action="addparameter"
parameter="zipcode" value="$thezip;"/>
<soapsource name="temperature" action="call"/>
<echo message="The temperature is $soapsource[ temperature, 0];" />
<echo message="That request took $soapsource[ temperature, totaltime];
milliseconds" />
</script>
</load>
|
Now let's look at how the individual functions work.
These are preambles to identify that the following are script elements for
the Load program to process.
<!-- Tells where to find the weather service -->
<variable name="serviceurl"
value="http://services.xmethods.net/soap/servlet/rpcrouter"/>
<!-- Establishes a variable to hold the zip code -->
<variable name="thezip" value="95008" />
|
You create two variables: the first holds the URI to the Web service and
the second holds the zip code value.
<!-- Identify the source of the delimited data -->
<soapsource name="temperature" target="urn:xmethods-Temperature"
method="getTemp" url="$serviceurl;"/>
|
Next you create a new soapsource object called temperature. The temperature object will call the
getTemp method on the xmethods-Temperature Web
service on the machine defined in the URL.
<soapsource name="temperature" action="addparameter"
parameter="zipcode" value="$thezip;"/>
|
This adds the zip code parameter to the temperature object. You can use
the addparameter action to add as many
parameters as you need. The WSDL of the Web service defines the expected
parameters.
<soapsource name="temperature" action="call"/>
|
Calling the Web service initiates an http connection to the target server,
marshals an XML document with the request and the added parameters, then
waits for the response.
<echo message="The temperature is $soapsource[ temperature, 0];" />
|
Load looks at the response document and finds the response field in
position 0. The value is displayed on the Load GUI for you to see.
<echo message="That request took $soapsource[ temperature, totaltime];
milliseconds" />
|
The totaltime method of the temperature object
returns the time in milliseconds it took to make and complete the request.
Once you are satisfied with the script, Load can run the script
concurrently to test the Web service under a simulated load.
<load>
<script sessions="25">
. . .
</script>
</load>
|
This creates 25 copies of the script and runs it concurrently. Load also
includes commands for looping, random values, posting dummy text, and
access to 20 other test objects.
Load is licensed under terms similar to those of the Apache Web server. You
download the Load program and all its source code. You can change Load to
fix bugs and add new features. The license even lets you build commercially
exploitable new products based on the code. |
Active SOAP
Examples of where SOAP makes better sense than current solutions:
- Single-sign-in between server systems. A sign-in Web service returns
a user id number indicating if a user is authorized to access Web site
resources.
- Database queries (much less buggy than JDBC drivers). The request
goes directly to your database server containing an SQL query, the
response is an XML document with the rows and fields.
- Store-and-forward queue mechanism for sending email confirmations.
When a user registers for a new site the server asks the "Thank you" Web
service to send an e-mail to the new user thanking them for registering.
|
|
Conclusion
Programming and delivering production-quality Web services is made easier
and faster when you test quality under the stress of multiple concurrent
requests. The scripting language and test objects in the open-source Load
utility can offer you a way to make your work more productive when testing
SOAP-based Web services.
Resources
About the author | | |
Frank Cohen is a software entrepreneur who has contributed to the worldwide success of personal computers since 1975. He began by writing operating systems for microcomputers, helping establish video games as an industry, helping establish the Norton Utilities franchise, leading Apple's efforts into middleware and Internet technologies, and most recently serving as principle architect for the Sun Community Server, Inclusion.net, and TuneUp.com. Frank maintains the open-source Load project and is CEO for PushToTest, a scalability and performance testing solutions company. More information is available at www.PushToTest.com. You can reach Frank at fcohen@pushtotest.com.
|
Rate this page
| 
