Message interchange is a common mechanism for exchanging data between two applications. Another common mechanism is reading from and writing to a shared persistent store, such as a database table or a file. This interchange of data results in a degree of coupling between the two applications.
With tightly coupled applications, both the client and service application are required to change at the same time. Loosely coupled applications allow the client application to change some time later than the service application.
In this article, I will present some ideas on how to reduce the coupling between two applications in a message interchange environment. This involves understanding the message contract, dealing with changing contracts using versioning, and implementing backward compatibility into the service application.
Understanding the message contract
The first step to reduce the coupling between two applications is to understand the message contract, how it is specified, and how it can change.
The tight coupling in data exchange is not a result of the data-transport mechanism; instead, it is caused by the nature of the exchanged data. Regardless of the transport mechanism, the two applications must agree on the following details:
- Syntax: The format of the data. This includes rules that identity the elements and the order in which they occur within a message. For example, the street address element must precede the city element in a shipping label message.
- Vocabulary: The valid data value or content. This includes rules that identify the valid values for an element. For example, the country element in a shipping label message can be restricted to the valid country names specified by the US Postal Service.
- Semantics: How to process the data. This includes rules about relationships between elements and how the application is supposed to apply meaning to elements. For example, the shipping label message might omit the country element if the destination city is a valid city in the USA. Also, all packages with a USA destination are sorted differently than those with a different country destination.
- Grammar: The combination of messages. Messages often do not act alone, but are part of a larger combination of messages to form a transaction. In a conversational message exchange, a request message will have a corresponding response message. In a one-way data exchange, a datagram message might have an error report message associated with it. For example, the shipping label message has an associated shipping confirmation message.
- Dialect: The client-specific subset of the messages. Most clients do not fully utilize all the data exchange available from the service application. For example, a vendor that only ships to USA-based customers would not use the international shipping support provided by the shipping service application.
- Expectation: The client demands on the exchange. Finally, the client application may require specific quality attributes. Some quality attributes include timing (how quickly message must be processed), reliability (guarantee of delivery), and security (such as encryption).
The details in the previous section are all required to define the message interchange. Some authors refer to this as the message treaty or the message interface. I prefer to call this the message contract since it represents an agreement between the two applications about all aspects of the message exchange.
Formal standards can specify parts of the contract:
- Document Type Definition (DTD) for syntax rules
- XML Schema Definition (XSD) for syntax and vocabulary rules
- Web Services Definition Language (WSDL) for syntax, vocabulary, and grammar rules
The other details of the contract are documented using textual descriptions, sequence diagrams, and sample code examples. These are put in a document that supplements the formal specifications.
It is beyond the scope of this article to present approaches to documenting the message contract. The important issue in this article is to understand that message exchange is more than just the format (syntax) and content (vocabulary). You must address all aspects of the message contract when you implement a message exchange between two applications.
No matter how well the original message contract is defined, it is going to eventually change. A change to any one of the contract details (syntax, vocabulary, and so on) results in a new contract.
The addition, change, or removal of elements is a syntax change. All syntax changes will have a corresponding vocabulary and semantic change. Vocabulary changes might result in a semantic change. And so forth.
In practice, I have observed that the most common form of change comes from adding new information -- new elements (syntax change) or new data values (vocabulary change) -- to the message. It is necessary to plan for all forms of change, though, and not just assume additive changes.
To illustrate a contract change, look at a message that a business used to accept shipping requests from residential customers. In this message, the business assumed all shipping was to residential addresses, so only one address line is provided.
Listing 1. shippingRequest v1 provides only one address line
<shippingRequest>
<destination>
<name>H.J. Simpson</name>
<street>742 Evergreen Terrace</street>
<city>Springfield</city>
</destination>
</shippingRequest>
|
Soon customers wanted packages shipped to office addresses instead of residential addresses, so the business had to change the message to accept additional address information.
Listing 2. shippingRequest v2 adds more address information
<shippingRequest>
<destination>
<name>H.J. Simpson</name>
<company>Springfield Nuclear Power</company>
<addressLine1>1 Springfield Way</addressLine1>
<addressLine2 />
<city>Springfield</city>
</destination>
</shippingRequest>
|
In this example, the new
<shippingRequest> has two new optional elements,
<company> and
<addressLine2>, and one renamed element,
<street> is now
<addressLine1>.
The service application must be changed to process the new
<shippingRequest> message. Since the two new elements are optional, the client application might not have to change.
However, the renamed element forces the client application to change.
When the service application is deployed with the new message contract, every client application must be updated at the same time or client messages will be rejected. This synchronization of deploying new client programs with the service programs can result in long lead times, increased project management, increased coordination between program owners and stakeholders, and a long downtime to ensure no messages are sent that the partner program cannot process.
Manage changing message contracts with versioning
The second step to reduce the coupling is to add versioning information to the message contract. This is a prerequisite to the final step of implementing backward compatibility in the service application.
Several issues are associated with the addition of versioning:
- How to add version information
- How to identify versions
- How to identify response version
One important note: I use XML-based messages in the discussion and examples in this article; however, versioning is not limited to just XML-based messages. Versioning should be added to all message forms: XML, tag/length, such as Electronic Data Interchange (EDI), fixed length, character delimited, or any other form. The concepts are the same; only the details vary.
You can choose from several approaches when you add version information to a message:
- DOCTYPE
- Root element tag name
- Root element tag attribute
- Namespaces
When you use DOCTYPE, you can specify the version with a uniquely named DTD identifier in the DOCTYPE statement. The use of the DOCTYPE to identify a DTD has the benefit that the XML parser can retrieve the referenced DTD and validate the message. Several disadvantages of DOCTYPE are the inability to retrieve the DTD URL through some XML parsers and that the DOCTYPE is lost when the message is wrapped in an enveloping XML, such as a SOAP envelope.
Listing 3 shows the use of DOCTYPE for the <shippingRequest> example.
Listing 3. Using DOCTYPE to specify version
<!DOCTYPE shippingRequest SYSTEM "shippingRequest_V2.dtd">
<shippingRequest>
. . .
</shippingRequest>
|
When you use the root element tag name, you can specify the version as part of the root element tag name. The tag name includes a version number for that message. You can combine this approach with the DOCTYPE approach to enable the XML parser to perform DTD-based validation. This approach allows the version to be easily retrieved by looking at the first element. Finally, you can even use this approach when the message is wrapped in an enveloping XML.
Listing 4 shows the use of the root element tag name for the <shippingRequest> example.
Listing 4. Using root element tag name for version
<shippingRequest_v1>
. . .
</shippingRequest_v1>
-OR-
<shippingRequest_v2>
. . .
</shippingRequest_v2>
|
The root element tag attribute approach is similar to the root element tag name approach. Instead of renaming the root element tag name, the version is added to a new version attribute on the root element tag. Use of an attribute instead of the tag name reduces the ability to perform DTD-based validation.
Listing 5 shows the use of the root element tag attribute for the <shippingRequest> example.
Listing 5. Using root element attribute for version
<shippingRequest version="1"> . . . </shippingRequest> |
Finally, in the namespaces approach, the ability to qualify sections of the message as belonging to a specific namespace identifies the version of that section. This has the benefit of managing a message at a smaller level than the whole message level. For example, if the address information was associated with the address namespace and the customer information was associated with the customer namespace, then it is possible to change just the address portion without having to change the customer information.
However, not all XML parsers recognize or properly parse namespaces, especially older parser versions. There are also deployment issues involved with whether to distribute a document that the namespace URI references and how to keep distributed copies synchronized.
Listing 6 shows the use of the namespace for the <shippingRequest> example.
Listing 6. Using namespaces for version
<?xml version="1.0"?> <shippingRequest xmlns="http://myexample.edu/shippingRequest/v2"> . . . </shippingRequest> |
In addition to deciding where to put the version information, you must decide how to identify the version. I look at the following three suggestions:
- Service application version
- Sequence number
- Date
Let's start with the service application version. All messages associated with a given service application are identified with the version number of that service. For example, if the service application is at version 1.5.1, then the messages is identified with version 1.5.1.
This approach has many problems, chief of which is the message contract will change with every new version of the application. In practice, messages change far less often than the application. So, pinning the message version to the application version results in more program changes than necessary.
With sequence number, each message in the message contract is assigned a version number, usually a sequentially assigned number starting with 1. When a message's contract changes, then its version number is incremented by one. The following table shows that each message is assigned its own version number.
| Message | Version |
|---|---|
| shippingRequest | 2 |
| shippingResponse | 4 |
| purchaseRequest | 1 |
| purchaseChangeRequest | 3 |
| purchaseResponse | 2 |
With the date approach, each message in the message contract is assigned a version identifier based on the date the message was last changed. For example, if the <shippingRequest> message was changed in August 2004, then its version number in the message contract could be "2004_08". This approach is similar to the sequence number approach with the advantage that the date informs the user when the new message version was implemented.
Response version identification
What about the case of a request message that has an expected response message? The client application will want the response to arrive in a format that the client is able to process. Maybe the client has not been updated to the latest response message version.
The solution is any request message or datagram message that could have a response message will need to specify the desired version of the response as part of the request. So, a request message must add an attribute to identify this desired response version.
Probably the easiest way to do this is to add a responseVersion attribute on the request message's root element. Listing 7 shows how this might be done using the root element tag attribute approach:
Listing 7. Adding response version attribute
<shippingRequest version="1" responseVersion="2"> . . . </shippingRequest> |
Implement backward compatibility in the service application
The last step to reduce coupling is to add backward compatibility in the service application. The service application must be able to handle one or more earlier versions of messages. In this way, a client that uses version N-1 of a message is able to communicate with a server that uses version N of a message.
Figure 1 shows the components that comprise the request message and response message path of the service application.
Figure 1. Backward-compatible message-processing components
A request message arrives through the request channel. The request message is partially parsed by the content-based router to determine which version translator to use for parsing the message. The translators for the current and previous versions will load the requested data-transfer object (DTO). Since the DTO is most likely designed according to the current request version, the previous version translators will need to use rules to populate missing elements with default values.
The request-processing path must pass to the response-processing path the additional information needed to deliver the response to the correct client. This additional information includes the requested version of the response message, the destination queue or address where the response message should be sent, and the correlation ID that the client uses to ensure it received the desired response message.
Once the service process finishes processing the request, it builds the response DTO. Using the response version identified in the request message, the content splitter invokes the appropriate response translator to build the response message. Usually, previous versions of the response will have fewer data elements than the current version, so the previous version messages will ignore the additional information found in the response DTO. Each response translator will put the composed response message on the response channel for delivery to the requesting client.
As Figure 1 illustrates, you must add a number of additional components to the application to support backward compatibility of message contracts. These additional components include the previous version translators, the content-based router, and the content splitter.
As I mentioned before, versioning reduces coupling, but it does not eliminate it. You must still address a number of issues and scenarios that might require simultaneous updates to the client and service applications, including (but not limited to):
- Maintenance overhead
- Client control
- Unavoidable (mandatory) changes
The owner of the service application will have to decide how many previous versions of a message to support. Every previous message version supported increases the maintenance work associated with the service application.
The amount of control over the client applications might determine the number of previous message versions supported. When the client applications are in-house versions, direct corporate control is available to ensure the client applications are upgraded which reduces the number of previous versions to support.
However, when the client applications are external, very little control is available. This is likely to increase the number of previous versions supported.
Some changes to messages are mandatory. This might be in the form of government requirements or business reporting needs. In this case, the client and service applications must be updated at the same time to ensure the mandatory changes go into effect.
When you add versioning to your message contracts, you can reduce the coupling between applications. The versioning must identify the version of the message and the accepted version of any replies to the message. But the addition of versioning to the message is not enough alone. You must modify the service application to support the parsing and build of messages with versioning.
While versioning reduces coupling and allows client application to be more independent of service changes, it comes at the price of increased maintenance work on the service application.
Learn
- Quality busters: What version is this? discusses the issues around identifying component versions and dependencies (developerWorks, November 2004).
- Quality busters: Losing messages introduces message versioning as one of several issues to address in a message-oriented architecture (developerWorks, December 2004).
- Enterprise integration patterns: Designing, building, and
deploying messaging solutions by Gregor Hohpe and Bobby Woolf is an encyclopedia of message-oriented design patterns and terminology(Addison-Wesley, 2004).
- Loosely coupled: The missing pieces of Web services by Doug Kaye presents many open issues to address in message-oriented solutions, especially those using Web services approaches (RDS, 2003).
- Software fortresses: Modeling enterprise architectures by Roger Sessions introduces a high-level modeling approach for application integration in an enterprise environment. In the book, the author introduces the idea of application fortresses and integration is based on treaties between the applications (Addison-Wesley, 2003).
- Read detailed suggestions on adding version information to XML-formatted messages using namespaces in Designing extensible, versionable XML formats (MSDN, July 2004).
- Other articles about versioning on the developerWorks site include:
- Best practices for Web services versioning outlines the scope of the versioning difficulties (developerWorks, January 2004).
- Tip: Namespaces and versioning discusses how to use XML namespaces to version formats (developerWorks, June 2002).
- Make minor backward-compatible changes to your Web services explains the changes you can make to an existing set of WSDL and XSD files that are backward compatible so that you can evolve your service (developerWorks, November 2004).
- The developerWorks Web Architecture zone specializes in articles covering various Web-based solutions.
Discuss
- Get involved in the developerWorks community by participating in
developerWorks
blogs.
Michael Russell has a Bachelors Degree in Physics and a Masters Degree in Computer Science. He was a logistics engineer, a technical services manager, and a certified IT architect at IBM for nearly 14 years; he is currently a Web application architect for a resort company in Orlando, Florida. He has experience in Windows, UNIX, and OS/400 environments and uses Web technology for entertainment through his own company, Vicki Fox Productions.
Comments (Undergoing maintenance)
Table of contents
Local resources
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
