In a previous article, Kyle Brown and Carolyn Norton have written about the notion of Defensive Architecture, and shown some of the decisions that you can make in developing your overall application architecture and infrastructure architecture to be more resilient in the face of failure and change. In this article, we will further examine these notions by looking at two examples where you can improve problem determination through the use of defensive techniques. We will first examine message-driven beans (MDBs) and runtime deployments supporting them, and then look at how an enterprise service bus (ESB) can be helpful in building a defensive infrastructure.
Keeping things simple is a two-edged sword. Often you can design an infrastructure with the goal of making it less complex; but in trying to keep things simple you make your application runtime more complex. Lets look at a simple problem that illustrates this principle:
In this example we have an application module (A) that not only takes input off a queue but also has data it needs to send to to another application (B) over the same queue in order to have that application process the data. B has mdb-x and mdb-y which process messages for specific destinations. In this case the flow is that Application A creates a message that is processed in Application B by mdb-x and which then creates another message that is processed by mdb-y.
There are, at least, two ways one can approach this deployment and how queues are utilized.
Figure 1. Process A sends message to Process B which puts another message onto the same queue
Figure 2. Process B1 outputs its message to a separate queue that flows to a B2 deployment
Let us examine both approaches and see compare and contrast the complexity of the runtime with a specific slant on problem determination in both environments.
Figure 1 shows what seems to be a very simple queue deployment. The application module is configured to feed data it needs to process in another step back to itself. There is less to deploy and configure because everything is self contained. It seems to adhere to our keep it simple philosophy.
Figure 2 shows what looks like a more complex administrative task. The same application module is deployed more than once; what's more there are several queues to maintain. Some will argue that having so many moving parts leaves things open for being mis-configured when deployed. But if administrators use scripts to build each environment (functional test, user acceptance test, performance test, production) that once the application is promoted to production the repeatable scripts will have been well tested and less likely to fail.
When looking at end to end throughput of an environment a crucial measuring point would be the queue depths and how quickly they drain. In Figure 1 when the queue depth is measured it is measured as an aggregate where the combined postings from both the remote process and the application module itself thereby making it difficult to tell if the queue is filling up due to the remote process or the local. There is no way to differentiate between the messages coming from one source or another.
Using the runtime in Figure 2 makes several things easily obtained. First, the queue depths of both the remote process and those generated by messages from the application module itself are easily distinguished from each other. Since applications only listen to one queue at a time it does require the deployment of another MDB to catch those messages.
Figure 2 can easily have the 2nd deployment of the application MDB occur on separate hardware. This allows for handy separation should any capacity increases occur. It also allows for separately tuning the JVM to the specific characteristics of the application's function if the MDBs are deployed to separate JVMs. It is not unusual to have different MDBs exhibit different performance characteristics because they have wildly different function.
There is a third option sort of a cross between Figure 1 and 2 where the B deployment remains in a single JVM but uses a 2nd queue to separate the messages. This may also be considered for MDBs that co-exist peacefully in the same JVM.
This same issue that sometimes “less does not equal more” comes up again in the case of mediations in an ESB. Consider the following problem: We have two SOAP messages in our application. One needs to be translated from one XML schema to another between the service requestor and the service provider. The other one, however, does not need translation.
In this case the simplest solution would be to place a mediation on the first message, but not on the second one. In fact, you might go so far as to not even use an ESB for the second message since there's no obvious need for mediations. However, using the ESB and perhaps adding a mediation in the second case, even if it does not do any translation is still valuable.
Consider the issue of problem determination of the two messages; in the first case, it's simple to log the message that has been translated both on its incoming (untranslated) and outgoing (translated) versions. In the second case where the message does not go through the ESB, problem determination is more difficult since you cannot easily examine the message as it flows from requestor to provider. So, routing the message through the ESB allows us to add a simple logging mediation when it is needed. Even though you've added some complexity at deployment time, the benefits gained in problem determination make it worthwhile.
While a runtime environment can seem to have a simple configuration it may impede the ability to properly conduct problem determination and/or performance tuning. This is a reason to always be looking from a different angle at alternative architectures that provide defensive capabilities in terms of how the environment can be monitored, controlled and performance tuned. Commonly the performance test environment tends to be where such defensive architectures readily become apparent.
Kyle Brown is a Distinguished Engineer with IBM Software Services for WebSphere specializing in SOA and Emerging Technologies. Kyle provides consulting services, education, and mentoring on SOA, object-oriented topics and J2EE technologies to Fortune 500 clients. He is a co-author of Java Programming with IBM WebSphere and Persistence in the Enterprise. He is also a frequent conference speaker on the topics of SOA, Enterprise Java, OO design, and design patterns.
Alexandre Polozoff is an IBM Master Inventor and a Senior Certified IT Specialist in the Software Group Performance Technology Practice across the WebSphere product stacks (Application Server, Portal, Process Server, etc) at large topology, international customer sites. He has authored numerous papers and blogs related to performance problem determination around J2EE technologies and speaks at various conferences.