Skip to main content

Problem Determination through Defensive Infrastructure Architecture: Part 1: Runtime deployments with queues and ESBs

Kyle Brown (brownkyl@us.ibm.com), Distinguished Engineer, IBM
Kyle Brown
Kyle Brown is a Distinguished Engineer with IBM Software Services for WebSphere specializing in SOA and Emerging Technologies. Kyle provides consulting services, education, and mentoring on SOA, object-oriented topics and J2EE technologies to Fortune 500 clients. He is a co-author of Java Programming with IBM WebSphere and Persistence in the Enterprise. He is also a frequent conference speaker on the topics of SOA, Enterprise Java, OO design, and design patterns.
Alexandre Polozoff (polozoff@us.ibm.com), Sr. Certified IT Specialist, IBM
Alexandre Polozoff
Alexandre Polozoff is an IBM Master Inventor and a Senior Certified IT Specialist in the Software Group Performance Technology Practice across the WebSphere product stacks (Application Server, Portal, Process Server, etc) at large topology, international customer sites. He has authored numerous papers and blogs related to performance problem determination around J2EE technologies and speaks at various conferences.

Summary:  The widely known "keep it simple strategy" (KISS) is a wise practice to follow when building out an infrastructure. Application deployments, packaging and placement requires considerable thought and effort to execute well. However, sometimes what appears to be a simple implementation can hinder problem determination leaving a production runtime unstable or, worse, unavailable. What is simpler for one side of the IT development or operations staff makes troubleshooting and root cause analysis that much harder because it is difficult to separate out discrete events or the lack of valuable but missing data. This article looks at some real life examples where "more is actually less" when it comes to KISS.

View more content in this series

Date:  14 Jul 2009
Level:  Intermediate PDF:  A4 and Letter (42KB | 6 pages)Get Adobe® Reader®
Activity:  3116 views

Introduction

In a previous article, Kyle Brown and Carolyn Norton have written about the notion of Defensive Architecture, and shown some of the decisions that you can make in developing your overall application architecture and infrastructure architecture to be more resilient in the face of failure and change. In this article, we will further examine these notions by looking at two examples where you can improve problem determination through the use of defensive techniques. We will first examine message-driven beans (MDBs) and runtime deployments supporting them, and then look at how an enterprise service bus (ESB) can be helpful in building a defensive infrastructure.


Keeping it Simple

Keeping things simple is a two-edged sword. Often you can design an infrastructure with the goal of making it less complex; but in trying to keep things simple you make your application runtime more complex. Lets look at a simple problem that illustrates this principle:

In this example we have an application module (A) that not only takes input off a queue but also has data it needs to send to to another application (B) over the same queue in order to have that application process the data. B has mdb-x and mdb-y which process messages for specific destinations. In this case the flow is that Application A creates a message that is processed in Application B by mdb-x and which then creates another message that is processed by mdb-y.

There are, at least, two ways one can approach this deployment and how queues are utilized.


Figure 1. Process A sends message to Process B which puts another message onto the same queue
 Process A sends message to Process B which puts another message onto the same queue

Figure 2. Process B1 outputs its message to a separate queue that flows to a B2 deployment
Process B1 outputs its message to a separate queue that flows to a B2 deployment

Let us examine both approaches and see compare and contrast the complexity of the runtime with a specific slant on problem determination in both environments.

Figure 1 shows what seems to be a very simple queue deployment. The application module is configured to feed data it needs to process in another step back to itself. There is less to deploy and configure because everything is self contained. It seems to adhere to our keep it simple philosophy.

Figure 2 shows what looks like a more complex administrative task. The same application module is deployed more than once; what's more there are several queues to maintain. Some will argue that having so many moving parts leaves things open for being mis-configured when deployed. But if administrators use scripts to build each environment (functional test, user acceptance test, performance test, production) that once the application is promoted to production the repeatable scripts will have been well tested and less likely to fail.

While on the surface Figure 2 looks more complex it actually makes problem determination simpler than the runtime depicted in Figure 1 for the following reasons:


Measuring throughput

When looking at end to end throughput of an environment a crucial measuring point would be the queue depths and how quickly they drain. In Figure 1 when the queue depth is measured it is measured as an aggregate where the combined postings from both the remote process and the application module itself thereby making it difficult to tell if the queue is filling up due to the remote process or the local. There is no way to differentiate between the messages coming from one source or another.

Using the runtime in Figure 2 makes several things easily obtained. First, the queue depths of both the remote process and those generated by messages from the application module itself are easily distinguished from each other. Since applications only listen to one queue at a time it does require the deployment of another MDB to catch those messages.


Easier Separation and Tuning

Figure 2 can easily have the 2nd deployment of the application MDB occur on separate hardware. This allows for handy separation should any capacity increases occur. It also allows for separately tuning the JVM to the specific characteristics of the application's function if the MDBs are deployed to separate JVMs. It is not unusual to have different MDBs exhibit different performance characteristics because they have wildly different function.

There is a third option sort of a cross between Figure 1 and 2 where the B deployment remains in a single JVM but uses a 2nd queue to separate the messages. This may also be considered for MDBs that co-exist peacefully in the same JVM.


A Similar Problem

This same issue that sometimes “less does not equal more” comes up again in the case of mediations in an ESB. Consider the following problem: We have two SOAP messages in our application. One needs to be translated from one XML schema to another between the service requestor and the service provider. The other one, however, does not need translation.

In this case the simplest solution would be to place a mediation on the first message, but not on the second one. In fact, you might go so far as to not even use an ESB for the second message since there's no obvious need for mediations. However, using the ESB and perhaps adding a mediation in the second case, even if it does not do any translation is still valuable.

Consider the issue of problem determination of the two messages; in the first case, it's simple to log the message that has been translated both on its incoming (untranslated) and outgoing (translated) versions. In the second case where the message does not go through the ESB, problem determination is more difficult since you cannot easily examine the message as it flows from requestor to provider. So, routing the message through the ESB allows us to add a simple logging mediation when it is needed. Even though you've added some complexity at deployment time, the benefits gained in problem determination make it worthwhile.


Conclusion

While a runtime environment can seem to have a simple configuration it may impede the ability to properly conduct problem determination and/or performance tuning. This is a reason to always be looking from a different angle at alternative architectures that provide defensive capabilities in terms of how the environment can be monitored, controlled and performance tuned. Commonly the performance test environment tends to be where such defensive architectures readily become apparent.


Resources

About the authors

Kyle Brown

Kyle Brown is a Distinguished Engineer with IBM Software Services for WebSphere specializing in SOA and Emerging Technologies. Kyle provides consulting services, education, and mentoring on SOA, object-oriented topics and J2EE technologies to Fortune 500 clients. He is a co-author of Java Programming with IBM WebSphere and Persistence in the Enterprise. He is also a frequent conference speaker on the topics of SOA, Enterprise Java, OO design, and design patterns.

Alexandre Polozoff

Alexandre Polozoff is an IBM Master Inventor and a Senior Certified IT Specialist in the Software Group Performance Technology Practice across the WebSphere product stacks (Application Server, Portal, Process Server, etc) at large topology, international customer sites. He has authored numerous papers and blogs related to performance problem determination around J2EE technologies and speaks at various conferences.

Comments (Undergoing maintenance)



Trademarks  |  My developerWorks terms and conditions

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=SOA and Web services
ArticleID=412936
ArticleTitle=Problem Determination through Defensive Infrastructure Architecture: Part 1: Runtime deployments with queues and ESBs
publish-date=07142009
author1-email=brownkyl@us.ibm.com
author1-email-cc=
author2-email=polozoff@us.ibm.com
author2-email-cc=

My developerWorks community

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).

My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Rate a product. Write a review.

Special offers