WebSphere Peformance - Alexandre Polozoff's Point of View

Performance Tuning - 101
The past few weeks meeting with various WebSphere Application Server-based customers reminded me of the importance of the basic and fundamental performance tuning tasks. The InfoCenter provides information on tunings at the OS level, TCP/IP, JVM, etc. I have visited no less than 3 different environments running WebSphere Application Server without these base tunings. Just by applying the base tunings to the OS and JVM we saw as much as 99% less garbage collection, improved response time, throughput and less CPU utilization with the same production loads. The best part of following these instructions is the administrator does not need to be a performance guru to realize these gains. These improvements also help save money requiring less capacity going forward.
In the "Tuning the JVM" section I have never been disappointed with the "Option 1" settings. Options 2 and 3 require the ability to place the application under load/stress test. If you do not have a load/stress test environment (i.e. you have to test in production) then stick with "Option 1". Notice that "Tuning Performance" has several sections for both application developers and WebSphere administrators. This is because we all know that to realize the best performance gains one has to optimize the application code. Runtime tuning can realize 5-15% but application code improvements can see 300% and higher performance improvements.
|
Windows slow TCP/IP "connection refused" means fail over performance is bad
As I travel the world working performance problems I never see Microsoft Windows environments used outside the developer's desktop. Surprisingly these past couple of weeks I've been working in an environment where Microsoft Windows is used for the IBM HTTP Server tier with the WebSphere Application Server plug-in. Under normal operating conditions everything seems to work nominally.
However, much to my surprise, if we took down any of the application servers in the cluster of this very large cell I saw an anomaly. When the plug-in was attempting to route traffic to the downed application servers there seemed to be a really long lag on the connection refused processing. In fact, I was seeing least a second to get through the TCP/IP roundtrip. This made no sense to me. One of my colleagues, Keys Botzum, took a Java application and ran it on both Windows and Solaris. The application simply tried to connect to localhost (to eliminate any DNS lookups or network latency from the test) on a port no one was listening to and looped around 20 times. On Windows the test took slightly over 20 seconds. On Solaris, less than a second (which was the behaviour I was expecting on Windows). If you are, or planning to, use Microsoft Windows on the IHS tier be aware of this strange failure scenario on Windows. I'll try to investigate and see if there are any Windows settings to help tune this. Though the plan is to move off Windows to Redhat Linux which right now sounds like the right move to me.
|
Maximo mustgather to follow for performance or stability problems
https://www-304.ibm.com/support/docview.wss?uid=swg21313341
This is the page to follow if there seem to be any Maximo performance or stability problems.
|
Client side performance measurements
http://www.alphaworks.ibm.com/tech/pagedetailer
IBM provides a tool to help with analysis of client side performance. One of the benefits for doing this is to help identify static objects that are not being cached at the browser. In addition, for JSF based applications, this tool helps identify large client side caches.
|
Metrics to monitor
I am often asked specifically what metrics should be monitored in WebSphere Application Server. This list can be considered a starting point. The WAS admin will frequently use "Custom" PMI setting and disable any metrics not in this list.
Thread pools Suggested: Pool Size, Active Thread Count Optional: ActiveTime (avg. time in use), ConcurrentHungThreadCount. For WESB, per-mediation: Mediation.ThreadCount
Connection pools: Database - JDBC Data Source Connection Pools. Messaging - JMS Queue Connection Factory Connection Pools. Suggested: PoolSize, WaitingThreadCount, WaitTime,
Optional: UseTime.
Messaging Engines Suggested: BufferedReadBytesCount, BufferedWriteBytesCount
Queues Suggested: AvailableMessageCount, LocalMessageWaitTime Transactions (per application server)
JVM Memory Management. (Java GC). Suggested: Heap Size, % Free after GC, % Time spent in GC. CPU:
|
April 11, 2011 10:02 PM
On Solaris this tuning page has the details that I often follow for Option 1.
|