Troubleshooting WLM issues in WebSphere Application Server
Kumaran Nathan 1100005HAT Comments (2) Visits (16150)
Sometimes troubleshooting Work Load Management (WLM) issue in IBM WebSphere Application Server can be challenging. This blog helps address common issues with this component before calling IBM support and save you time.
1. How does WLM participate in load balancing EJB requests in at WebSphere Cluster?
For an Overview of WLM load balancing for EJB see: Clusters and workload management
2. My workload is not balanced. Why isn't WLM working?
WLM does not manage load, despite the name. WLM balances requests in the form of method calls/invocations. If the requests drive varying load on the servers, then you may correctly see that the load, as measured by CPU consumption for example, is not balanced. The "pattern problem" occurs when you have an even number of members and an even number of method calls such as "create" and "invoke". For example, with 2 members the pattern could be that all the lightweight create requests execute on one server, and the heavyweight invoke requests end up on the other server. In that case the "load" on the servers (measured in CPU utilization) is not equal among the servers.
A workaround to this problem is adjusting the weights of the cluster to non-equal values. A typical recommendation for normalization are cluster weights of 19 and 23.
If you are observing uneven load distribution among an even number of cluster members, it is recommended to switch to an odd number of members. Note, an odd number of cluster members can also experience pattern problems based on the number of method calls.
3. How do you check the routing patterns for EJB calls in the trace when customer says their standalone EJB client is not balancing requests properly?
Some people use a stand alone EJB client and wants the EJB requests to a cluster from the client to be load balanced. Usually the client does a call to the cluster to get the EJB Home for the EJB first and then will make the actual call. The call to get the EJB_HOME is counted as one batch of work in WLM and the actual service call is counted as the 2nd batch of work. So if the cluster has 2 members, quite often the routing pattern will be such that all the calls to get the EJB_HOME end up in one member and the actual call will end up in the 2nd member. Some people don't like this since from their view point it looks like things are not balanced since all the requests end up in the same server. This is just how it's designed.
To verify this is what is happening in the WLM trace, look for patterns of p1=operation=c and which will shortly be followed by a getConnection call. This should tell us what the operation is and which server handled it.
and so on ....
4. I see error: CORBA.NO_IMPLEMENT: No Cluster Data Available, with WLML
If there are multiple core groups, make sure they are bridged. Verify the core groups are bridged. Check the basics. There is a special cluster called the LSD Cluster which the Node Agents use to get overall cell cluster information. If the core groups are not bridged then the LSD won't contain information about all the nodes and clusters. This is an example stack trace:
>> at com.
>> at com.
>> at com.
>> at com.
>> at com.
>> at com.
Additional Info : This message is seen in the Nodeagent. When a client makes the first request to a cluster, WLM plugin has no information yet about the target cluster members in order to do routing in an attempt to route the request, it sends it to node agents in the target cell. The node agents are expected to have data about the clusters which they can use to forward the request to a cluster member.
5. What config files describe how my cluster members are balanced and what do I look for?
Look at cell
<?xml version="1.0" encoding="UTF-8"?>
See also: My workload is not balance(Q2) , why?
6. What is the impact of running ORB.init() on WLM?
WebSphere only supports running with a single copy of the ORB as documented here: Object Request Brokers
You may see a Forward Limit reached exception:
Caused by: org.
Each orb.init causes an instance of the WLM interceptor to be registered.
NO_IMPLEMENT exception means that a requested object could not be located. For example, a NO_IMPLEMENT error is raised when a server does not exist or is not running when a clients initiates a request. Creating multiple instances of ORB can also cause this issue.
This can be easily identified using a javacore or thread dump.
This is what an ORB reader thread looks like in a javacore:
This is what an ORB listener thread id looks like in a javacore:
The O= value is the ORB instance id.
7. Why I am getting NoAv
Applicable Targets 
The above message informs you that WLM doesn't know about endpoints and it can select to route the request to a target memb
Bridge coregroups on the server side if the nodeagent is not part of the cluster member coregroup.
8. How are cluster members with a weight of 0 treated?
To make a member unavailable for load without stopping the member, the cluster weight can be set to zero. WLM will then route around this member and distribute load to the remaining cluster members. However if all cluster members have a weight of 0 then WLM will balance load among all cluster members equally.
9. Why does cell name need to be unique?
WLM makes routing decisions on the client side, using information gathered from the servers. Routing information is organized by cell
When creating an InitialContext, its important to connect to the proper JNDI NameSpace. If the wrong namespace is used you can run into problems later when trying to locate EJB's. The java
We recommend that you bootstrap to a list of servers in the cluster. Do NOT bootstrap to a NodeAgent.
NEVER set the java
Provider URL Corbaloc is processed by JNDI / naming, not WLM. Naming makes a request to each host (in indeterminate order) until it successfully connects with the name service on a host. After the EJB IOR is retrieved from the name service, WLM uses its cluster information to route requests. The original Corbaloc string is not involved in EJB routing.
Big Thanks to WLM Developer Tom Seelbach who reviewed and provided most of the technical content of this blog.