Using SUBSTATE to troubleshoot channels

If your channel is running slowly or appears to have no messages to send, use the SUBSTATE field to help resolve the problem.

Environment

Diagram representing the message flow between a local and remote queue manager.

Diagnosing the problem

Sender channel status

CHSTATUS(MQ23.TO.MQ24)  XMITQ(TO.MQ24)    CONNAME(127.0.0.1)      
CURRENT     CHLTYPE(SDR)    STATUS(RUNNING)  SUBSTATE(MQGET)        
RQMNAME(MQ24)

Receiver channel status

CHSTATUS(MQ23.TO.MQ24)      CONNAME(127.0.0.1)      CURRENT     
CHLTYPE(RCVR)           STATUS(RUNNING)     SUBSTATE(RECEIVE)       
RQMNAME(MQ23)

If you see the channel in this state, the channel behaves as it has no messages to send. As an aside, this can be a useful check to make at the same time as checking for uncommitted messages on the transmission queue, when you suggest that the channel is not moving the messages just put.

It is worth noting at this point that the resting state of a server-connection channel is similar to a receiver channel as it spends its time sitting in a network receive waiting for the client to send it an API call to issue.

Slow channels

The sub status of channels is also useful, when other evidence suggests that your channel is running really slowly. Having described the resting state of channels in the previous section, if you see the sender, rather than the receiver channel, is sitting in a network receive, this can indicate that the line turn around on the network is slow.

This is because the sender channel is waiting on the receiver to process all the messages that have been sent in the batch and then respond to the End of Batch (EoB in the following diagram) notification to say the batch has been successfully completed. Diagram representing the sender channel waiting on the receiver to process all the messages.

This state happens at the end of every batch, but it is usually so fleeting, that you are unlikely to see it. If you see it for a prolonged period of time it can be indicative of major network issues, such as:

Router problems
Retransmission of dropped packets
Other issues that cause the network to run slow
Something else that causes the receiving end to slow down, such as message retry, or issues in a exit

At this point looking at how the receiver channel is doing is the next step.

Hung channels

When a channel is hung, either stuck in BINDING state for a long time, or even hung in RUNNING state where fields such as Number of Bytes Sent (NUMBYTES) is not increasing, then sub status can help.

When a channel, whether a queue manager channel or a server-connection channel, is starting up there are a number of different operations that it has to do before it can be considered to be RUNNING. All these different operations happen while the channel is in BINDING state.

To help break that down into a more granular view of what the channel is doing, there are sub status values for all these major operations. Some of the more common ones you see are listed below:

Connecting to the network: When there are problems reaching the host, perhaps because the IP address is not longer in service, this can take a very noticeable number of seconds to timeout and return an error. Until it does, the channel shows SUBSTATE(NETCONNECT).
Making a Domain Name Server (DNS) request: If the local DNS doesn't know how to resolve your CONNAME, it may be able to ask other related DNS (if your system is configured this way). This can all take considerable time. While this is happening, the channel shows SUBSTATE(NAMESERVER).
SSL/TLS Handshake: When using SSL or TLS, by specifying a cipher spec on a channel definition in the SSLCIPH field, this means that the channel must complete an SSL handshake before any other data can flow across the channel. While this is happening, the channel shows SUBSTATE(SSLHANDSHAKE).
In an exit: There are various different exits than can be used on a channel. While the exit is processing, the channel will show a sub state reflecting which exit it is in, for example, SUBSTATE(SENDEXIT). If the exit is taking a long time then you may see this for more than a fleeting moment.

Resolving the problem

See DISPLAY CHSTATUS for a summary of all the sub states and when they can be seen.