Monitoring application balancing

You can use the DISPLAY APSTATUS command to monitor the state of application balancing across a uniform cluster, and to investigate why the application is not balanced if that is unexpected.

Monitoring the current state of applications across queue managers in a cluster

From any queue manager in a uniform cluster you can get an overview of the current state of applications across all the queue managers of a cluster by running the DIS APSTATUS command.

[MQ 9.2.0 Jul 2020] From IBM® MQ 9.2.0 the TYPE field is also displayed in the output.

For example, after a queue manager has just started you might see output like the following:

 
    1 : DIS APSTATUS(*) type(APPL) 
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         CLUSTER(UNIDEMO)
   COUNT(8)                                MOVCOUNT(8) 
   BALANCED(UNKNOWN)
   TYPE (APPL)

This shows that there is an application called MYAPP in the uniform cluster, and currently there are eight instances, all eight of which are considered movable around the uniform cluster. The balanced value of UNKNOWN is a temporary value, which indicates that the queue manager has not yet been though an attempt to rebalance the application if necessary.

After a short time, you are more likely to see the following output:

 
    1 : DIS APSTATUS(*) type(APPL) 
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         CLUSTER(UNIDEMO)
   COUNT(8)                                MOVCOUNT(8) 
   BALANCED(NO)
   TYPE (APPL)

This output shows that the application has eight instances, but that they are not balanced across the uniform cluster. At this point, it is worth looking at the distribution of applications in the cluster.

To do this, run the DIS APSTATUS command again. Note that you can run this command against any queue manager in the uniform cluster:

     
1 : DIS APSTATUS(*) type(QMGR) 
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         ACTIVE(YES)
   COUNT(6)                                MOVCOUNT(6) 
   BALSTATE(HIGH)                          LMSGDATE(2019-05-24)
   LMSGTIME(13:11:10)                      QMNAME(UNID001)
   QMID(UNID001_2019-05-24_13.09.35)    
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         ACTIVE(YES)
   COUNT(1)                                MOVCOUNT(1) 
   BALSTATE(LOW)                           LMSGDATE(2019-05-24)
   LMSGTIME(13:11:03)                      QMNAME(UNID002)
   QMID(UNID002_2019-05-24_13.09.39)    
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         ACTIVE(YES)
   COUNT(1)                                MOVCOUNT(1) 
   BALSTATE(LOW)                           LMSGDATE(2019-05-24)
   LMSGTIME(13:11:07)                      QMNAME(UNID003)
   QMID(UNID003_2019-05-24_13.09.43)
   TYPE (QMGR)

From this, you can see at this point in time that queue manager UNID001 has six instances, but queue managers UNID0002 and UNID0003 only have one each. The BALSTATE output gives an indication of the balance state the last time the queue manager reported it. However, be aware that the instance count might be more recent than the BALSTATE field.

This output also gives a good indication that the uniform cluster is distributing information about this application. The application is known to all queue manager instances in the uniform cluster, and the last message date and time is very recent.

Also, the ACTIVE field indicates that all queue managers in the cluster are considered to be communicating with one another; if ACTIVE is set to NO on any queue manager, it indicates that regular communication with it has been interrupted.

If left to rebalance itself, the output would eventually end up something like:

 
    1 : DIS APSTATUS(*) type(QMGR) 
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         ACTIVE(YES)
   COUNT(3)                                MOVCOUNT(3) 
   BALSTATE(OK)                            LMSGDATE(2019-05-24)
   LMSGTIME(13:14:22)                      QMNAME(UNID001)
   QMID(UNID001_2019-05-24_13.09.35)    
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         ACTIVE(YES)
   COUNT(3)                                MOVCOUNT(3) 
   BALSTATE(OK)                            LMSGDATE(2019-05-24)
   LMSGTIME(13:13:53)                      QMNAME(UNID002)
   QMID(UNID002_2019-05-24_13.09.39)    
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                         ACTIVE(YES)
   COUNT(2)                                MOVCOUNT(2) 
   BALSTATE(OK)                            LMSGDATE(2019-05-24)
   LMSGTIME(13:13:47)                      QMNAME(UNID003)
   QMID(UNID003_2019-05-24_13.09.43)
   TYPE (QMGR)

The BALSTATE of the queue managers is now OK indicating that a steady state has been achieved.

Monitoring individual application instances

You can look at the individual application instances, but you do this for each queue manager. Looking at the output on UNID001:

    
 1 : DIS APSTATUS(*) type(LOCAL) 
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                      
   CONNTAG(MQCT02DFE75C02EA0A20UNID001_2019-05-24_13.09.35MYAPP)
   CONNS(1)                                IMMREASN(NONE)
   IMMCOUNT(0)                             IMMDATE( )
   IMMTIME( )                              MOVABLE(YES)
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                      
   CONNTAG(MQCT02DFE75C02E50A20UNID001_2019-05-24_13.09.35MYAPP)
   CONNS(1)                                IMMREASN(NONE)
   IMMCOUNT(0)                             IMMDATE( )
   IMMTIME( )                              MOVABLE(YES)
AMQ8932I: Display application status details.
   APPLNAME(MYAPP)                      
   CONNTAG(MQCT02DFE75C02E60A20UNID001_2019-05-24_13.09.35MYAPP)
   CONNS(1)                                IMMREASN(NONE)
   IMMCOUNT(0)                             IMMDATE( )
   IMMTIME( )                              MOVABLE(YES)
   TYPE (LOCAL)
One MQSC command read.

The presence of MOVABLE(YES) indicates this instance can be moved if needed to another queue manager in the cluster. In the following example, the application instance is not movable because it its not connecting as a client:

 
    3 : DISPLAY APSTATUS('ServerApp') TYPE(LOCAL)
AMQ8932I: Display application status details.
   APPLNAME(ServerApp)
   CONNTAG(MQCT02DFE75C01800B20UNID001_2019-05-24_13.09.35ServerApp)
   CONNS(1)                                IMMREASN(NOTCLIENT)
   IMMCOUNT(0)                             IMMDATE( )
   IMMTIME( )                              MOVABLE(NO)
   TYPE (LOCAL)

You can use the CONNTAG field to see the individual queue manager connections from that instance, which can be useful if the application instance is IMMREASN(NOTRECONN). In the following example, the client application is not movable, and investigating the connect options shows it does not have MQCNO_RECONNECT in the CONNOPTS field.


     1 : DISPLAY APSTATUS('ClientApp') TYPE(LOCAL)
AMQ8932I: Display application status details.
   APPLNAME(ClientApp)
   CONNTAG(MQCT02DFE75C01CB0B20UNID001_2019-05-24_13.09.35ClientApp)
   CONNS(1)                                IMMREASN(NOTRECONN)
   IMMCOUNT(0)                             IMMDATE( )
   IMMTIME( )                              MOVABLE(NO)
   TYPE (LOCAL) 

     2 : DISPLAY CONN(*) TYPE(CONN) WHERE(CONNTAG eq 'MQCT02DFE75C01CB0B20UNID001_2019-05-24_13.09.35ClientApp') ALL
AMQ8276I: Display Connection details.
   CONN(02DFE75C01CB0B20)
   EXTCONN(414D5143554E49443030312020202020)
   TYPE(CONN)
   PID(14656)                              TID(20)
   APPLDESC(IBM MQ Channel)                APPLTAG(ClientApp)
   APPLTYPE(USER)                          ASTATE(NONE)
   CHANNEL(SYSTEM.DEF.SVRCONN)             CLIENTID( )
   CONNAME(127.0.0.1)
   CONNOPTS(MQCNO_HANDLE_SHARE_BLOCK,MQCNO_SHARED_BINDING)
   USERID(MyUserid)                        UOWLOG( )
   UOWSTDA( )                              UOWSTTI( )
   UOWLOGDA( )                             UOWLOGTI( )
   URTYPE(QMGR)
   EXTURID(XA_FORMATID[] XA_GTRID[] XA_BQUAL[])
   QMURID(0.0)                             UOWSTATE(NONE)
   CONNTAG(MQCT02DFE75C01CB0B20UNID001_2019-05-24_13.09.35ClientApp)
   TYPE (CONN)