Global TCP stall detail (-G -D) report

This report is displayed when both the -G and -D options are specified on the trmdstat command. It displays the contents of individual global TCP stall event records. The information presented in this report is derived from EZZ8671I, EZZ8672I, EZZ8673I, and EZZ8674I types of syslog messages. In the Connections Reset and Connections Would Have Been Reset sections, information is grouped and sorted by remote IP address.

>trmdstat -GD /tmp/tstlog.log
trmdstat for z/OS CS V2R1  Tue Dec  6 13:16:51 2011                                                   
                                                                                                       
Command Entered     : trmdstat -GD /tmp/tstlog.log                                       
Log Time Interval   : Oct 29 18:02:33  - Oct 29 18:53:33                                               
Stack Time Interval : Oct 29 18:02:19  - Oct 29 18:53:21                                               
TRM Records Scanned : 504                                                                              
                                                                                                       
                             Global TCP Stall Events                                                   
                                                                                                       
                                                   Small    Write                                      
                                Stalled            Window   Block                                      
    Date and Time        Type   Percent TotalConns Percent Percent  Duration  Correlator   Action      
---------------------- -------- ------- ---------- ------- ------- ---------- ---------- -----------   
10/29/2011 18:02:19.08 Enter        50%       1004     49%      0%                     3 noresetconn   
10/29/2011 18:53:21.22 Exit         25%       2008     25%      0%       2920          3 noresetconn   
                                                                                                       
                             Connections Reset                                                         
                                                                                                       
No records to display                                                                                  
                                                                                                       
                             Connections Would Have Been Reset                                         
                                                                                                       
Remote IP Address: 10.11.2.1                                                                           
                                                                                                           
                       Local                                         Remote ConnID/  SendQSize/            
    Date and Time      Port               Local IP Address           Port   JobName  WindowSize Correlator 
---------------------- ----- --------------------------------------- -----  -------- ---------- ---------- 
10/29/2011 18:02:19.08 20000 10.11.1.2                                1119  00000091       8000          3
                                                                            USER13            0           
10/29/2011 18:02:19.08 20000 10.11.1.2                                1140  000000A5      20000          3
                                                                            USER13            0           
                                                                                                           
Remote IP Address: 10.12.2.1                                                                               
                                                                                                           
                       Local                                         Remote ConnID/  SendQSize/            
    Date and Time      Port               Local IP Address           Port   JobName  WindowSize Correlator 
---------------------- ----- --------------------------------------- -----  -------- ---------- ---------- 
10/29/2011 18:02:19.08 20000 10.12.1.2                                1117  0000008F      16000          3 
                                                                            USER13            0            
10/29/2011 18:02:19.08 20000 10.12.1.2                                1513  0000021C       9500          3 
                                                                            USER13            0            
                                                                                                          
Remote IP Address: 2001:db8:10::11:2:1                                                                    
                                                                                                          
                       Local                                         Remote ConnID/  SendQSize/           
    Date and Time      Port               Local IP Address           Port   JobName  WindowSize Correlator
---------------------- ----- --------------------------------------- -----  -------- ---------- ----------
10/29/2011 18:02:19.09 25000 2001:db8:10::11:1:2                      1456  000001EA      10000          3
                                                                            USER22            0           
10/29/2011 18:02:19.09 25000 2001:db8:10::11:1:2                      1352  00000182       8000          3
                                                                            USER22            0           

                                                                                                          
Remote IP Address: 2001:db8:10::12:2:1                                                                    
                                                                                                          
                       Local                                         Remote ConnID/  SendQSize/           
    Date and Time      Port               Local IP Address           Port   JobName  WindowSize Correlator
---------------------- ----- --------------------------------------- -----  -------- ---------- ----------
10/29/2011 18:02:19.09 25000 2001:db8:10::12:1:2                      1310  00000158      20000          3
                                                                            USER22            0           
10/29/2011 18:02:19.09 25000 2001:db8:10::12:1:2                      1102  00000088      16000          3
                                                                            USER22            0

The following information describes the areas of the global TCP stall event detail report:

Date and Time

The stack date and time when the event occurred.

Type

Enter: A global TCP stall condition was entered
Exit: A global TCP stall condition was exited.
ExitPlcy: A global TCP stall condition was exited because IDS Global TCP Stall policy was no longer in effect.

Stalled Percent

The percentage of the active TCP connections that were stalled. A TCP connection is considered stalled if one or more of the following conditions are true:

The TCP send window size is less than 256 or is less than the smaller of the largest send window that has been seen for the connection and the default MTU. The TCP send window size is set based on values provided by the TCP peer. The default MTU for IPv4 is 576. The default MTU for IPv6 is 1280.
The TCP send queue is full and the data is not being retransmitted.

TotalConns

The total number of active TCP connections.

Small Window Percent

The percentage of the active TCP connections that were stalled because the TCP send window size is less than the smaller of the MSS of the connection and the default MTU. A TCP connection can be stalled due to multiple conditions. For example, a TCP connection might be included in both the Small Window Percent value and the Write Block Percent value.

Write Block Percent

The percentage of the active TCP connections that were stalled because the TCP send queue is full and the data is not being retransmitted. A TCP connection can be stalled due to multiple conditions. For example, a TCP connection might be included in both the Small Window Percent value and the Write Block Percent value.

Duration

The duration, in seconds, of the global TCP stall. Present only for Exit and ExitPlcy records.

Correlator

The correlator for a global TCP stall condition. The correlator can be used to correlate global TCP stall enter and exit conditions with individual TCP connections that contributed to the condition. The individual connections are reset or would have been reset.

Action

The action specified in the IDS policy for the global TCP stall attack type. The action value can be resetconn or noresetconn.

Results:

If the value is resetconn, all stalled TCP connections were reset. If you requested detailed syslogd messages for the global TCP stall attack type in the IDS policy, the Connections Reset section contains an entry for each stalled connection that was reset during the global TCP stall attack.
If the value is noresetconn, stalled TCP connections were not reset. However, if you requested detailed syslogd messages for the global TCP stall attack type in the IDS policy, the Connection Would Have Been Reset section contains an entry for each connection that was stalled at the time that the global TCP stall was detected.

Remote IP Address

The remote IP address of the stalled TCP connections described in the table.

Local Port

The local port number of the stalled TCP connection that was reset or would have been reset.

Local IP Address

The local IP address of the stalled TCP connection that was reset or would have been reset.

Remote Port

The remote port number of the stalled TCP connection that was reset or would have been reset.

ConnID

The ID of the TCP connection that was reset or would have been reset.

JobName

The job name of the TCP connection that was reset or would have been reset.

SendQSize

The amount of data queued to send queue for the stalled TCP connection.

WindowSize

The size of the send window for the stalled TCP connection.

messages suppressed

The number of attack messages suppressed with attack type, date and time. This data comes from an EZZ9327I message. See in The trmdstat report general concept for a detailed description.