TCP/IP Storage Increase

Body

It's looking like I'm seeing some sort of storage creep, and it seems to be related to z/OS CommServer TCP/IP.  Sometimes I'm not quite sure how to approach identifying if the storage increase is related to TCP/IP, or even how to get to the root cause. Over time, I've learned it's helpful to use some simple commands to gain a general knowledge of my system's TCP/IP storage use. This has allowed me to not only protect my system, but also to quickly get to the bottom of any issues.

There are several commands I use to periodically monitor and collect information regarding TCP/IP's storage usage for current use, high-water mark, and limit (if I have it configured). Analyzing the data, I have learned over time what's typical for TCP/IP storage use during normal and acceptable peak workloads. My automation issues these commands every 15 minutes so they are recorded in the system log. This way there's historical information to pinpoint a problem area and time frame should TCP/IP's storage usage appear to be abnormal.

• D TCPIP,,STOR - issue this command for each of your TCP/IP stacks   
• D NET,CSM - this command can be used to determine overall CSM ECSA and CSM Fixed storage utilization
• D NET,CSM,OWNERID=ALL - use this command to identify what application is using CSM storage and how much it is using

TCP/IP Storage - 

D TCPIP,[stackname],STOR

d tcpip,,stor
TCPIP STORAGE                                                   
    TCPCS    STORAGE            CURRENT   MAXIMUM     LIMIT     
    31-BIT                                                          
         ECSA                     2858K     3313K   NOLIMIT     
         PRIVATE                  8631K     8634K   NOLIMIT     
         ECSA MODULES             9671K     9671K   NOLIMIT     
    64-BIT                                                          
         HVCOMMON                    1M        1M   NOLIMIT     
         HVPRIVATE                   1M        1M   NOLIMIT     
         TRACE HVCOMMON           2579M     2579M     2579M   

• Limits for ECSA and Private storage can optionally be configured in the TCP/IP Profile (GLOBALCONFIG statement, parameters ECSALIMIT and POOLLIMIT).
• There are no recommended limits that will work for every system because every system is different.
• This output does not include CSM storage.

ECSA storage - The use of common storage can be controlled with the TCP/IP GLOBALCONFIG profile statement - ECSALIMIT.  ECSALIMIT limits can be set to keep TCP/IP from monopolizing all common storage on a system. This is to protect other subsystems' ability to access common storage in the event TCP/IP hits a situation where it consumes too much ECSA. This parameter is intended to improve system reliability by limiting TCP/IP's common storage use.  

Private storage - The amount of storage TCP/IP uses in its user region. There are several ways in which you can limit TCP/IP's use of private storage:

• with the TCP/IP GLOBALCONFIG profile statement - POOLLIMIT
• via the REGION keyword in TCP/IP's start up JCL. This can also be overridden by installation exits such as IEFUSI.  If you choose to limit the region size you should also set a POOLLIMIT in the profile.

ECSA modules - This is common storage used by TCP/IP load modules.
64-Bit common area size
64-Bit private area size
Trace HVCommon - 64-Bit common storage used for tracing

Here are some considerations you should make when choosing to define ECSA  or private (pool) storage limits.   

• Accommodate for temporary application "hang" conditions, where TCP/IP must buffer large amounts of inbound or outbound data. Add a reasonable fudge factor to the observed maximum usage values. It is not uncommon to set limits that are 50% over the peak usage.
• Care should be taken when coding the ECSALIMIT parameter. Setting it too low can cause TCP/IP to terminate prematurely.
• The benefit of specifying limits is that you will receive warning messages before storage obtain calls start failing when there is not enough storage available to satisfy the requests.
• ECSALIMIT does not include any of your CSM storage used by TCP/IP.
• When choosing to limit Private storage, make sure you don't use a value that is lower than or equal to what your installation exit (IEFUSI) enforces. 
• Remember that the values set for ECSALIMIT and POOLLIMIT can be changed via OBEYFILE command processing (VARY TCPIP,,OBEYFILE).

CSM storage - The Communications Storage Manger (CSM) is a VTAM component that allows authorized host applications to share data with VTAM, TCP/IP, and other CSM users without the need to physically copy the data.  

• Your CSM storage will be located in either ECSA or data space storage, and can be fixed or pageable.
• CSM storage definitions are controlled by SYS1.PARMLIB member IVTPRM00 which is read by VTAM during initialization.  
• The limits you set can be dynamically changed with a MODIFY CSM command. This allows you to control the amount of CSM storage that can be used in ECSA or can be  FIXED at any point in time.

D NET,CSM - This command provides a quick overview of how much storage has been allocated by CSM, and how much of it is in-use or free for use by a CSM user. You'll find that CSM can be in either ECSA or data space storage. The command output also lets you know what you have defined as the maximum in your IVTPRM00 parmlib member.

  D NET,CSM
BUFFER BUFFER                                              
SIZE   SOURCE                INUSE      FREE     TOTAL     
------------------------------------------------------     
   4K  ECSA                   144K      112K      256K     
  16K  ECSA                    16K      240K      256K     
  32K  ECSA                     0M      512K      512K     
  60K  ECSA                     0M        0M        0M     
 180K  ECSA                     0M      360K      360K     
TOTAL  ECSA                   160K     1224K     1384K     
------------------------------------------------------     
   4K  DATA SPACE 31            0M      256K      256K     
  16K  DATA SPACE 31            0M        0M        0M     
  32K  DATA SPACE 31            0M        0M        0M     
  60K  DATA SPACE 31            0M        0M        0M     
 180K  DATA SPACE 31            0M        0M        0M     
TOTAL  DATA SPACE 31            0M      256K      256K     
------------------------------------------------------     
   4K  DATA SPACE 64         4352K      128K     4480K     
  16K  DATA SPACE 64            0M      256K      256K     
  32K  DATA SPACE 64           96K      416K      512K  
  60K  DATA SPACE 64            0M        0M        0M  
 180K  DATA SPACE 64            0M      360K      360K  
TOTAL  DATA SPACE 64         4448K     1160K     5608K  
------------------------------------------------------  
TOTAL  DATA SPACE            4448K     1416K     5864K  
------------------------------------------------------  
TOTAL  ALL SOURCES           4608K     2640K     7248K  
FIXED  MAXIMUM =      120M  FIXED  CURRENT =     6877K  
FIXED  MAXIMUM USED =     6877K SINCE LAST DISPLAY CSM  
FIXED  MAXIMUM USED =     6877K SINCE IPL               
ECSA   MAXIMUM =      120M  ECSA   CURRENT =     1633K  
ECSA   MAXIMUM USED =     1633K SINCE LAST DISPLAY CSM  
ECSA   MAXIMUM USED =     1633K SINCE IPL               
CSM DATA SPACE 1 NAME: CSM64001                         
CSM DATA SPACE 2 NAME: CSM31002                 

D NET,CSM,OWNERID=ALL - Use this command to see how much CSM storage each of the CSM 'users' are currently using. If you want to see only the CSM usage of TCP/IP,  you can also specify the user by "OWNERID=TCP ASID". (An example of OWNERID command output is not shown.)  

Considerations to make when choosing to set your CSM limits:

• ECSA CSM can't be larger than your system ECSA limit which you defined in your system parameters in parmlib member IEASYSnn (CSA).
• When setting FIXED CSM, ensure that you have enough real frames to back the FIXED allocation.

So, now that you've learned to monitor TCP/IP's ECSA, private, and CSM storage usage, you may be thinking "what's next" if you think you're seeing a TCP/IP storage-related problem.  If no SVC dumps are generated for the issue, when you see that storage use is on the rise, take a console dump of the TCP/IP address space.  If you think the problem is related to your CSM storage usage by TCP/IP,  include the CSM dataspace in the dump. Here's a sample console dump command you can use:

  DUMP COMM=(tcpip storage growth)
    R xx,JOBNAME=(tcpipprocname),CONT
    R xx,SDATA=(NUC,RGN,CSA,SQA,LSQA,TRT),CONT
    R xx,DSPNAME=(1.CSM*),END

If needed, the IBM Support Center can assist you with identifying the cause of your TCP/IP storage growth.