IBM Support

MustGather: Collect Troubleshooting Data / Read First / Fix List for TCPIP CSM Storage
for z/OS Communications Server

Troubleshooting


Problem

This document contains information on debugging CSM related problems for IP on z/OS 1.4 and higher. This is a replacement for Information APAR II13951.

Resolving The Problem

Table of Contents


General Information
Gathering Documentation
Recommended Maintenance
Additional Information






General Information

PARMLIB MEMBER IVTPRM00 should initially be coded at 100M MAX ECSA and 100M MAX FIXED. Monitor system for one week with DISPLAY CSM command to determine peak usage. Adjust IVTPRM00 MAX ECSA and MAX FIXED values to 1.5 times the highest value indicated in DISPLAY CSM output.

NOTES:
  • When coding IVTPRM00, FIXED MAX and ECSA MAX values must start in column one and only one space is allowed between fields.
  • If CSM MAX ECSA is coded higher than 90% of the z/OS allowable ECSA (IEASYS00 ECSA) then CSM MAX ECSA will be adjusted to 90% of z/OS ECSA.
  • IEASYSxx CSA parm must be adjusted to include the additional amounts of ECSA that CSM will be using.
  • A shortage of CSM FIXED storage may result in ENOBUFS errors on a variety of TCP/IP Socket calls. The system programmer should consider increasing CSM MAX FIXED if ENOBUFS errors are reported by sockets applications running on the z/OS image.
  • To dynamically change the ECSA and FIXED CSM values use the following command:

  • MODIFY net,CSM,ECSA=value,FIXED=value
    Note:
    * net is your VTAM proc name
    * value for ECSA can be in K or M ranging from 1024K to 2048M
    * value for FIXED caan be in K or M ranging from 1024K to 30720M
    Guideline: CSM ECSA and CSM data space can be requested to be fixed. z/OS ensures the real frame that is backing the storage matches the CSM data space type of request. For example: CSM DSPACE64 buffers are backed with a real frame that is above the 2G line.
  • In z/OS® V1R13, the processing of IPAQENET and IPAQENET6 INTERFACE statements is enhanced. Coding the WORKLOADQ parameter on these INTERFACE statements now enables the QDIO inbound workload queueing function for Enterprise Extender (EE) traffic. An additional ancillary input queue (AIQ) is established for inbound traffic for EE if HPR=RTP is specified as a VTAM start option. Each AIQ increases storage utilization by an amount of 36K of fixed ECSA, plus potentially the READSTORAGE value (64K multiplied by the number of SBALs) of fixed CSM 4K data space storage. If you have configured QDIO inbound workload queuing, you must ensure that there is sufficient fixed ECSA and 4K CSM dataspace storage for the AIQ for EE traffic. For additional information regarding storage needs when using WORKLOADQ, see "IP Services: Ensure storage availability for ancillary input queue for Enterprise Extender Traffic" in z/OS Migration.
  • Back to Top




    Gathering Documentation (for releases V1R13 and above)

    If the maintenance listed in this document does not resolve the problem, the following documentation needs to be obtained:

    1. D net,CSM,OWNERID=ALL
      Determines how much storage is being used and by what job/ASID.

      Issue: D net,CSM without OWNERID to identify current and highwater values for FIXED and ECSA.

    2. If the DISPLAY CSM output indicates VTAM owns the storage:
      Issue: Modify net,TRACE,TYPE=VTAM,MODE=INT,SIZE=4M,
      OPTION=(CSM,CIO,SMS,PSS,PIU,MSG,SSCP,API,NRM,XBUF,APPC,CIA)

      NOTE: Do the following if using 8K VIT (MODE=EXT) buffering, as an alternative to specifying MODE=INT:
        • Specify BFRNUM to 10.
        • Ensure you only specify FE1,FEF,FF1 GTF options.
        • Ensure large enough BLOCKSIZE is specified for GTF
      (IEFRDER DD BLKSIZE defined to at least 8K-details in OW26052)


    3. If the DISPLAY CSM output indicates TCPIP owns the storage:
      TRACE CT,64M,COMP=SYSTCPIP,SUB=(TCPIP)
      R xx,options=(STORAGE,VTAM,VTAMDATA,MESSAGE,INTERNET,TCP),END


    4. Turn on CSM MONITOR function
    F net,CSM,MONITOR=YES

    If possible, let the monitor function run for a period of time during which storage growth occurs before taking the dump mentioned in step 5.

    5. For releases beginning with V1R13, dump VTAM, TCP/IP, and the CSM dataspaces using a dump command similar to the following:

    DUMP COMM=(csm storage growth)
    R xx,JOBNAME=(net,tcpipprocname),SDATA=(SQA,LSQA,CSA,RGN,TRT,NUC),CONT
    R xx,DSPNAME=(1.CSM*),END



    For releases prior to V1R13, dump VTAM, TCPIP, their dataspaces and the CSM dataspaces using a dump command similar to the following:
    DUMP COMM=(csm storage growth)
    R xx,JOBNAME=(net,tcpipprocname),SDATA=(SQA,LSQA,CSA,RGN,TRT,NUC),CONT
    R xx,DSPNAME=('net'.ISTITDS1,'tcpipprocname'.*,1.CSM*),END

    ADDITIONAL NOTE:
    Please reference technote # 1159164 for additional important information regarding a storage shortage when TCP devices have invalid MVS MIH values.       

Back to Top



Recommended Maintenance



TCP/IP z/OS 1.6 R160
APAR# PTF# PUT# Abstract
PQ87507 UQ88074 0405 TCPIP HUNG WHEN CSM CONSTRAINED
PQ87653 UQ88101 0406 EZBAFINB ABEND WHEN CSM CRITICAL
PQ88324 UQ88862 0406 ECSA GROWTH TERMINATING TN3270 SESSIONS
PQ89399 UQ89476 0406 STORAGE LEAK WHEN CSM IS CONSTRAINED
PQ91417 UQ89961 0408 CSM EXHAUSTED - LONG RUNNING CONNECTIONS
PQ90094 UQ90367 0408 60K CLAW PACKING AND SHORT PACKETS
PQ84529 UQ91651 0409 4K CSM LEAK W/ AUTHENTICATION & ENCRYPTION
PQ87325 UQ91529 0409 CSM 180K ECSA GROWTH DURING FTP
PQ98609 UK00169 0504 32K ECSA Outbound Packing Buffers
PK00576 UK00573 0503 EE Packets wait on Policy Lock
PK01098 UK01084 0504 231 key 6 Growth with DBCSTRANSFORM
PK03118 UK02560 0505 231 Key 6 and CSM ECSA 4K with IQDIO
PK04027 UK02523 0505 CSM 4K ECSA due to caching multicast ARP
PK07385 UK053050508 CSM FIXED SHORTAGE CAUSES OBEYFILE LOOP
PK09341 UK06733 0510 Serviceability APAR
PK16464 UK10588 0602 CSM growth using Firewall
PK46885 UK27411 0708 FIXED CSM growth with IUDRs and data

TCP/IP z/OS 1.6 (r169)

APAR# PTF# PUT# Abstract
PQ89399 UQ89477 0406 STORAGE LEAK WHEN CSM IS CONSTRAINED




TCP/IP z/OS 1.7 R170
APAR#PTF#PUT#Abstract
PK05314 UK03793 0506 231 key 6 Growth with DBCSTRANSFORM
PK06506 UK04299 0506 32K ECSA Outbound Packing Buffers
PK06529 UK04062 0506 231 Key 6 and CSM ECSA 4K with IQDIO
PK07385 UK05306 0508 CSM FIXED SHORTAGE CAUSES OBEYFILE LOOP
PK09341 UK06734 0510 Serviceability APAR
PK20978 UK13220 0604 Overlay - Record Structure FTP
PK32215UK192500611CSM 32K Dataspace buffer growth
PK44941 UK260990707 FIXED CSM growth with IUDRs and data
PK55824 UK31400 0712 CSM Dataspace and ECSA 4K buffers
PK61002 UK35461 0805 Consolidate Storage on Receive queue





TCP/IP z/OS 1.8 R180
APAR#PTF#PUT#Abstract
PK26464 UK15524 0606 180K ECSA Buffer overlaid x'FF' pattern
PK32215 UK19251 0611 CSM 32K Dataspace buffer growth
PK44941 UK26100 0707 FIXED CSM growth with IUDRs and data
PK55824 UK31401 0712 CSM Dataspace and ECSA 4K buffers
PK57383 UK32596 0802 CSM 4k ECSA Containing IUDR for IGMP
PK61002 UK35462 0805 Consolidate Storage on Receive queue
PK70053 UK39694 0810 Some CSM growth with Loop in Shareport
PK74282 UK41811 0812 CSM FIXED growth - Large # appl connections





TCP/IP z/OS 1.9 R190
APAR#PTF#PUT#Abstract
PK44941 UK26101 0707 FIXED CSM growth with IUDRs and data
PK55824 UK31402 0712 CSM Dataspace and ECSA 4K buffers
PK57383 UK32597 0802 CSM 4k ECSA Containing IUDR for IGMP
PK61002 UK35463 0805 Consolidate Storage on Receive queue
PK69049 UK38323 0808 16K ECSA CSM Growth with FRCA
PK69071 UK38816 0809 CSM ECSA due to deleted XCF link
PK70053 UK39695 0810 Some CSM growth with Loop in Shareport
PK74282 UK41812 0812 CSM FIXED growth - Large # appl connections
PK88593 UK48513 1000 CSM ECSA FIXED growth after PK79768
PK94784 UK50082 0910 CSM ECSA FIXED growth





TCP/IP z/OS 1.10 R1A0
APAR#PTF#PUT#Abstract
PK68520UK38177 0808 ABEND S0C4 in WEB SERVER - Fixed CSM growth
PK69049 UK38322 0808 16K ECSA CSM Growth in FRCA
PK69071 UK38815 0809 CSM ECSA due to deleted XCF link
PK70053 UK39693 0810 Some CSM growth with LOOP in Shareport
PK74282 UK41810 0812 CSM FIXED growth - Large # appl connections
PK82625 UK45669 0905 CSM HDR growth for image buffers
PK88593 UK48512 1000 CSM ECSA FIXED growth after PK79768
PK92542 UK49584 0910 CSM ECSA FIXED growth (**PE ** see PM03639)
PK94784 UK50080 0910 CSM ECSA FIXED growth
PM03639 UK53325 1001 PE fix for PK92542
PM29360UK6403211024K CSM ECSA with EE Connectivity Test response
PM31494UK65650110332K Fixed CSM Dataspace growth containing IUDR Packing Buffers
PM43451UK703041108CSM FIXED ECSA 180K STORAGE GROWTH



TCP/IP z/OS 1.11 R1B0
APAR#PTF#PUT#Abstract
PK90619 UK48409 0908ECSA growth with CSM headers
PK92542 UK49584 0910CSM ECSA FIXED growth (**PE ** see PM03639)
PK94784
UK50081 0910CSM ECSA FIXED growth
PM03639 UK53326 1001 PE fix for PK92542
PM19021UK598551000High CSM 32K ECSA Buffers with Hipersockets
PM29360UK6403311024K CSM ECSA with EE Connectivity Test response
PM31494UK65651110332K Fixed CSM Dataspace growth containing IUDR Packing Buffers
PM32715UK652951103CSM Fixed 64K Dataspace growth when large FRCA cache is active
PM43595UK703871109CSM 4K DSPACE64 ECSA SKMB Growth - DRS Receive Window Size
PM52626UK753251202Multiple CSM Dataspaces


TCP/IP z/OS 1.12 R1C0
APAR#PTF#PUT#Abstract
PM19021UK598561000High CSM 32K ECSA Buffers with Hipersockets
PM29360UK6403411024K CSM ECSA with EE Connectivity Test response
PM31494UK65652110332K Fixed CSM Dataspace growth containing IUDR Packing Buffers
PM32715UK652961103CSM Fixed 64K Dataspace growth when large FRCA cache is active
PM39080UK689541107CSM 180K Fixed Dataspace growth
PM43595UK703881109CSM 4K DSPACE64 ECSA SKMB Growth - DRS Receive Window Size
PM52626UK753261202Multiple CSM Dataspaces
PM81034UK917041304CSM Fixed 32K Dataspace storage leak


TCP/IP z/OS 1.13 R1D0
APAR#PTF#PUT#Abstract
PM39080UK689551107CSM 180K Fixed Dataspace growth
PM43595UK703891109CSM 4K DSPACE64 ECSA SKMB Growth - DRS Receive Window Size
PM52626UK753271202Multiple CSM Dataspaces
PM81034UK917061304CSM Fixed 32K Dataspace storage leak
PI05014UI122421312GROWTH OF SP231K6 ECSA STORAGE WITH STRANDED SKMB/SKDB ZBLOCKS
PI24894UI21515F412FTP JOB ENDS WITH CODE S0C4 SEQ_READ_FILE_MULTI +0X00000560
PI35004UI25555F503 FIXED CSM DATASPACE STORAGE LEAK WHEN USING IQDIOMULTIWRITE


TCP/IP z/OS 2.1 R210
APAR#PTF#PUT#Abstract
PI05583UI123611312GROWTH OF SP231K6 ECSA STORAGE WITH STRANDED SKMB/SKDB ZBLOCKS
PI32328 UI25527F503FIXED CSM DATASPACE STORAGE LEAK WHEN USING IQDIOMULTIWRITE
PI38699UI27273F506FIXED CSM GROWTH FOR CERTAIN TCP CONNECTIONS
PI59617UI37397F606POTENTIAL FOR ELEVATED 4K ECSA CSM USAGE
PI62708UI38145F606CSM 4K ECSA FRAGMENTATION DUE TO IPASSIST IMAGE BUFFERS
PI65912UI40995F610POTENTIAL FOR AN INCORRECT TCP WINDOW SIZE TO BE SENT
PI67916UI41140F610ABEND0C4 IN EZBTCFWR + X'AE90'OVERLAYING NEXT CSM BUFFER STORAGE
PI72070UI43643F702CSM FIXED STORAGE GROWTH WITH DATA ON TCP SEND QUEUE
PI72631UI43289F702LOCK CONTENTION WHEN CSM STORAGE GOES CONSTRAINED
PI77346UI46480F706SYSTEM ECSA STORAGE WAS FULL OF TCPIP DUCBS
PI82517UI47950F707TCPIP USAGE OF CSM STORAGE EFFICIENCY IMPROVEMENT

TCP/IP z/OS 2.2 R220
APAR#PTF#PUT#Abstract
PI46872UI30827F511FIXED CSM GROWTH FOR CERTAIN TCP CONNECTIONS
PI59617UI37405F606POTENTIAL FOR ELEVATED 4K ECSA CSM USAGE
PI62708UI38146F606CSM 4K ECSA FRAGMENTATION DUE TO IPASSIST IMAGE BUFFERS
PI63280UI39328F608POTENTIAL GROWTH OF 4K CSM FIXED DATA SPACE USING LOOPBACK EE
APPLICATION
PI65912UI40996F610POTENTIAL FOR AN INCORRECT TCP WINDOW SIZE TO BE SENT
PI67916UI41141F610ABEND0C4 IN EZBTCFWR + X'AE90'OVERLAYING NEXT CSM BUFFER STORAGE
PI71824UI43341F701SP231 KEY6 GROWTH EE ZEN SKMB SKDB MPROTO MESSAGE TRIPLE
PI72070UI43644F702CSM FIXED STORAGE GROWTH WITH DATA ON TCP SEND QUEUE
PI72631UI43290F702LOCK CONTENTION WHEN CSM STORAGE GOES CONSTRAINED
PI76201UI44740F703USING FTP SMC-D WITH ATTLS RESULTS IN POOR PERFORMANCE AND MANY SMALL OUTBOUND DATA PACKETS.
PI77346UI46481 F706SYSTEM ECSA STORAGE WAS FULL OF TCPIP DUCBS
PI80158UI47694F707TCPIP USAGE OF CSM STORAGE EFFICIENCY IMPROVEMENT
PI86457UI50159F711CSM 60K DATA SPACE STORAGE GROWTH
PI87015UI53802F803 TCPPULLUP causes CSM HVCOMM storage constrained condition
PI87665UI51644F712IVT5607I CSM HVCOMM STORAGE AT CONSTRAINED LEVEL


TCP/IP z/OS 2.3 R230
APAR#PTF#PUT#Abstract
PI81483UI47973F708SYSTEM ECSA STORAGE WAS FULL OF TCPIP DUCBS
PI84331UI48895F708TCPIP USAGE OF CSM STORAGE EFFICIENCY IMPROVEMENT
PI86457UI50160F711CSM 60K DATA SPACE STORAGE GROWTH
PI87015UI53803F803TCPPULLUP causes CSM HVCOMM storage constrained condition
PI87665UI51645F712IVT5607I CSM HVCOMM STORAGE AT CONSTRAINED LEVEL









Back to Top

Additional Information

Additional Information regarding VTAM CSM usage (HPDT and CSM):
z/OS Communications Server: SNA Network Implementation Guide, Chapter 23
z/OS Communications Server CSM Guide

[{"Product":{"code":"SSSN3L","label":"z\/OS Communications Server"},"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Component":"All","Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"1.6;1.7;1.8;1.9;1.10;1.11;1.12;1.13;2.1;2.2;2.3","Edition":"","Line of Business":{"code":"LOB35","label":"Mainframe SW"}}]

Historical Number

II13951

Document Information

Modified date:
15 June 2018

UID

swg21318426