PK12869: WTOR BUFFER SHORTAGE CAUSES IMS TO HANG

APAR status

• Closed as program error.

Error description

• WTOR buffer shortages are occurring in z/OS.  The IMS
  outstanding reply occasionally hangs for a minute or more,
  then is freed up again and processing.  During this hang,
  no work is getting done in IMS.  If the IMS WTOR were running
  under a TCB other than CTL, then transactions could still be
  processing while we wait for a buffer.
  (In this case, eventually IMS was terminated with abend878.
  This apar will not address the storage shortage, only the
  fact that the IMS CTL TCB is hung while waiting for a WTOR
  buffer).
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED: All IMS V8R1 DB/TM and DCCTL users.          *
  ****************************************************************
  * PROBLEM DESCRIPTION: An MVS (z/OS) out-of-WTOR buffer        *
  *                      condition causes IMS transaction        *
  *                      processing to stop until a WTOR buffer  *
  *                      is available for IMS.                   *
  ****************************************************************
  * RECOMMENDATION: INSTALL CORRECTIVE SERVICE FOR APAR/PTF      *
  ****************************************************************
  During an MVS out-of-WTOR buffer situation, a command is entered
  on the system console using the IMS outstanding WTOR (DFS996I
  message).  The command is processed, and IMS re-issues the WTOR.
  Because there are no WTOR buffers available, MVS suspends the
  caller.
  
  IMS invokes the system console WTOR from under the IMS CTL TCB.
  This TCB is where much processing related to transaction
  activity occurs.  Because the TCB is suspended, no other work
  can be processed, and IMS transaction processing stops.
  
  In systems using OTMA or APPC, work from these sources can
  continue to be received, because both OTMA and APPC have their
  own TCBs separate from CTL.  However, the work will probably not
  be processed due to interlocks with IMS ITASKs that are unable
  to be dispatched under CTL.  This "receipt-but-no-processing" of
  work can lead to an out of storage condition in the IMS address
  space, as storage continues to be allocated for input messages
  that do not get processed.  Eventually, IMS can abend with an
  out-of-storage type abend (S878, S40D, S80A).
  
  Bypass: The number of WTOR buffers is determined by the RLIM
  and RMAX parameters in the CONSOLxx PARMLIB member used for IPL.
  RLIM is the number of WTOR buffers allocated.  RMAX is the
  maximum WTOR reply number.  You can increase the number of WTOR
  buffers at IPL time by changing these values in the CONSOLxx
  member.  See the "MVS Initialization and Tuning Reference"
  manual chapter on CONSOLxx for further information.
  
  You can also dynamically change RLIM and RMAX via the following
  z/OS commands:
  
    CONTROL M,RLIM=  (or K M,RLIM= )
    CONTROL M,RMAX=  (or K M,RMAX= )
  
  Increasing the number of WTOR buffers may relieve the WTOR
  buffer shortage and allow the IMS WTOR to be issued.
  
  See the "MVS Systems Commands" manual section on the CONTROL
  command for further information.  See also the section titled
  "Responding to WTOR Buffer Shortages" in the "Console
  Characteristics and Operations" chapter of the same manual.
  
  The current values of RLIM and RMAX can be displayed by issuing
  the command:
  
    CONTROL M,REF   (or K M,REF )
  
  Additional keywords for searchability:
  
    ABEND878 ABEND40D ABEND80A IEA230E MSGIEA230E IEA231A
    MSGIEA231A
  

Problem conclusion

•  AIDS: RIDS/SYS RIDS/CNTRL SYS CNTRL
    DEP: NONE
    GEN:
  
  *** END IMS KEYWORDS ***
  This APAR creates a new IMS control region TCB -- type WTR.
  This TCB runs no ITASKs of its own (except for the common
  termination ITASK).  Instead, it can be specified as the target
  of an ISWITCH call for code that calls a service that can
  potentially go into an extended OS WAIT.  The WTR TCB takes the
  WAIT, and frees the ISWITCHing ITASK's original TCB to dispatch
  other work.
  
  The new WTR TCB is used to prevent the suspension of the IMS CTL
  TCB when the IMS system console WTOR (MSGDFS996I *IMS READY*) is
  issued.
  
  The IMS system console WTOR DDM, DFSDNSC0, is changed to ISWITCH
  to the new WTR TCB prior to the WTOR macro, and to ISWITCH back
  to CTL immediately following the WTOR.  If there is a WTOR
  buffer shortage and the operating system suspends the IMS
  DFS996I WTOR caller, it will be the WTR TCB that is suspended,
  and not CTL.  CTL will be free to continue dispatching, and
  transaction work will continue.  The ISWITCH to the WTR TCB is
  skipped during IMS shutdown processing (because the WTR TCB is
  terminated during shutdown, prior to CTL TCB termination -- so
  it might not be there to ISWITCH to).
  
  Note that this APAR does not prevent the temporary "loss" of the
  IMS system console WTOR during a WTOR buffer shortage.  MVS
  still suspends the caller until it has a buffer; it is just that
  the caller is now running under a non-critical IMS TCB.  It also
  does not prevent ITASK waits due to interlocks between the
  system console and other IMS ITASKs below the TCB level (for
  example, if a command directed against the system console had to
  wait until the system console ITASK was not busy).
  
  Code changes:
  -------------
  
  DFSDNSC0 - System Console DDM
  
    Issue ISWITCH TO=WTR,STACK=YES prior to issuing the WTOR for
    the DFS996I message.  ISWITCH TO=UNSTACK after returning from
    the WTOR.
  
  DFSDSPSW - ISWITCH dispatcher work area table
  
    Add field DSPKDWTR in the table of IMS dispatcher work areas.
    This field is set to either the WTR TCB dispatcher work area,
    or else the CTL TCB dispatcher work area if the WTR TCB is not
    created (for example, for DBCTL, FDBR regions).
  
  DFSFDMP0 - IMS dumping module
  
    Add the new WTR TCB to the TCB table (TCBTBL) and
    TCB-to-caller table (TCB2CLR)
  
  DFSFMOD0 - IMS TCB attach module
  
    ATTACH the WTR TCB during DFSFMOD0 initial processing for
    non-DBCTL and non-FDBR region types.  Add WTR TCB data to the
    table of TCBs.  Add an ETXR for the WTR TCB (label WTRETXR).
  
  DFSFWTR0 - WTR TCB ESTAE routine (**NEW MODULE**)
  
    ESTAE routine to process abnormal termination of the
    new WTR TCB.
  
  DFSKDP00 - IMS dispatcher init/term module
  
    Compare-and-swap the address of the CTL TCB dispatcher work
    area into DSPKDWTR, but only if DSPKDWTR is zero.  This
    ensures that there is a target TCB DWA in the WTR's DWA field
    during early initialization (when WTR might not have gotten
    started yet), and for environments where the WTR TCB is not
    started (for example DBCTL, FDBR).
  
    Add a TCB-specific init routine KCDSP164 to save the address
    of the WTR TCB's DWA in DSPKDWTR, and to create the WTR common
    term ITASK.
  
  DFSKDT - Dispatcher TCB characteristics table
  
    Add definition for new WTR TCB.
  
  DFSTCBTB - IMS TCB table entry
  
    Add definition for new WTR TCB.
  
  DFSXWTR0 - WTR TCB init routine (**NEW MODULE**)
  
    Initialization routine for the new WTR TCB.
  
  ISWITCH - IMS TCB switch macro
  
    Support ISWITCH TO=WTR.  Also, support inline macro check for
    IMS shutdown and a change of target TCB to an alternate TCB
    during shutdown.
  
  ITRMWRK - Modify ITASK work area
  
    Add flags, TCB pointer, and common termination ECB pointer for
    the new WTR TCB.
  
    Note that there was no reserved space available to use for a
    new common termination ECB pointer.  This APAR adds the new
    WTR common term ECB pointer (IWTRTECB) plus three reserved
    words.  This has the effect of changing the offsets of fields
    that follow the ECB table:  ICRTM0, ICRTM0P, IWRKFDRA, and
    IWRKFDRL.  It also changes the size of the ITRMWRK block
    (IWRKSZ).
  
    This APAR also fixes a problem in the PL/X section of this
    macro.  The common terminate ECB table was not defined
    correctly in PL/X.  In particular, field ITRMECBS (the name
    for the entire ECB table sub-structure) was defined at the
    same level as the ECB pointers.  ITRMECBS thus became a 4-byte
    field, and added an extra 4 bytes in the PL/X mapping of the
    structure (compared with the assembler mapping).  This caused
    the offsets of all fields after ITRMECBS to be off by 4 bytes.
  
    This APAR changes the ECB pointers to level 5 so that they are
    underneath the ITRMECBS structure.
  
  HMK8800J
  
    Include JCLIN for new parts DFSFWTR0 and DFSXWTR0.
  
  Reassemblies:
  
    The following modules were reassembled for ITRMWRK field
    offset changes (includes modules that use the DFSCRTM macro,
    which references field ICRTM0):
  
      DFSABND0, DFSFCTT0, DFSFDLF0, DFSFDLI0, DFSFDLX0, DFSFSTM0,
      DFSISI00, DFSRWES0, DFSSDL20, DFSTERM0, DFSTMOD0, DFSXRID0
  
    The following modules were reassembled to pick up the new WTR
    TCB type for dispatcher trace entry formatting:
  
      DFSATRA0, DFSERA60, DFSERA61
  
    The following module was reassembled to include the new WTR
    TCB in the dispatcher statistics area length for the X'450F'
    log record (ST450F_REC_LEN):
  
      DFSSTAT0
  
  Note: This APAR affects both SYSGEN and NON SYSGEN
  modules.  If, after implementing this maintenance, you receive
  the following symptom:
  
    ABENDU3050 SUBCODE001 with R2 = X'00000031' during IMS
    initialization
  
  then most likely this service is only partially implemented.
  This is commonly caused by one of the following:
  
   1. Service in APPLY only status and performing a SYSGEN.
      This can be corrected by ACCEPTing the service
      and re-running SYSGEN or re-processing this service.
  
   2. Not running SMP/E JCLIN after SYSGEN and subsequently
      APPLYing.  This can be corrected by running SMP/E JCLIN
      pointing to the STAGE 2 created by SYSGEN and
      re-processing this service.
  
  If you have any questions relating to this, contact the IBM
  Support Center.
  
  Control block length change:
  ----------------------------
  This APAR changes the length of the Modify ITASK work area
  (macro ITRMWRK).  The old length was X'370' bytes.  The new
  length is X'380' bytes.  The following EQU symbol values have
  changed:
  
                  Old Value      New Value
  
    IFLGSZ         X'184'          X'194'
    IWRKSZ         X'370'          X'380'
  
  
  Additionally, the following field offsets have changed in the
  ITRMWRK block:
  
                  Old Offset     New Offset
  
    ICRTM0         X'348'          X'358'
    ICRTM0P        X'34C'          X'35C'
    IWRKFDRA       X'350'          X'360'
    IWRKFDRL       X'354'          X'364'
  
  Any usermods or other code dependent on any of these fields must
  be reassembled after applying this maintenance.  Also, any code
  using macro DFSCRTM must be reassembled.
  
  Log record length change:
  -------------------------
  This APAR changes the length of the X'450F' dispatcher
  statistics log record.
  
    Old length:  X'0D60' bytes
    New length:  X'0DA0' bytes
  
  Note that the format of the log record is not changed.  However,
  there is now one additional TCB statistics entry for the new WTR
  TCB type.
  
  Documentation change:
  ---------------------
  This APAR changes the IMS V8 Failure Analysis Structure Tables
  (FAST) for Dump Analysis manual (LY37-3743), chapter 2, under
  ABENDU0071:
  
    - In the "Analysis" section, add module DFSXWTR0 to the list
      of modules that issue this abend.
  
    - Add a new section for module DFSXWTR0 as follows:
  
  DFSXWTR0
  
  Explanation
  ABENDU0071 is a standard abend, issued by module DFSXWTR0.
  
  Analysis
  DFSXWTR0 invokes lower level modules to perform specific
  initialization.  ABENDU0071 results from a nonzero return code
  passed back to DFSXWTR0 from one of the lower level modules.
  The PSW at entry to abend points to the instruction where the
  abend was issued.  Register 15 at abend contains the abend
  subcode.  Register 8 at abend contains the return code from the
  failing service.
  
    Key       Label       Description
    -------   ----------- ----------------------------------------
    R15=X'01' LOWRAB01    DFSBCB GET for a QSAV failed. Register 8
                          contains the return code from DFSBCB.
  
    R15=X'02' LOWRAB02    DFSCDSP failed to create an IMS
                          dispatcher work area for the WTR TCB.
                          Register 8 contains the DFSCDSP return
                          code.
  

Temporary fix

• *********
  * HIPER *
  *********
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  PK13170 UK08698

Modules/Macros

•    DFSABND0 DFSATRA0 DFSDNSC0 DFSDSPSW DFSERA60
  DFSERA61 DFSFCTT0 DFSFDLF0 DFSFDLI0 DFSFDLX0 DFSFDMP0 DFSFMOD0
  DFSFSTM0 DFSFWTR0 DFSISI00 DFSKDP00 DFSKDT   DFSRWES0 DFSSDL20
  DFSSTAT0 DFSTCBTB DFSTERM0 DFSTMOD0 DFSXRID0 DFSXWTR0 HMK8800J
  ISWITCH  ITRMWRK
  

Fix information

• Fixed component name

  IMS V8

• Fixed component ID

  5655C5600

Applicable component levels

• R800 PSY UK08698

     UP05/11/04 P F511