Diagnosis of MVS System Logger Problems

Extended waits by CICS® VR for logging can be caused by problems within the MVS system logger or other areas of MVS.

You can investigate these by looking at the MVS console messages. Look at the following information:
  • Console messages and dumps
    Look for the following information:
    • Outstanding WTOR messages
    • IXGxxx messages
    • Allocation, catalog and HSM error messages
    • I/O errors for log stream data sets and/or LOGR couple data sets
    • IXCxxx messages, indicating problems with the coupling facility structure or couple data sets
    • 1C5 abends, and other abends from the IXGLOGR address space

      Log stream data sets are of the form IXGLOGR.stream_name.Annnnnnn. The high level qualifier (IXGLOGR) might be different if the HLQ parameter was specified when the log stream was defined.

  • Global resource serialization (GRS) resource contention
    To check GRS resource contention by displaying GRS enqueues and latch usage on all machines in the sysplex, issue either of the following MVS commands: 
          D GRS,C
      D GRS,RES=(SYSZLOGR,*)
    A normal response looks like: 
          D GRS,C

      ISG020I 12.06.49 GRS STATUS 647
      NO ENQ CONTENTION EXISTS
      NO LATCH CONTENTION EXISTS

      D GRS,RES=(SYSZLOGR,*)

      ISG020I 14.04.28 GRS STATUS 952
      NO REQUESTORS FOR RESOURCE  SYSZLOGR *
    A response showing GRS contention looks like this:1 
          D GRS,C

      ISG020I 12.06.31 GRS STATUS 619
      LATCH SET NAME:  SYS.IXGLOGER_LCBVT
      CREATOR JOBNAME: IXGLOGR   CREATOR ASID: 0202
        LATCH NUMBER:  7
          REQUESTOR  ASID  EXC/SHR    OWN/WAIT
          IXGLOGR    0202  EXCLUSIVE  OWN
          IXGLOGR    0202  SHARED     WAIT

      D GRS,RES=(SYSZLOGR,*)

      ISG020I 19.58.33 GRS STATUS 374
      S=STEP    SYSZLOGR  91
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV26      MSLDELC1           002F       008F6370 EXCLUSIVE   OWN
      S=STEP    SYSZLOGR  93
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV26      MSLWRTC1           002E       008DED90 EXCLUSIVE   OWN
      MV26      MSLWRTC1           002E       008DB990 EXCLUSIVE   WAIT
      MV26      MSLWRTC1           002E       008DB700 EXCLUSIVE   WAIT
      MV26      MSLWRTC1           002E       008F60C8 EXCLUSIVE   WAIT
      S=SYSTEMS SYSZLOGR LPAYROL.TESTLOG.TLOG1
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV27      IXGLOGR            0011       008F7398 EXCLUSIVE   OWN
      MV26      IXGLOGR            0011       008F7398 EXCLUSIVE   WAIT

    This shows which tasks have exclusive enqueues on the log streams, and which tasks are waiting for them. It is quite normal for enqueues and latches to be obtained, occasionally with contention. They are indications of a problem only if they last for more than a minute or so.

    Long term enqueuing on the SYSZLOGR resource can be a sign of problems even if there is no contention.

    You can choose to display only those log streams exclusively enqueued on, or being waited on, by CICS VR jobs in the sysplex. Issue the following MVS command: 
          D GRS,RES=(DFHSTRM,*)
    A typical response to this command looks like this: 
          ISG020I 14.51.28 GRS STATUS 541
      S=SYSTEMS DFHSTRM  PAYROL.CICSVRVR.DFHLGLOG
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL91           0042       007D9108   SHARE     OWN
      MV29      PAYROL93           0044       007D9138   SHARE     OWN
      S=SYSTEMS DFHSTRM  PAYROL.FWDRECOV.UTL3
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL91           0042       007D9108   SHARE     OWN
      MV29      PAYROL93           0044       007D9138   SHARE     OWN
      S=SYSTEMS DFHSTRM  PAYROL.IYK8ZET1.DFHJ02
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL91           0042       007D9108   SHARE     OWN
      S=SYSTEMS DFHSTRM  PAYROL.IYK8ZET1.DFHLOG
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL91           0042       007D9108 EXCLUSIVE   OWN
      S=SYSTEMS DFHSTRM  PAYROL.IYK8ZET1.DFHSHUNT
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL91           0042       007D9108 EXCLUSIVE   OWN
      S=SYSTEMS DFHSTRM  PAYROL.IYK8ZET3.DFHJ02
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL93           0044       007D9138   SHARE     OWN
      S=SYSTEMS DFHSTRM  PAYROL.IYK8ZET3.DFHLOG
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL93           0044       007D9138 EXCLUSIVE   OWN
      S=SYSTEMS DFHSTRM  PAYROL.IYK8ZET3.DFHSHUNT
      SYSNAME        JOBNAME         ASID     TCBADDR   EXC/SHR  OWN/WAIT
      MV29      PAYROL93           0044       007D9138 EXCLUSIVE   OWN
  • Coupling facility status
    To display the MVS system logger couple data set status, issue the following MVS command: 
        D XCF,CPL,TYPE=LOGR
    A normal response looks like this: 
          D XCF,CPL,TYPE=LOGR
      IXC358I  14.47.51  DISPLAY XCF 391
      LOGR COUPLE DATA SETS
      PRIMARY    DSN: SYS1.SYSPLEX2.SEQ26.PLOGR
                 VOLSER: P2SS05     DEVN: 230D
                 FORMAT TOD         MAXSYSTEM
                 12/20/95 09:25:48          8
      ALTERNATE  DSN: SYS1.SYSPLEX2.SEQ26.ALOGR
                 VOLSER: P2SS06     DEVN: 2C10
                 FORMAT TOD         MAXSYSTEM
                 12/20/95 09:27:45          8
      LOGR IN USE BY ALL SYSTEMS
    If the response shows that LOGR is not in use by all systems, there might be a problem to investigate. Look for IXCxxx messages which might indicate the cause of the problem and issue the following command to attempt reconnection to the couple data set: 
          SETXCF CPL,TYPE=(LOGR),PCOUPLE=(couple_dataset_name)
    To display all structures with Failed_persistent connections, issue the following MVS command: 
        D XCF,STR,STRNM=*,STATUS=FPCONN

    The MVS system logger resolves any failed connections.

  • SMF and RMF statistics

    SMF 88 log stream statistics records and RMF coupling facility usage reports are useful for analyzing problems that are affecting performance. Increasing the amount of coupling facility storage allocated to a structure can improve both MVS system logger performance and CICS VR performance.

1 You might also see latch set name SYS.IXGLOGER_MISC.