OA69599: MISSING SMF30 Subtype2 RECORDS, ABEND878-10, or Z/VM GUEST TIMER DIE LOOP

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• When running z/OS 3.2 there are different symptoms that may be
  encountered:
  
  SYMPTOM 1: SMFTYPE30 SUBTYPE2 records are not being written
  for some address spaces. Type30, Subtype2, (aka Type30.2).
  
     KNOWN IMPACT:
     SMF TYPE30 SUBTYPE2 records will not be written for some
     address spaces
  
     VERIFICATION STEPS:
     You must be running z/OS 3.2 as this APAR does not apply to
     earlier releases:
     1. Missing Type30 Subtype2 (Type30.2) records are often
     first noticed in CICS regions, but may still apply to other
     address spaces. Check whether expected Type30.2 records are
     missing or not being recorded after running your IFASMFDP
     archive/dump (aka "SMFDUMP") program.
  
  SYMPTOM 2: A z/VM guest at z/OS 3.2 under CP may become
  unresponsive. This can occur after the following sequence:
  
     - Issuing CP SLEEP
     - After a variable time frame issuing a CP BEGIN to the
       z/OS guest
  
     The z/VM guest becomes unusable. In systrace this appears
     as repeated SSRV 120 (Timer DIE) entries. The PSW of these
     entries points to IEEMB839+x'F0E'
  
     KNOWN IMPACT:
     z/VM guest will loop and not be usable
  
     VERIFICATION STEPS:
     You must be running z/OS 3.2 as this APAR does not apply to
     earlier releases:
     1. Capture a SADump of the unresponsive VM system. In the
     dump the System Trace table (IP SYSTRACE TIME(L)) will show
     repetitive entries in SMF module/csect IEEMB839
        a. The command 'IP SYSTRACE' should show repeated
           SSRV 120 (Timer DIE) entries
        b. Check the PSW addresses using either IP W xxxxxxxxx
           or browse the PSW address in storage to confirm that
           the PSW is IEEMB839 +x'F0E'.
        c. Or contact IBM Service to examine the dump for
           confirmation.
  
  SYMPTOM 3: Storage leak in SP230 Key0. This appears as a
  continuous growth of 8K blocks. The 8K blocks contain
  "Hold data" for SMF Type30 records if SMFPRMxx includes EXCP
  Sections (if SYS/SUBSYS > TYPE > DETAIL is specified). Very
  high additional growth may be seen if Empty EXCP Sections are
  being included eg. EMPTYEXCPSEC(NOSUPPRESS).
  
     VERIFICATION STEPS:
     You must be running z/OS 3.2 as this APAR does not apply to
     earlier releases:
     1. Identify the DECIMAL ASID# of the address space
        experiencing the Private Storage growth
        (it may have encountered an ABEND878-10 but, if not,
        should be growing steadily)
     2. In either an SVC Dump or a SADmp, go to IPCS and issue:
           IP VERBX VSMDATA 'NOG SUMM ASID(decimal ASID#)'
     3. Max to the bottom of the report.
     4. Find the entries (there will be many) that are only for
        SP# 230, KEY 0, which will look something like the
        example below:
     LOCAL SUBPOOL USAGE SUMMARY
         TCB/OWNER   SP#   KEY      BELOW    ABOVE    TOTAL
         ---------   ---   ---      -----    -----    -----
         .......etc..............
  
         8FD6A0      230    0        1000 114E7000 114E8000
         (extremely large storage utilization)
         .......etc............
     5. Find the largest entry, it really should be a very
        noticeably large value compared to all the other entries
        you see.
     6. Then "F 114E7000 PREV"
        (you need to replace this hex number with what you find
        under the ABOVE column in your SUMMARY report above)
     7. Once you find it you'll see this information below, you
        need to PAGE UP to look at the DQE's and Sizes of 8000.
    .......etc..........
    DQE:  Addr 7F03C000 Size   8000   TCB: 008FD6A0 SP/K: 230/ 0
     FQE: Addr 7F03C000 Size     10   TCB: 008FD6A0 SP/K: 230/ 0
    DQE:  Addr 7F078000 Size   8000   TCB: 008FD6A0 SP/K: 230/ 0
     FQE: Addr 7F078000 Size     10   TCB: 008FD6A0 SP/K: 230/ 0
    .......etc........
    **** Subpool  230, key  0 Total alloc:  114E8000
            (1000 Below, 114E7000 Above)
     8. Then save the TCB address that shows up for all the
        "Size   8000" entries, and issue:
           "IP SUMM FO ASID(x'__')"  .... use the HEX ASID#
     9. Then Max to the bottom of the SUMMARY FORMAT report,
        and PAGE UP until you see the following:
             * * * *   T C B   S U M M A R Y   * * * *
   JOB CICCSM1  ASID 02EA ASCB 00F85700 FWDP 00F85E80
                BWDP 00F86100 PAGE 00000058
        TCB AT     CMP       NTC       OTC       LTC       TCB
  --> 008FD6A0 00000000 00000000 00000000 008FED90 008FF068
        008FF068 00000000 00000000 008FD6A0 00000000 008FED90
        008FED90 00000000 008FF068 008FD6A0 008F90A0 008F90A0
     10. The TCB you saved from the VERBX VSMDATA report must
         match the FIRST TCB in the TCB SUMMARY
  

Local fix

• Contact IBM support for a circumvention.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: Installations running HBB77F0 with INTERVAL  *
  *                 specified in SMFPRMxx                        *
  ****************************************************************
  * PROBLEM DESCRIPTION: Various symptoms related to SMF         *
  *                      interval timer processing               *
  ****************************************************************
  The SMF interval processing timer queue element is not correctly
  reset at the end of a job step. This can result in the following
  symptoms:
  - Timer DIE loop
  - Storage leak in SP230 Key 0
  - SMF Type 30 Subtype 2 not properly recorded
  
  Additional Search Keywords: SMFTYPE30 SUBTYPE2 SP230 KEY0
  LOOP
  

Problem conclusion

• SMF step end processing is corrected to reset the time
  of next interval processing for the job.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  UJ10136

Modules/Macros

• IEFTB726
  

Fix information

• Fixed component name

  SMF SCHEDULER

• Fixed component ID

  5752SC100

Applicable component levels

• R7F0 PSY UJ10136

     UP26/08/12 P F608 Ž

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.