IV82177: TEMS SHUTDOWN WHILE UPDATING EVENT MAPS

IBM Tivoli Monitoring 6.3.0 Fix Pack 7 (6.3.0-TIV-ITM-FP0007)

APAR status

• Closed as program error.

Error description

• APAR Type:            Field
  Approver Initials:    RB
  Severity:             3
  Reported Release:     630
  Compid:               5724C04MS Tivoli Enterprise Management
  Server
  ABSTRACT:             TEMS Shutdown while updating event maps
  PROBLEM DESCRIPTION:
  There has been cases where TEMS would shut down while
  processing event
  map updates. This might affect event map inserts and deletes
  also, but
  that has not yet been seen. This problem is not common. It is
  most
  likely to occur if there are hundreds of event map updates
  being done
  sequentially. This has only been seen in FTO synchronization
  thus
  far, but this is not FTO specific. This problem can confirmed by
  seeing the following in the TEMS log:
  
    ...kglkycbt.c,841,"kglky1rs") Unknown isam rc = -4
    ...kfaibloc.c,813,"IBL_Process") Error: KEY1 failed with rc
  <8>
    ...kdsstc1.c,1658,"ProcessTable") ProcessTable Locator status
  = 64
    ...kdsvws1.c,2421,"ManageView") ProcessTable EVNTMAP Select
  Error
       status = ( 64 ).  KFAOMTEC
    ...kdsrqc1.c,2624,"AccessRowsets") Could not obtain rowset 2,
  status
       code = 64
    ...kfaoteib.c,685,"Get_User_Map_Entry") Unable open event map
  request.
       status <64>
    ...kfaoteib.c,699,"Get_User_Map_Entry") Exit: 0x8
  
  RECREATE INSTRUCTIONS:
  1. Create 5,000 event map records.
  2. Restart the TEMS
  3. Update all 5,000 event map records using SQL submitted
  sequentially
     via a program.
  
  FTO re-syncronizing 5,000 event map records can also be used to
  recreate
  this issue.
  1. Have 5,000 event map records at both HUBs.
  2. With the standby stopped, update all 5,000 event map
  records. An
     updated time stamp on the records is sufficient.
  3. Start the standby HUB. Updating the 5,000 event map records
  may
     result in this issue at the standby HUB.
  

Local fix

• Retry. This is simply a timing issue. Retrying will eventually
  get you
  around the problem. The fewer the number of event map updates,
  the less
  likely the problem is.
  

Problem summary

• Submitting multiple event map updates consecutively without
  delay can result in the HUB Tivoli Enterprise Monitoring Server
  shutting down.  This problem is not likely to occur when only a
  small number of event map updates are made.  This problem will
  not occur if event forwarding is not used.
  This problem is not common.  It is most likely to occur if there
  are hundreds of event map updates being done sequentially.  This
  has only been seen in FTO synchronization thus far, but this is
  not FTO specific.  This problem can confirmed by seeing the
  following in the TEMS log:
  
    ...kglkycbt.c,841,"kglky1rs") Unknown isam rc = -4
    ...kfaibloc.c,813,"IBL_Process") Error: KEY1 failed with rc
  <8>
    ...kdsstc1.c,1658,"ProcessTable") ProcessTable Locator status
  = 64
    ...kdsvws1.c,2421,"ManageView") ProcessTable EVNTMAP Select
  Error status = ( 64 ).  KFAOMTEC
    ...kdsrqc1.c,2624,"AccessRowsets") Could not obtain rowset 2,
  status code = 64
    ...kfaoteib.c,685,"Get_User_Map_Entry") Unable open event map
  request.  status <64>
    ...kfaoteib.c,699,"Get_User_Map_Entry") Exit: 0x8
  

Problem conclusion

• The code was changed to better protect the event map table
  during the update and fetch processes.
  
  The fix for this APAR is contained in the following maintenance
  packages:
  
    | fix pack | 6.3.0-TIV-ITM-FP0007
  

Temporary fix

• Restart the Tivoli Enterprise Monitoring Server, then submit the
   remaining updates.  Introduce a slight delay between the
  updates if possible.
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  OA52202

Fix information

• Fixed component name

  TEMS

• Fixed component ID

  5724C04MS

Applicable component levels