IZ34110: TEMS CRASHES DUE TO ROWDICT BEING FREED

APAR status

• Closed as program error.

Error description

• The stack trace will show that the crash is likely to occur
  while a
  rowDict is being freed.
  
  MSVCR71! free+0x39
  KSM!asDic::~asDict+0x33
  KSM!asDict::`vector deleting destructor'+0x4d
  KSM!rowDict::~rowDict+0xe9
  KSM!rowDict::'vector delient destructor'+0x4d
  KSM!rwDestroy+0x2f
  KSM!RWSlist::apply+0x2f
  KSM!RWSlistCollectables::apply+0x1a
  KSM!RWHashTalble::apply+0x6c
  KSM!RWSet::clearAndDestory+0x19
  KSM!RWCollection::clearAndDestroy+0x34
  KSM!IBInterface::accessListChange+0x27dc
  KSM!IBInterface::sendLogEvent+0x3c5
  KSM!IBInterface::updateIB+0x1aa0
  
  
  As the trace above shows, "updateIB" will likely appear
  somewhere higher
  in the stack.
  updateIB is called to handle an EIBLOG notification from the
  Hub.
  One example of such a notification occurs when an agent switches
  to an
  RTEMS and the Hub tells the RTEMS which Situations are
  distributed to
  that agent.
  In this case, updateIB reads the Situation's definition and
  calls
  accessListChange.
  accessListChange calls a function called updateHubInterest to
  change a
  property called "HUB" to 'Y".
  A defect in this latter function results in there being multiple
  copies
  of the HUB property.
  
  A rowDict has the capacity to hold up to 64 "key/value" pairs
  and the
  "endIdx" data member reflects the last slot that is in use.
  As array indices are zero-based, the value should never exceed
  63.
  Processing a sufficient number of EIBLOG notifications will
  cause the
  capacity of the dictionary to be exceeded.
  In the view below, the fact that the index stands at 64 means
  that data
  has been written into unallocated storage, beyond the extent of
  the
  array.
  
  Results from this point will be unpredictable but a crash is
  likely to
  occur soon after, in this case, when an unrelated control block
  is being
  freed.
  
  
  Detailed Recreation Procedure:
  
  Start adding in scale agents to reach target of 1200 agents per
  RTEMS.
  
  
  Related Files and Output:
    Indicate log location, trace levels used, command output,
    configuration files, coredumps, etc.
  

Local fix

Problem summary

• TEMS may crash when many agents come online and situations are
  distributed to them.
  

Problem conclusion

• Any operation that causes a large number of EIBLOG requests
  to be processed may cause this problem to occur but it is
  intermittent.  The crash is caused by freeing a corrupted data
  object.
  
  The fix for this APAR will be included in the following
  maintenance vehicle:
      | interim fix | 6.1.0.7-TIV-ITM-IF0004
  
  Note: Search the IBM Technical support web site for maintenance
  package availability
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  IZ36011 OA26951 OA28053 OA29278

Fix information

• Fixed component name

  TEMS

• Fixed component ID

  5724C04MS

Applicable component levels

• R610 PSY

     UP