IT06246: CDR CLEANSTART COMMAND NOT PURGING TRG_SEND_SREP TABLE

APAR status

• Closed as program error.

Error description

• When cdr cleanstart is used to reset the replay position and
  throw away transactions that have been spooled to disk, it
  currently is only removing rows from the trg_send_stxn table.
  However, when rows getting written to spool, data is also put in
  the trg_send_srep table, and when transactions are removed
  normally from spool, this data is also removed.  But in the case
  of cdr cleanstart the data is not.
  
  Leaving this data behind can cause other ER problems, like
  seeing long running CDRFlushQue threads or long running GCReq
  threads.  These long running threads would have the following
  stack traces:
  
  CDRFlushQue:
  yield_processor_mvp
  mt_lock_wait
  mt_lock
  rqmGetTxnByStamp
  rqmMarkSpoolReplDelete
  qrDoFlushQueueForRepl
  queuerFlushQueueForRepl
  cdrTrampolineThread
  th_init_initgls
  startup
  
  GCReq:
  mt_lock_wait
  mt_lock
  rqmGetTxnByStamp
  rqmMarkSpoolReplDelete
  qrFlushQueueForRepl
  queuerReplDropped
  dac_participate_repl
  gsi_participate_repl
  clientRequestThread
  cdrTrampolineThread
  th_init_initgls
  startup
  
  The key pieces in the stacks above is the function
  rqmMarkSpoolReplDelete, as it's the function which indicates the
  trg_send_srep table is being scanned.  So in the case of after a
  cleanstart there should be nothing in the spool file, so this
  should not take time as there should be nothing to do if other
  ER onstats (like onstat -g rqm sendq) indicate that there are no
  transactions in spool, but if it taking cpu cycles, that would
  indicate the presence of the left over, non-purged rows in the
  trg_send_srep table.
  
  If the server does have transactions in spool, then taking time
  in this function would be normal, as going through the  spool
  files cleaning things up is required, but when onstat -g rqm
  sendq shows no transactions in spool, but there are threads
  spending a lot of time in rqmMarkSpoolReplDelete() then that
  would indicate the customer had done a cleanstart and now has
  these left over rows which is then causing issues.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * all customers with stuff in the stable queue when cdr        *
  * cleanstart is issued.                                        *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Update to IBM Informix Server 11.70                          *
  ****************************************************************
  

Problem conclusion

• Problem Fixed In IBM Informix Server 11.70
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  INFORMIX SERVER

• Fixed component ID

  5725A3900

Applicable component levels

• RA10 PSN

     UP

• RA10 PSY

     UP

• RB10 PSN

     UP

• RB10 PSY

     UP

• RB50 PSN

     UP

• RB50 PSY

     UP

• RB70 PSN

     UP

• RB70 PSY

     UP

• RC10 PSN

     UP

• RC10 PSY

     UP