PI53592: THE LOCK ON USAGE COUNT TABLE

APAR status

• Closed as program error.

Error description

• Production issue (one occurrence)
  
  +BIP4362E BPD4 TMEP02 25 JAVA NODE ERROR: Caught unexpected
  XAException.
  Error executing XAResource
   .commit().  Server returned XAER_PROTO. ERRORCODE=-4203,
  SQLSTATE=null
    Error code: -6. : ImbNativeTrace(698)
  
  
  The initial error was followed by all execution groups raising
  usage count errors at 10 min intervals until the DBA killed a
  long running/orphaned connection.
  
  A failure seems to have occurred during JDBC commit after usage
  count updates.
  
  FTM code does not do the commits, that is managed by Broker,
  RRS and DB2.
  

Local fix

• Because this occurred once and not all logs are available for
  investigation and reproduction is extremely difficult we can
  only theorize on what has happened.
  Our current thinking is that an error during the JDBC commit
  has occurred (reason unkown) and this has left the transaction
  in a intermediate state.
  
  We have discussed theoretically this type of situation with
  broker L3 developers.
  Because the ODBC update is co-ordinated and the JDBC is not it
  looks as though the transaction may have been left in an
  inconsistent state (ODBC committed, JDBC not). We think because
  Broker can not rollback the ODBC transaction it can not
  automatically recover the  unit of work.  This has then left
  locks open at DB2 that can only be recovered manually.  The
  implementation of the usage count means that these locks impact
  all other execution groups (since they update shared counters)
  until it is manually recovered at DB2.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All users of Financial Transaction Manager   *
  *                 v3.0                                         *
  ****************************************************************
  * PROBLEM DESCRIPTION: The FTM usage count records are shared  *
  *                      by all execution groups running for a   *
  *                      given business day.                     *
  *                      If an error occurs resulting in a lock  *
  *                      anomaly then all execution groups will  *
  *                      encounter issues accessing the shared   *
  *                      counters.                               *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  BIP4362E Caught unexpected XAException
  Message: (jcc)(t4)(2041)(12326)(3.63.107)
  Error executing XAResource.commit().
  Server returned XAER_PROTO. ERRORCODE=-4203, SQLSTATE=null
  BIP4362E com.ibm.fxh.profiling.messages
  :UsageMonitor.Error Failed to update usage count.
  

Problem conclusion

• The fix for this APAR can be found in Fixpack v3.0.0.4 on Fix
  Central(http://www.ibm.com/support/fixcentral) - search
  for fixpack ID : 3.0.0-FTM-MP-fp0004.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  PI48554

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  FTM FOR MP

• Fixed component ID

  5725F7930

Applicable component levels

• R300 PSY

     UP