Fixes are available
9.0.5.0: WebSphere Application Server traditional Version 9.0.5 Refresh Pack
9.0.5.1: WebSphere Application Server traditional Version 9.0.5 Fix Pack 1
9.0.5.2: WebSphere Application Server traditional Version 9.0.5 Fix Pack 2
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
9.0.5.3: WebSphere Application Server traditional Version 9.0.5 Fix Pack 3
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21
APAR status
Closed as program error.
Error description
An application server that propagates transactions to a backend application server running in a cluster with transactional high availability enabled and the transaction logs stored in a relational database receives repeated failures to commit transactions when invoking the commit method on the backend server's subordinate transaction coordinator. The front end application server receives the exception org.omg.CORBA.OBJECT_NOT_EXIST with minor code 0. The callstack from the backend server is of the form: >> SERVER (id=abababab, host=somehost) TRACE START: >> org.omg.CORBA.OBJECT_NOT_EXIST: vmcid: 0x0 minor code: 0 completed: No >> at com.ibm.ws.Transaction.JTS.JTSServantManager.keyToObject(JTSServ antManager.java:283) >> at com.ibm.ejs.oa.EJSOAImpl.keyToObject(EJSOAImpl.java:553) >> at com.ibm.ejs.oa.EJSRootOAImpl.keyToObject(EJSRootOAImpl.java:277) >> at com.ibm.rmi.corba.ObjectManager.lookupServant(ObjectManager.java :104) >> at com.ibm.CORBA.iiop.ServerDelegate.getServantForRequest(ServerDel egate.java:394) >> at com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:4 61) >> at com.ibm.rmi.iiop.ORB.process(ORB.java:616) ... The front end application server will report a HeuristicMixedException to the application. Although the transaction committed, the subordinate branch in the backend server rolled back and a mixed transactional outcome occurred. Investigation shows that: 1 - The backend server had previously started to recover a peer server's transaction logs (stored in a database) and generated a CWRLS0013I message indicating this. 2 - The peer had later reclaimed its transaction logs 3 - The backend server had not generated a CWRLS0014I message to indicate that it had finished processing the peer's transaction logs.
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: WebSphere Application Server users of * * transactional high availability * **************************************************************** * PROBLEM DESCRIPTION: Transactions repeatedly fail to commit * * with OBJECT_NOT_EXIST minor code 0. * * Transaction outcome is mixed. * **************************************************************** * RECOMMENDATION: * **************************************************************** A timing window existed when using transactional high availability with the transaction service logs stored in a relational database whereby an application server performing peer recovery would not correctly clear up state when handing back the logs. This resulted in WLM misrouting requests for the second phase of the transaction 2PC protocol (ie commit) to the application server that were intended for the peer application server, the prepare phase having been routed correctly. More importantly, it also resulted in the incorrect processing of those requests resulting in a org.omg.CORBA.OBJECT_NOT_EXIST Exception with minor code 0 being thrown. This Exception and minor code indicates that the target transaction does not exist (as opposed to indicating that the transaction was not located in the target runtime). This results in a javax.transaction.HeuristicMixedException in the front end application server and the transaction is forgotten. Later when the subordinate transaction, which is in-doubt, attempts to determine the transaction outcome via replay_completion it is told that the transaction does not exist and therefore, in accordance with the standard 2PC presumed abort optimization, the subordinate transaction branch is rolled back.
Problem conclusion
The synchronization of the transaction service code was modified so that the timing window did not occur and any state associated with peer recovery, including WLM routing for the transaction service endpoints, was correctly cleaned up. The code that processed inbound protocol requests was also modified to perform additional checks which would result in a org.omg.CORBA.TRANSIENT being thrown in certain circumstances. The fix for this APAR is currently targeted for inclusion in fix packs 8.5.5.16 and 9.0.5.0. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
Temporary fix
Comments
APAR Information
APAR number
PH05716
Reported component name
WEBS APP SERV N
Reported component ID
5724H8800
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2018-11-27
Closed date
2019-06-25
Last modified date
2019-06-25
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WEBS APP SERV N
Fixed component ID
5724H8800
Applicable component levels
R850 PSY
UP
R900 PSY
UP
Document Information
Modified date:
27 April 2022