IBM Support

Trace Analysis: WTRN0063E An illegal attempt to commit a one phase capable resource with existing two phase capable resources has occurred

Technical Blog Post


Abstract

Trace Analysis: WTRN0063E An illegal attempt to commit a one phase capable resource with existing two phase capable resources has occurred

Body

 

It amazes me that the smallest configuration missteps can cause the biggest problems in an environment. The good thing is most of the time small changes are easy to fix. In this blog, I am going to teach you how to understand and resolve the infamous WTRN0063E. Thus, you'll go from zero to hero next time this exception is thrown in the SystemOut.log file. You'll be able to explain the issue and resolution to the application developer, the System Admin, and your manager with ease and without opening a PMR (Problem Record) with IBM.
 
 
Scenario:
 
You are in the process of a migration/upgrade to a new release of WebSphere Application Server. The application was recompiled to run on the later version of IBM Java. The application developers stated the core functionality will not change. The application development team starts testing the application and the WTRN0063E exception is sometimes thrown in the environment. The exception causes the transaction to fail and rollback. The business wanted answers yesterday to why the transaction is failing in the new environment.
 
What do you do?
 
 
Understanding:
 
1. Let's take a step back and review the full exception message:
 
[12/31/99 23:52:12:828 EDT] 0000006b RegisteredRes E   WTRN0063E: An illegal attempt to commit a one phase capable resource with existing two phase capable resources has occurred.
 
The exception being thrown tells us the application is leveraging global transactions, the primary purpose is to enlist more than one resource in the same transaction. Note, the application controls starting the transaction, looking up resources, and calling commit on the transaction when the work has been completed. Next, the message states, the application attempted to enlist a one-phase capable resource with two-phase capable resources. Thus, the application successfully registered one or more two-phase resources before failing when registering the one-phase capable resource. Note the "RegisteredRes" in the first part of the exception.
 
 
2. A one-phase and two-phase what?!?
 
Applications use resource providers, such a JDBC data sources or connection factories that can be either two-phase(XA) or one-phase(non-XA). A non-XA resource can support only local transactions, but an XA resource can support global transactions, as well as local transactions.
 
If an application uses two or more resource providers that support only local transactions, atomicity cannot be assured because of the one-phase nature of these resources. To ensure atomic behavior, the application must use resources that support XA coordination and must access those resources in a global transaction.
 
Ah ha, we now know that one of the resources the application calls has to be one-phase resource. WebSphere noticed the application was mixing XA and non-XA resources in the same transaction and threw the exception.
 
Perfect! It's the application's fault and the development team needs to resolve. Well, not quite. We need some more information from all involved.
 
 
Asking three right questions and why?
 
1. What are the JNDI names of the resources the application is calling?
 
Why Ask: We always want to ensure all the resources the application expects to call are configured to be two-phase(XA). This could be a extremely time consuming or difficult question to answer depending on the complexity of the application. It's not a requirement for the development team to answer in order to resolve the issue. It's also valuable information to have when reviewing a transaction trace.
 
 
2. What are the databases involved and are they configured for XA transactions?
 
Why Ask: We want to ensure the DBA has configured the database to support XA transactions. If the DBA states, they have not, then there is a disconnect between the application team and the DBA team. The DBA will need to configure the database to support XA transaction, which may be difficult. Alternatively, the application team could, you know, rewrite the entire application to not use global transactions. That's a joke if you didn't catch it.
 
 
3. Do we have Last Participant Support(LPS) enabled?
 
Why Ask: The $100 dollar question will be why did it work in environment A and why is it not working in environment B. If LPS was enabled in environment A and it is currently not enabled in environment B, then you know why the environment performing differently.
 
 
Those are the first three questions everyone involved in the issue should discuss and review. Again, these questions are not required to determine root cause, but they are strongly recommended to assist in faster resolution.
 
You are probably thinking to yourself that I forgot the most obvious question. What is the JNDI name of the one-phase(non-XA) capable resource causing the exception in the environment? This is an excellent question, but it can not be answered with only the SystemOut.log files. To determine the one-phase(non-XA) resource, a transaction trace needs to be analyzed. Thus, we would need to apply a trace and reproduce the issue. Depending on the environment, this could be easy or nearly impossible. It is always best to determine any configuration issues before applying a transaction trace.
 
 
Resolution:
 
1. Check the resources used by the application in WebSphere to ensure they are configured for two-phase(XA):
 
The implementation class name determines if a datasource is non-XA or XA. Login into the administrative console, click Resources > JDBC > JDBC providers > JDBC_provider. Then note the value for the implementation class name property.
 
Below are some of the one-phase implementation JDBC driver classes by vendor:
 

image
(click image to enlarge)

 
 
Cross check any data sources with a JDBC provider configured with a one-phase implementation against the data sources used by the application. If there is a match, then the exception was related to a misconfiguration of the JDBC provider. Thus, the JDBC provider needs to be recreated with a two-phase implementation JDBC driver class. It is recommended to recreate the JDBC provider and related data sources instead of updating the implementation class.
 
Note: If you modify the implementation class name of the JDBC provider after you have created the provider, you might disconnect the provider from the template used to create it. As a result, data sources created from this JDBC provider do not have an associated template; you must manually configure a working data source through setting custom properties.
 
 
2. Enable Last Participant Support:
 
Last participant support enables the use of a single one-phase commit capable resource with any number of two-phase commit capable resources in the same global transaction. You can have multiple interactions that involve the one-phase commit resource in the same transaction, but only one such resource can be involved.
 
At transaction commit, the two-phase commit resources are prepared first using the two-phase commit protocol, and if this is successful, the one-phase commit-resource is then called to commit. The two-phase commit resources are then committed or rolled back, depending on the response of the one-phase commit resource.
 
Enabling LPS maybe resolve the issue if the databases are configured for two-phase transactions and the application is attempting to enlist a single one-phase resources in the same global transaction. As outlined earlier, if this property is enabled in another environment, it would explain why the transactions are failing in a new environment.
 
Once enabled, this property specifies to all applications to accept the possibility of a heuristic hazard occurring in a two-phase transaction that contains a one-phase resource. Thus, it is recommended to gather trace before applying this property for faster analysis and root cause.
 
In WebSphere Application Server version 8.0 or higher, Last Participant Support can be enabled at the server. In WebSphere Application server version 7.0, Last Participant Support can be enabled only at the application level.
 
Please use the following steps to enable the property via the administrative console for WebSphere Application Server version 8.0 or higher:
  1. In the administrative console, click Servers > Server Types > WebSphere application servers > server_name. The properties of the application server, server_name, are displayed in the content pane.
  2. Click [Container Settings] Container Services > Transaction Service. The Transaction Service settings page is displayed.
  3. Ensure that the Configuration tab is displayed.
  4. Check the box next to "Accept heuristic hazard"
  5. Restart the application server.
 
Please use the following steps to enable the property via the administrative console for WebSphere Application Server version 7.0:
  1. In administrative console page, click Applications > Application Types > WebSphere enterprise applications > application_name. Under Detailed Properties, click Last participant support extension.
  2. Check the box next to "Accept heuristic hazard"
 
Note: Alternatively, LPS can be enabled per the application via deployment descriptors. See product documentation section Assembling an application to use one-phase and two-phase commit resources in the same transaction.
 
 
3. Capture and Analyze a Transaction Trace:
 
Enable the following trace string from JVM startup and reproduce the WTRN0063E exception. There are several ways to analyze the trace file and determine the resource causing the issue. I prefer reviewing the transaction life cycle from beginning to the exception. It's a habit that is beneficial in analyzing other types of transactions issues such as transaction timeouts.
 
Trace String:
Transaction=all:EJBContainer=all:WAS.j2c=all:RRA=all
 
 
First, we need to find the transaction associated with the WTRN0063E exception.
 
// Find the WTRN0063E exception and note the thread id(0000006b). Next, the WTRN0086I provides the application, module, and bean names involved with the transaction. This is valuable information for the application developers to assist in isolating the components of the application involved. In this case, the application, module, and bean names are as follows:
 
// Application name: IBMWTRN
// Module name: illegal-trans-ejb.jar
// Bean name: TwoPhaseListenerBean
 
[12/31/99 23:52:12:828 EDT] 0000006b RegisteredRes E   WTRN0063E: An illegal attempt to commit a one phase capable resource with existing two phase capable resources has occurred.
[12/31/99 23:52:12:828 EDT] 0000006b RegisteredRes >  logDiagnostics Entry
                                 1
[12/31/99 23:52:12:828 EDT] 0000006b RegisteredRes E   WTRN0086I: XAException encountered during prepare phase for transaction IBMWTRN#illegal-trans-ejb.jar#TwoPhaseListenerBean
0000014E97B52FBA000000015ED2C4F4A66D09710929EC41363582D70A7C7FDDAEE7AFFE0000014E97B52FBA000000015ED2C4F4A66D09710929EC41363582D70A7C7FDDAEE7AFFE00000001. Local resources follow.
 
// A diagnoseResource entry provides a com.ibm.ws.Transaction.JTA.OnePhaseResourceImpl with handle cbfbf69b. You will use this one phase resource entry to find the transaction id.
 
[12/31/99 23:52:12:830 EDT] 0000006b RegisteredRes >  diagnoseResource Entry
                                 com.ibm.ws.Transaction.JTA.OnePhaseResourceImpl@cbfbf69b
 
// The following entry will be written after the diagnoseResource Entry above. Please note the transaction id of 1590871300
[12/31/99 23:52:17:024 EDT] 0000006b RegisteredRes 1   (SPI) RESOURCE registered with Transaction. TX: 1590871300, Resource: com.ibm.ws.Transaction.JTA.OnePhaseResourceImpl@cbfbf69b
 
// Search the trace files for the transaction id, 1590871300, until you find following entry with a BEGIN in the message.
// This denotes the beginning of the transaction.
[12/31/99 23:52:13:797 EDT] 0000006b TransactionIm 1   (SPI) Transaction BEGIN occurred for TX: 1590871300                                      
                                                                       
// Since we found the beginning of the transaction, we can isolate all of the work occurring on thread 0000006b.
// Registering a resource with the Unit of Work(UOW) /Transaction      
[12/31/99 23:52:17:022 EDT] 0000006b TranManagerIm <  getUOWCoord Exit  
                                                                       
com.ibm.ws.tx.jta.TransactionImpl@2439640a#tid=1590871300              
                                                                       
// The application may register several resources. You want to follow the thread taking note of the entries.
// In this case, we are registering the resource with JNDI name jdbc/OnePhaseDS   
[12/31/99 23:52:17:022 EDT] 0000006b WSRdbManagedC >  lazyEnlist Entry  
                                 WSRdbManagedConnectionImpl@2e43cb45   
                                 [ConnectionManager]@25119451          
JNDI Name <jdbc/OnePhaseDS>                                         
                                                                       
// Note the proceeding entry called "enlistOnePhase".
// Thus, the resource with JNDI name jdbc/OnePhaseDS is defined as one phase resource instead of two phase resource
[12/31/99 23:52:17:023 EDT] 0000006b EmbeddableTra >  enlistOnePhase    
Entry                                                                  
                                                                       
com.ibm.ws.tx.jta.TransactionImpl@2439640a#tid=1590871300              
                                 LocalTransactionWrapper@:cbfbf69b     
localTransaction:com.ibm.ws.rsadapter.spi.WSRdbSpiLocalTransactionImpl@6
62b24e5  enlisted:falseHas Tran Rolled Back = false                    
registeredForSynctruemcWrapper.hashcode()488826963                     
   
// This is the same message we used earlier to determine the transaction id.                                                                
[12/31/99 23:52:17:024 EDT] 0000006b RegisteredRes 1   (SPI) RESOURCE   
registered with Transaction. TX: 1590871300, Resource:                 
com.ibm.ws.Transaction.JTA.OnePhaseResourceImpl@cbfbf69b               
                                                                       
// Another resource is registered with the UOW/transaction with JNDI name jdbc/ResourceTwo. It does not matter if this resource is two-phase or one phase,
// the transaction will fail because a one-phase has already been registered with the transaction. Thus, it is illegal for the application to enlist any more resources(XA or non-XA) in the same transaction.
[12/31/99 23:52:17:030 EDT] 0000006b TranManagerIm <  getUOWCoord Exit  
                                                                       
com.ibm.ws.tx.jta.TransactionImpl@2439640a#tid=1590871300              
[12/31/99 23:52:17:031 EDT] 0000006b ConnectionMan 3   This CM is       
[ConnectionManager]@940be997                                           
JNDI Name <jdbc/ResourceTwo>                                          
shareable <true>        
 
 
We have identified that the application is enlisting a one-phase resource in a global transaction. To resolve the issue, we should reconfigure the resources defined in WebSphere Application Server to be XA as expected by the application.
 
For fun, let me know the significance of the time stamps from the trace excerpts in the comments below. In addition, please let me know if you have any questions or future trace analysis requests in the comments.

 

 

title image (modified) credit: (cc) Some rights reserved by geralt

 

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"","label":""},"Component":"","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"","label":""}}]

UID

ibm11080465

Page Feedback