Meet the experts: Gang Chen on transactions

This question and answer article features WebSphere consultant Gang Chen who answers questions about transactions within J2EE applications and WebSphere Application Server V5 and V6.

Share:

Gang Chen, Consulting I/T Specialist, IBM

Photo: Gang ChenGang Chen is a Consulting I/T specialist in IBM Software Services for WebSphere (ISSW), servicing the New York Metro area. Gang is an expert in helping enterprise customers with their complex transactional requirements. He works with several major Wall Street customers in building mission critical transactional systems.



23 February 2005

Introduction

In this article, Gang focuses on XA distributed transactions and on the inner workings of the two-phase commit protocol. He also focuses on responsibilities and features of various transactional components within WebSphere® Application Server, recovery processes for various transactional failure scenarios, high availability features of the transactional system within WebSphere Application Server (hereafter called Application Server), and how to improve performance and scalability of applications with complex transactional scenarios.

Question: I need WebSphere to validate users in two different LDAP servers. These two servers are not only two different realms or domains, but they are also in different machines. I have been looking for information and I have found that I need to use a custom security configuration. The example I found is using the com.ibm.websphere.security.FileRegistrySample file. However, I need to access two active directories.

Answer: You need to write a custom user registry (interface is UserRegistry). You can find information on how to write one in the WebSphere Application Server Information Center. Solving your particular problem will be a substantial effort. You can consult IBM Software Services for WebSphere, which contains reusable assets that may help solve this problem.

Question: For several years we have been reusing legacy RPG programs on iSeries via JDBC calls from within WebSphere. Some of these transactions have been coordinated with JMS/MQ transactions using an XA datasource. We recently came unstuck when upgrading to V5R3 on iSeries, where it appears that IBM have tightened up on XA compliance by closing all open files after an XA commit. This includes closing files that were opened by the legacy RPG. Because the RPG expects the files to be open on the second and subsequent calls to it, under V5R3, it now fails! My question is why did IBM make this choice? To developers like ourselves, it makes legacy reuse harder, if not impossible without significantly modifying the RPG code. At the very least I would have expected some kind of LEGACY=TRUE property to be available on the datasource. Will IBM consider something like this? I think it's also wrong for the XA driver to close files, which it did not open in the first place. We have had no choice but to stop using the XA driver, and therefore lose all of the benefits that we previously got from it.

Answer: The behavior you described is likely fixed by PTF SI17049 (APAR SE18368). When applying this PTF, make sure that you also have the latest levels of the following cumulative PTFs:

  • SF99529
  • SF99503
  • SF99286
  • SF99285
  • SF99269

Question: Does WebSphere V5 support the Read-Only optimization of the 2PC protocol?

Answer: Yes, WebSphere Application Server V5 supports the Read-Only feature for the resource providers that it also supports the Read-Only optimization defined by the XA specification. The following is a detailed description of how it works.

WebSphere transaction manager sends the prepare request to all the resource managers in XA phase 1 (prepare phase) in a WebSphere global transaction. A resource manager, for instance DB2®, detects that it is not asked to update the shared data in this transaction branch. That is, the application performed a read-only action on this resource manager. When responding to the transaction manager of the prepared outcome, the resource manager asserts the read-only information in the prepared vote. In phase 2 (commit phase), WebSphere transaction manager skips the communication with this resource manager, and it will not disk force or log any activities related to this resource manager. With this optimization, global transaction reduces the network traffic and amount of data written to the transaction log files. This implies better performance.

Question: I am trying to deploy an application on WebSphere Application Server V5.1 using Oracle® as my backend. I am using CMP beans. While deploying the application on V5.1, I am getting "Access Intent Error". The error is:

[1/18/05 16:14:45:768 IST] 3c9a7781 ConcreteBeanC E PMGR1001E: No such
DataAccessSpec :FindByAccessorAndObjectType. The backend datastore does
not support the SQLStatement needed by this AccessIntent: access intent:
(pessimistic update-weakestLockAtLoad)(collections: transaction/25)
(resource manager prefetch: 0).

[1/18/05 16:14:45:799 IST] 3c9a7781 ConcreteBeanC E PMGR0000E: Call stack:

com.ibm.ws.ejbpersistence.utilpm.PersistenceManagerException: PMGR1001E:
No such DataAccessSpec :FindByAccessorAndObjectType. The backend
datastore does not support the SQLStatement needed by this AccessIntent: access 
intent: (pessimistic update-weakestLockAtLoad)(collections: transaction/25) 
(resource manager prefetch: 0).
    at com.ibm.ws.ejbpersistence.beanextensions.ConcreteBeanClassExtensionImpl.
    createPMException(Unknown Source)
    at com.ibm.ws.ejbpersistence.beanextensions.ConcreteBeanClassExtensionImpl.
    getDataAccessSpec(Unknown Source)
    at com.ibm.ws.ejbpersistence.beanextensions.ConcreteBeanStatefulInstanceExtensionImpl.
    getDataAccessSpec(Unknown Source)
    at com.ibm.ws.ejbpersistence.beanextensions.ConcreteBeanStatefulInstanceExtensionImpl.
    getInputRecord(Unknown Source)

After Googling™ and referring some documents which are published on the IBM site, I found that one needs to apply "Access Intent policy". This access intent policy goes as a part of ibm-ejb-jar-ext.xmi. So I applied the access intent policy as "wsPessimisticRead". After re-deploying the application, I am getting following error:

[1/27/05 16:46:24:211 IST] 3c98a101 Helpers  W NMSV0605W: A Reference object looked 
up from the context "java:" with the name "comp/env/ejb/ContainerLocal" was sent to the 
JNDI Naming Manager and an exception resulted. Reference data follows:

Reference Factory Class Name: com.ibm.ws.naming.util.IndirectJndiLookupObjectFactory
Reference Factory Class Location URLs: <null>
Reference Class Name: java.lang.Object
Type: JndiLookupInfo
Content: JndiLookupInfo: jndiName="local:ejb/ejb/com/quest/metadata/ejb/ContainerLocal";
 providerURL=""; initialContextFactory=""

Exception data follows:

javax.naming.NameNotFoundException: Name ejb/ejb not found in context "local:".
    at com.ibm.ws.naming.ipbase.NameSpace.getParentCtxInternal
    (NameSpace.java(Compiled Code))
    at com.ibm.ws.naming.ipbase.NameSpace.lookupInternal(NameSpace.java:1009)
    at com.ibm.ws.naming.ipbase.NameSpace.lookup(NameSpace.java:932)
    at com.ibm.ws.naming.urlbase.UrlContextImpl.lookup(UrlContextImpl.java:1261)
    at com.ibm.ws.naming.urlbase.UrlContextImpl.lookup(UrlContextImpl.java:1253)
    at com.ibm.ws.naming.urlbase.UrlContextImpl.lookup(UrlContextImpl.java:1307)
    at com.ibm.ws.naming.jndicos.CNContextImpl.lookupExt(CNContextImpl.java:1185)

Are the "Access Intent error" and "JNDI Error" related to each other in some context? Do I need to configure something more apart from applying access intent policy?

Answer: Let's address your original problem first. If you have not configured access intent, all of your data is accessed under the default access intent policy (wsPessimisticUpdate-WeakestLockAtLoad). On Oracle databases, the weakest lock is an update lock. This means that the SQL query must contain a USE AND KEEP UPDATE LOCKS clause. However, not every SQL statement necessarily supports USE AND KEEP UPDATE LOCKS. For example, if the query is being run against multiple tables in a join, USE AND KEEP UPDATE LOCKS is not supported. To avoid this problem, you should apply an access intent policy that supports optimistic concurrency. If update access is not required, use wsPessimisticRead or wsOptimisticRead.

Access intent is a WebSphere Application Server run-time service that enables you to more precisely manage an application's persistence. The access intent service defines a set of declarative annotations used by the Enterprise JavaBeans (EJB) container and its agents to make performance optimizations for entity bean access. It has no direct relationship with the JNDI reference of your Enterprise Java Beans. So, I don't think the Access Intent errors are related to the JNDI problem. Therefore, I would recommend you to review the EJB deployment descriptor and verify the EJB bindings during the deployment time. Specifically, check the "Reference" section of your EJB deployment descriptor (both EJB reference and Resource reference). The problem is most likely a configuration issue.

Question: We use WebSphere Application Server V5 and DB2 V7 on OS/390. Can we use Entity-EJB and JDBC access in one transaction? The DB2 connection is made via a DB2 client.

Answer: In general, you can invoke Entity-EJB (both CMP and BMP beans) and JDBC access in one transaction. There are a few things you should pay attention for the above transaction scenario. First, Entity-EJB only supports the Container Managed Transaction (CMT). This implies that you should not use the Bean Managed Transaction (BMT) for your JDBC access. The transaction boundary for all the participants needs to be defined by the WebSphere container through the Bean transaction attributes of the deployment descriptor, and calling the JDBC native API's commit/rollback functions is illegal in this scenario. Second, you need to consider the scenario where the application accesses the transactional data concurrently. This requires the careful design of isolation conditions, database locking, and transaction isolation level. For detailed description about the EJB transaction and Data isolation, refer to Chapter 8 of the IBM Redbook, EJB 2.0 Development with WebSphere Studio Application Developer.

Question: I am trying to use J9 JRE.

  1. Is J9 targeted only for devices like PDA? Or, can it be used as a replacement for JRE from Sun® Microsystem®?
  2. I have an application using Jena jar files together with Xerces jar. Works perfectly with the JRE provided from Sun, but gives the following error when I try with J9 JRE:
    Exception in thread "main" java.lang.NoSuchFieldError: org.apache.xerces.impl.dv
    .ValidatedInfo: field actualValueType not found
    at com.hp.hpl.jena.datatypes.xsd.XSDDatatype.convertValidatedDataValue(X
    SDDatatype.java:370)

I checked the xerces jar file, but the actualValueType that J9 says it cannot find is defined (and also works with JRE from Sun).

Answer:

  1. The J9 JVM was originally written for J2ME, but is now becoming available for J2SE and J2EE. It is not currently available, but will be shortly.
  2. This is most likely a bug in J9. Please report the problem to IBM support through the PMR process.

Question: We need the cursor holdability feature from JDBC 3.0. Our usage: We open a transaction and read a big result set. Then we rollback this transaction. After this, we loop over the big result. For every result, we open a transaction, do business, and commit or rollback the transaction. Our environment is WebSphere Application Server 5.1, Oracle 9i, DB2UDB or DB2Iseries database. We use thin JDBC drivers in an XA transaction. This is a problem with Oracle 9i. We use the original Oracle 10g driver, which does not support the cursor holdability feature. Now we do some tests with the i-net ORANXO JDBC 3.0 driver.

Questions:

  • Why does the commit or rollback of a transaction close the result set? We want to loop over big results for reorganization (1000000 result sets).
  • Do you any solution to correct transaction handling with that much data?
  • Can I mix local transaction and XA transaction in WebSphere? I know I cannot, but this could be the workaround for me because with the local transaction, the cursor will have holdability.
  • What is the difference between a local transaction and an XA transaction on the Application Server level?

Answer: I cannot comment much about Oracle's JDBC driver implementation for handling the result set at the end of an XA transaction. One possible reason is that the resource manager wants to avoid the situation, where it has to hold some resource locks if the result set is held across the transaction boundaries. In general, the resource locks should be released upon transaction completion.

You cannot mix local transaction with XA transactions in WebSphere. However, if you mean using one-phase resource with XA resources in the same WebSphere global transaction, the answer is yes. You can use the Last Participant Support (LPS), which is available from WebSphere Business Integration Server Foundation Version 5.1 and WebSphere Application Server Version 6.0. LPS allows you to mix two-phase resources with one one-phase resource in the same global transaction.

A local transaction is a unit of work between a WebSphere connection and the resource manager. Java™ Transaction Service (JTS) is not involved in the local transaction, but JTS supplies an "extra" function to control what happens if a connection is not closed. This is called Local Transaction Containment (LTC). On the other hand, a global transaction is where JTS acts as the coordinator between WebSphere and the various resource managers to ensure that all resources involved are completed in the same direction. JTS also adds recovery after server failure, of course.

Question: I'm new to this WebSphere, so please help me out. I understand WebSphere Portal uses IBM HTTP Server that has a cgi-bin in its directory. So I assume that it's possible to run Perl CGI scripts with this server. My server machine hosts a portal so typing the hostname directs the browser to the WebSphere Portal root directory (/wps/portal/). How do I remote access through http the cgi-bin under the IBMHTTPServer directory?

Answer: I would assume that the question asks how to use CGI in IBM HTTP Server, even the scenario is under WebSphere Portal. The confusing part is the access URL from the browser (/wps/portal) and how to direct your request to the deployed CGI scripts. In IBM HTTP Server, the translation of the browser URL to the Web content is handled by the HTTP Server Mapping definition, which is configured through the HTTP Server Administration Console. You need to define such mappings before using the CGI scripts. This is a two-step process:

  1. Deploy your CGI scripts to the IBM HTTP Server CGI directory, The default directory is C:\Program Files\IBM HTTP Server\cgi-bin if you are using Windows™ based OS.
  2. Configure the mapping for your CGI applications using the Administration Console. This step requires you to create directory aliases, which is simply a mapping of the URL path to the server directory, for example, URL path /cgi-bin maps to directory name C:\Program Files\IBM HTTP Server\cgi-bin. Some other configurations are required also for CGI support. For more information, see the IBM HTTP Server Information Center.

Question: We run WebSphere Application Server 5.0.2 and when we start the subsystem QEJBAS5, the server is started. We have other jobs which need to run when the server is ready for ebusiness. These are tasks that can only be started when the server is up and running. Where do we need to add some scripting in order to start those tasks once the server is up and running?

Answer: I can think of two ways to accomplish that:

  1. From the iSeries work management point of view, you should add the jobs into the Application Server subsystem: QEJBAS5 as auto start jobs by the ADDAJE (Add Auto-start Job Entry) command. Every time the subsystem is started, the jobs will be called. Be careful for the situation where the jobs are called and the Application Server is not ready yet. If the jobs are only jobs that are waiting for requests from the Application Sever, there is no communication from the jobs to the Application Server. This situation is fine.
  2. From the Application Server point of view, you may need to develop a Java application to submit jobs from the Application Server. Then deploy the application into the Application Server defined as auto-start application. When the Application Server gets ready, it calls the application, then submits the jobs to iSeries.

Question: I am developing a software for a client who is using WebSphere 5.1. I can create the EAR file using the ANT tool. I need to automate the deployment of the ear file also thru a Java program. Can you give me some information on how to do this?

Answer: You certainly can automate the deployment process of Enterprise Applications (EAR), In fact, it is the recommended practice that has been used by many customers. WebSphere Application Server V5.1 provides a powerful scripting tool, wsadmin, which is a perfect candidate to construct the deployment automation process. It carries the same functions as the WebSphere Administration Console does, including installing an enterprise application. The wsadmin utility is written in Java using JMX API. To make it perfect, you can use the ANT and wsadmin together to provide an end-to-end automated deployment solution.

If you want to implement the automation process in Java, you can use the JMX API. WebSphere system management is built on top of JMX specification. The server management engine provides sets of interfaces that you can connect to the embedded JMX server and perform the administration tasks, such as installing an EAR application. Your Java application will invoke the JMX client interface provided by WebSphere. This is the same programming model used by the wsadmin utility. For more information about using wsadmin and JMX, see the WebSphere Information Center.

Question: We are trying to find a solution of getting WebSphere Application Server configuration (v 5.x) in a tabular form for cross verification, such as the following listing:

  1. Clusters
  2. Application servers
    1. Log file location
    2. JVM settings
    3. End points
    4. Log file locations and settings
  3. Enterprise applications
    1. Associated application server
    2. Whatever important general properties possible
    3. Session management
  4. Datasources
    1. All possible properties
  5. Virtual hosts

Is it possible through some ready made script? Is there any tool for that? How can we get this?

Answer: This is a similar question to the previous question, both are seeking automated solution for WebSphere system administration, either installing applications or gathering configuration information. You can consider wsadmin and JMX applications to perform the task you require. Different customers may have totally different system configuration. It is not practical for having such a universal information gathering tool bundled with the product release. I am not aware of a ready-made script that is publicly available for your situation. You can consult IBM Software Services for WebSphere, which contains reusable assets that may meet your requirements.

Question: What about WS-Transaction? Any other way to integrate transactions between COM+/.Net and WebSphere Application Server?

Answer: The Web services transaction specification consists of three subsets: WS-Coordination (WS-COOR), WS-Atomic Transaction (WS-AT), and WS-BusinessActivity (WS-BA). The IBM developerWorks Web site hosts a portal of WS-Transaction information, such as the downloadable specifications. WebSphere Application Server V6.0 supports the Web Services Atomic Transaction that is an implementation of WS-AT and WS-COOR. It enables distributed Web service applications, and the resources they use, to take part in distributed global transactions. Besides Web services, you can consider the WebSphere CORBA application as an integration solution for COM+/.NET. However, we highly recommend Web services as a standard integration between WebSphere and .NET.

Question: In WebSphere, can we say that we have a way to stop a transaction like TP Monitor (CICS/Tuxedo)? If we do not provide that function, WebLogic neither?

Answer: Yes, you can stop an in-flight transaction. WebSphere Administration Console provides a system management task, "Manage transactions", which allows you to manage the transactions and to finish (stop/abort) a transaction as shown in Figure 1.

Figure 1. Manage transactions
Manage transactions

If you choose to finish a transaction manually on an application server, it is recorded as having completed in the transaction service logs for that server. It will not be eligible for recovery during server start up. If you finish a transaction, you are responsible for cleaning up any in-doubt transactions on the resource managers affected. For more information on how to use this service, see "Managing active transactions" in the WebSphere Information Center.

Question: A question about transaction management. I'm using WebSphere Application Server 4.0 and MQ Series for backend integration using xml messaging. Now we have the case of multiple backends, an application that needs to be created on multiple backends, all transaction issues including consistency should be there. Please advise and elaborate on what is the best solution to guarantee transaction management on the multiple messages sent to multiple backends so that at least if one message failed, you can reverse (roll back) the effect of previous messages. Can WebSphere transactional services help on this? Can MQ Series help on this? Or, do we need for example, a programmatic approach, and specific xml messages to handle transaction management and tell the backend to reverse the last message effect?

Answer: When an application needs to access more than one resource (backend) and needs to guarantee job consistency, a global transaction is required to coordinate the interactions among the resources, application and transaction manager, as defined by the XA specification. WebSphere Application Server 4.x is compliant with the XA specification. It uses the two-phase commit protocol to guarantee the All-or-Nothing semantics that either all the resources commit the change permanently or none of them precede the update (rollback). Such a transaction scenario requires the participating resource managers, like WebSphere MQ, to support the XA specification and provides the XA interface for a two-phase commit. The good news is that WebSphere MQ does support the XA specification and two-phase commit protocol, and it perfectly coordinates with WebSphere Application Server when handling global transactions.

There are two ways of using WebSphere global transaction, as defined by J2EE standard: Container Managed Transaction (CMT) and Bean Managed Transaction (BMT). The CMT is the easiest and the recommended approach for transaction management. You do not need to write any code to control the transaction behavior. Instead, the J2EE container, WebSphere in this case, controls all the transaction logic. It is a service provided by WebSphere.

My tutorial, WebSphere Distributed Transaction Support for a Generic JMS Provider, provides steps on how to configure a WebSphere global transaction that accesses a DB2 database and a JMS provider within a unit of work. Sample code is available.

Question: What are the problems faced while migrating from WAS 4.0.5 to WAS 5.0?

Answer: This is a generic question that covers many areas. I recommend the IBM Redbook, Migrating to WebSphere V5.0: An End-to-End Migration Guide.

From an end user's point of view, the transaction service provided by both WebSphere versions are almost unchanged, even the underlying implementation of transaction engine has undergone certain major improvements in the areas such as high availability, performance, and so on. However, the WebSphere Business Integration Server Foundation (WBI SF) Version 5.1 has extended the transaction service beyond the J2EE standard. It introduces two transaction-related Programming Model Extensions (PME):

  1. Last Participant Support (LPS): you can mix a one one-phase resource with multiple two-phase resources within the same WebSphere global transaction.
  2. ActivitySession : It provides both a way to extend transaction boundaries beyond method invocation and to specify EJB activation and passivation by means of these new boundaries. ActivitySession becomes a new entity in the transactional scenario defining new boundaries for a unit of work.

Question: I am working on an application that moves data that is persisted in an LDAP server, into our database for 24 hour caching. This is a batch process that uses WebSphere MQ, and MDB to drive the batch processing. There are a total of 90,000 messages that get transacted throughout this process. Currently the process has started to take up to 5 hours to complete, and my task is to make sure the application can run in a clustered, multithreaded environment. The application was originally configured with one listener port configured on one server, and only one session and message configured on that listener port. I have been testing different combination of settings. So far, a maximum of 20 sessions with 1 message has been the best performance, reducing the transaction time by 70%. What is the maximum recommended listener port setting, and the ramification, if any, for turning up the maximum session to high?

Answer: There is no golden rule to define the listener port settings for the best performance. In fact, performance is always a complex topic that requires significant testing and tuning. If we really want to play the "Rule of Thumb", I have two recommendations to address your problem: scaling the system and tuning the queue of components.

If you can distribute your load to more WebSphere servers, your application throughput will be increased to handle more incoming messages. The frequently used technique is scaling: horizontal and vertical. If your server still has a lot of free CPU cycle, the method you want to try first will be vertical scaling, which multiple server cluster members work in parallel to process the incoming messages. Once you have bottlenecked the vertical scaled server machine, you can consider to expand your application cluster to other machines, horizontal scale.

A queue of components, not the JMS queue, is a group of interconnected queues that represent various components. You should consider them as an integrated system in performance tuning. In your case, the queue may start from the LDAP server, to WebSphere MQ, to WebSphere JMS listener, to MDB, and finally, to the destination database. Each component has their own optimized process capability and they are interconnected with each other to impact the overall performance. You need to understand this queue system and find out the bottleneck in the pipe. For example, if the database can only process 10 messages at time, increasing the JMS listener max session from 20 to 30 will not improve performance.

I am curious about how the application handles the transaction in your case. Using the WebSphere global transaction, where two-phase commit protocol is in place, will significantly decrease the overall performance. Unless you cannot avoid processing the incoming messages and database update in a single unit of work, consider using local transaction for performance reasons.

Acknowledgments

The author wishes to thank the following people for their help and effort on this article:

  • Roland Barcia
  • Keys Botzum
  • Neville Brailsford
  • Harry Tian
  • Chris Bailey
  • Bobby Woolf

Resources

About Meet the experts

Meet the experts is a monthly feature on the developerWorks WebSphere Web site. We give you access to the best minds in IBM WebSphere, product experts who are waiting to answer your questions. You submit the questions, and we post answers to the most popular questions.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=56738
ArticleTitle=Meet the experts: Gang Chen on transactions
publish-date=02232005