PH68082: CONNECTION HANDLE TO DATABASE REMAINS ACTIVE UNDEFINITELY

APAR status

• Closed as program error.

Error description

• For background, the metrics code is byte code injected into the
  JCA code during initialization of the runtime. This means when
  the MCWrapper.incrementHandleCount() or
  MCWrapper.decrementHandleCount() runs, the corresponding logic
  in ConnectionPoolMonitor.incConnectionHandleCount() and
  ConnectionPoolMonitor.decConnectionHandleCount() is executed
  (this logic is part of the `MCWrapper` class). They are one to
  one.
  
  The issue occurs when high load causes transaction rollbacks,
  which also result in the `J2CA0021E` being thrown. From data
  seen, there have two occurrences of J2CA0021E which correspond
  to two managed connections not decreasing.
  
  We see from PMI trace there is an
  ConnectionPoolMonitor.incConnectionHandleCount() (This
  corresponds to the MCWrapper.incrementHandleCount)  call and no
  corresponding ConnectionPoolMonitor.decConnectionHandleCount()
  (meaning no MCWrapper.decrementHandleCount is called).
  
  We see that the MCWrapper being used is:
  [2025-08-20T21:55:22.258+0100] 000000db id=8015e22e
  com.ibm.ejs.j2c.ConnectionManager    3 Using MCWrapper@4170e35d
  Following the JCA trace and matching with the code we only see
  the following after:
  
  [2025-08-20T21:55:24.861+0100] 000000db id=8015e22e
  com.ibm.ejs.j2c.ConnectionManager                            3
  getConnection failed for using uow is com.ibm.tx.jta.embeddable
  .impl.EmbeddableTransactionImpl@2aa00063#tid=581392204,active=1
  ,suspended=0thread=000000DB,gtrid=00000198c94115750000000122a75
  74cf31a9f1f0376c240c38a3ca74dedc04e7b08ec19tran wrapper id is 0
  [2025-08-20T21:55:24.918+0100] 000000db id=8015e22e
  com.ibm.ejs.j2c.ConnectionManager                            <
  allocateConnection Exit
  
  Notice that the connection handle is held at 1.
  
  The remaining `allocatConnection()` logic under certain
  conditions would decrease the handle count of the MCWrapper. But
  the data under that logic block do not appear. So the handle
  count is never  decreased. MCWrapper.decrementHandleCount()
  doesn't get called so
  ConnectionPoolMonitor.decConnectionHandleCount() doesn't get
  called.
  
  Further more, After this failure, the `4170e35d` MCWrapper ID is
  no longer seen in the remaining logs. The timed out transaction
  leads to the non decreasing of handle count in the
  ConnectionManager.allocateConnection() logic AND the fact that
  the MCWrapper seems to be forgotten and is never cleaned up
  (which looking into the JCA code would decrease the handle
  counts back to 0 leading to the monitor code to reduce the stat
  count).
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:  All users of WebSphere Liberty              *
  ****************************************************************
  * PROBLEM DESCRIPTION: Connection handle count not             *
  *                      decremented on rollback, leaving        *
  *                      orphaned MCWrapper and inflated pool    *
  *                      stats                                   *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  Under high load, transaction rollbacks are triggering
  J2CA0021E errors, and in those cases the connection handle
  count is not being decremented. The PMI trace shows calls to
  ConnectionPoolMonitor.incConnectionHandleCount() (via
  MCWrapper.incrementHandleCount()), but no corresponding call
  to decConnectionHandleCount().
  Specifically, for the affected MCWrapper (example:
  MCWrapper@4170e35d), the handle count remains at 1 after
  allocateConnection() fails. The expected cleanup logic that
  would call MCWrapper.decrementHandleCount() is never reached,
  and the wrapper ID disappears from subsequent logs. As a
  result, the connection handle count is left inflated, and the
  monitor statistics never recover.
  Transaction rollback paths cause incomplete
  execution of allocateConnection() cleanup logic, leaving
  orphaned MCWrapper instances with non-decremented handle
  counts and inconsistent pool monitoring stats.
  

Problem conclusion

• The issue occurred because decrementHandleCount was never
  invoked when a resource exception happened during
  allocateConnection. This left the connection handle count
  inflated and the MCWrapper untracked. The fix introduces a
  call to decrementHandleCount in the exception path, ensuring
  handle counts are properly reduced and monitoring statistics
  remain accurate.
  
  Open Liberty GitHub issue:
  https://github.com/OpenLiberty/open-liberty/issues/32789
  
  The fix for this APAR is targeted for inclusion in fix pack
  25.0.0.10.
  
  For more information, see 'Recommended Updates for WebSphere
  Application Server':
  https://www.ibm.com/support/pages/node/715553
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  LIBERTY PROFILE

• Fixed component ID

  5724J0814

Applicable component levels