PI09569: WHEN BACKEND IS STALE/SLOW UNDER LOAD THEN GOES BACK TO NORMAL SERVER CANNOT RECOVER UNTIL RESTART

Download for IBM Worklight V6.0.0 Fix Pack 2 and IBM Mobile Foundation V6.0.0 Fix Pack 2
Download for IBM Worklight V6.1.0 Fix Pack 1
Download the cumulative interim fix for IBM Worklight V5.0.6 Fix Pack 2 and IBM Mobile Foundation V5.0.6 Fix Pack 2

APAR status

• Closed as program error.

Error description

• When a timeout is defined for an adapter procedure in the
  adapter.xml
  this timeout is not enforced on the HTTP request that eventually
  gets executed. If the Worklight server is under substantial load
  while backend is down (or becomes very slow) - this results in
  the following scenario:
  
  1. Clients correctly get timeout error messages according to the
  timeout
  defined in theadapter.xml
  2. HTTP connections from the Worklight server to the backend get
  accumulated
  while waiting for response from the backend to no avail
  3. If the backend goes back to normal, the Worklight server does
  not recover
  i.e. connections are not properly released back to connection
  pool
  and the server does not connect to the backend anymore.
  4. Only a restart of the Worklight server solves this as of now
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * Administrators or developers that use HTTP adapters in their *
  * environment.                                                 *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * Although timeouts are enforced correctly between the         *
  * Worklight client and server, for adapter procedure           *
  * invocations (be it defined explicitly in adapter.xml or the  *
  * default 30 seconds if not defined), this timeout is not      *
  * enforced on the socket and connections opened between the    *
  * Workligh server and the HTTP backend. Therefore on under     *
  * load and while having a slow/unresponsive backend, these     *
  * connections/sockets stay open even after the Worklight       *
  * server has responded to the client request with a timeout    *
  * message.  The open connections accumulate and the adapter    *
  * becomes stale. Then, when the backend comes back to normal   *
  * operation, the Worklight server does not recover until the   *
  * server is restarted.                                         *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * -                                                            *
  ****************************************************************
  

Problem conclusion

• The code was modified so that timeout (explicitly defined or
  default) for adapter procedure invocations is enforced on the
  Worklight Server-to-backend connections and sockets as well as
  for Server-Client connections.
  
  The fix for this APAR is currently targeted for fixpack 6.0.0.2
  and fixpack 6.1.0.1.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WORKLIGHT CONSU

• Fixed component ID

  5725I4301

Applicable component levels

• R506 PSY

     UP

• R600 PSY

     UP

• R610 PSY

     UP