RS03387: RES XU DEADLOCK LEADING TO THE ERROR MESSAGE "THE DEFAULT CONNECTION MANAGER POOL IS FULL"

APAR status

• Closed as program error.

Error description

• When using htds execution with trace enabled you might face a
  blockage where the connection pull gets full.
  There can be a multithread race condition leading to a dead lock
  when using B2X testers and sequential algorithm (which may lead
  to "The default connection manager pool is full")
  
   at
  ilog/rules/engine/IlrInterpreter.invokeFunction(IlrInterpreter.j
  ava:414)
  at
  ilog/rules/engine/IlrInterpreter.invokeFunction(IlrInterpreter.j
  ava:377)
  at
  ilog/rules/engine/dataio/IlrTranslatedDataAccessStrategy.invokeF
  unction(IlrTranslatedDataAccessStrategy.java:548(Compiled Code))
  at
  ilog/rules/engine/dataio/IlrTranslatedDataAccessStrategy.get(Ilr
  TranslatedDataAccessStrategy.java:422(Compiled Code))
  (entered lock: ilog/rules/factory/IlrReflect@0x00000007C0357450,
  entry count: 1)
  (entered lock: ilog/rules/factory/IlrReflect@0x00000007C039FB98,
  entry count: 2)
  at
  ilog/rules/engine/dataio/IlrBusinessDataXmlService$IlrBusinessDa
  taConverter.writeAttributeInternal(IlrBusinessDataXmlService.jav
  a:909(Compiled Code))
  at
  ilog/rules/engine/dataio/IlrBusinessDataXmlService$IlrBusinessDa
  taConverter.writeAttribute(IlrBusinessDataXmlService.java:895(Co
  mpiled Code))
  at
  ilog/rules/engine/dataio/IlrBusinessDataXmlService$IlrBusinessDa
  taConverter.writeObject(IlrBusinessDataXmlService.java:856(Compi
  led Code))
  (entered lock: ilog/rules/factory/IlrReflect@0x00000007C039FB98,
  entry count: 1)
  at
  ilog/rules/engine/dataio/IlrBusinessDataXmlService$IlrBusinessCo
  nverterManager$MContext.writeObjectInternal(IlrBusinessDataXmlSe
  rvice.java:297(Compiled Code))
  at
  ilog/rules/base/xml/IlrXmlDefaultConverterManager$MarshallingCon
  text.writeObject(IlrXmlDefaultConverterManager.java:334(Compiled
  Code))
  at
  ilog/rules/base/xml/IlrXmlDefaultConverterManager.writeObject(Il
  rXmlDefaultConverterManager.java:83)
  at
  ilog/rules/base/xml/IlrXmlAbstractService.fillDocument(IlrXmlAbs
  tractService.java:221)
  at
  ilog/rules/engine/dataio/IlrBusinessDataXmlService.fillDocument(
  IlrBusinessDataXmlService.java:137)
  at
  com/ibm/rules/res/xu/engine/cre/internal/CREBOMConverter.toDocum
  ent(CREBOMConverter.java:195)
  (entered lock:
  ilog/rules/engine/dataio/IlrBusinessDataXmlService@0x00000007C03
  9FA78, entry count: 1)
  at
  com/ibm/rules/res/xu/engine/cre/internal/CREBOMConverter.toXML(C
  REBOMConverter.java:232)
  at
  com/ibm/rules/res/xu/engine/cre/internal/CREBOMConverter.toBOM(C
  REBOMConverter.java:214)
  at
  com/ibm/rules/res/xu/bom/internal/BOMConverterBase.toBOMParamete
  rs(BOMConverterBase.java:64)
  at
  com/ibm/rules/res/xu/engine/internal/BaseEngineManager.getParame
  tersAsBOM(BaseEngineManager.java:603)
  (entered lock:
  com/ibm/rules/res/xu/engine/cre/internal/CREManager@0x00000007C0
  39EAB8, entry count: 1)
  at
  ilog/rules/res/xu/spi/IlrManagedXUConnection.engineGetParameters
  AsBOM(IlrManagedXUConnection.java:864)
  (entered lock:
  ilog/rules/res/xu/spi/IlrManagedXUConnection@0x00000007C03A00C8,
  entry count: 1)
  at
  ilog/rules/res/xu/cci/IlrXUConnection.engineGetParametersAsBOM(I
  lrXUConnection.java:411)
  at
  com/ibm/rules/res/xu/client/internal/jca/XUInteraction.engineGet
  ParametersAsBOM(XUInteraction.java:521)
  at
  com/ibm/rules/res/xu/client/internal/jca/XUInteraction.dispatchE
  xecution(XUInteraction.java:80)
  at
  com/ibm/rules/res/xu/client/internal/jca/XUInteraction.execute(X
  UInteraction.java:254)
  at
  com/ibm/rules/res/xu/client/internal/XUSession.executeOperation(
  XUSession.java:174)
  at
  com/ibm/rules/res/xu/client/internal/XURuleEngineSession.getPara
  metersAsBOM(XURuleEngineSession.java:301)
  at
  com/ibm/rules/res/xu/client/internal/XURuleEngineSession.getPara
  metersAsXMLBOM(XURuleEngineSession.java:253)
  at
  ilog/rules/res/session/impl/trace/IlrTraceBuilder.serializeParam
  etersUsingBOMSerialization(IlrTraceBuilder.java:543)
  at
  ilog/rules/res/session/impl/trace/IlrTraceBuilder.serializeParam
  eters(IlrTraceBuilder.java:225)
  at
  ilog/rules/res/session/impl/trace/IlrTraceBuilder.serializeInput
  ParametersAndKeepResult(IlrTraceBuilder.java:214)
  at
  ilog/rules/res/session/impl/IlrStatefulSessionBase.execute(IlrSt
  atefulSessionBase.java:566)
  at
  ilog/rules/res/session/impl/IlrStatefulSessionBase.execute(IlrSt
  atefulSessionBase.java:513)
  at
  ilog/rules/res/session/impl/IlrStatefulSessionBase.execute(IlrSt
  atefulSessionBase.java:236)
  at
  com/ibm/rules/htds/DecisionServiceController.executeRuleset(Deci
  sionServiceController.java:221)
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * Users running decision services using HTDS                   *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * When using htds execution with trace enabled you might face  *
  * a blockage where the connection pull gets full.              *
  * There can be a multithread race condition leading to a dead  *
  * lock when using BOM to XOM mapping (B2X) testers and         *
  * sequential algorithm (which may lead to "The default         *
  * connection manager pool is full")                            *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  

Problem conclusion

• The code is fixed.
  

Temporary fix

• As a workaround, use a tester in an extender class, instead of
  an extender implemented in IRL.
  Here is an example of a tester in an extender:
  public class CustomerExtender {
  public static String getName(Map customer) { return (String)
  customer.get("name"); }
  public static int getMoney(Map customer) { return ((Integer)
  customer.get("money")).intValue(); }
  public static void setMoney(Map customer, int value) {
  customer.put("money", value); }
  public static Map create(String name) {
    Map result = new HashMap(3);
  result.put("name", name);
  result.put("money", 0); // as we assumed it always contain an
  Integer
  result.put("className", "Customer"); return result;
  }
  // THE TESTER
  public static boolean tester(Map customer) { return
  "Customer".equals(customer.get("className")); }
  }
  
  The extender has to be declared in the B2X part of the BOM
  class.
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WS DECISION SVR

• Fixed component ID

  5725B6901

Applicable component levels