IT32171: A HANG IS OBSERVED WITH CALLABLE FLOWS DEPLOYED IN SAME INTEGRATION SERVER.

APAR status

• Closed as program error.

Error description

• While using CallableFlowInvoke node calling CallableInput flow
  deployed in same integration server, a hang is observed
  
  The
  hang stack of CallableFlowInvoke node is reported as
  
  #0
  0x0000003ee0e0ba5e in pthread_cond_timedwait@@GLIBC_2.3.2 ()
  
  #1 0x00007f9ab8bfa71a in
  ImbConditionVariable::wait(ImbSimpleMutex::Mutex&, int) ()
  #2
  0x00007f9a4a61fb9d in ImbCallableFlowManager::invokeFlow(ImbCall
  ableFlowManager::CallableFlowName, ImbCommonBuffer*,
  ImbCommonBuffer*, ImbCommonBuffer*, bool, int,
  ImbCallableFlowInvokeNode*) ()
  #3 0x00007f9a4a64673a in ImbCalla
  bleFlowInvokeNode::requestData(ImbRequestTemplateNode::RequestCo
  ntext*, ImbMessageAssembly const&) ()
  
  
  
  The hang stack of
  CallableInput node, which is the root cause of the problem,
  will be as below
  
  #0 0x0000003ee0e0e334 in __lll_lock_wait ()
  
  #1 0x0000003ee0e0c090 in pthread_cond_broadcast@@GLIBC_2.3.2
  ()
  #2 0x00007f9ab8bf9de6 in ImbConditionVariable::signalAll()
  ()
  #3 0x00007f9a4a62fc9a in ImbCallableFlowManager::FlowReplyCa
  llbackLocal::setResponse(ImbCallableFlowManager::FlowReply*) ()
  
  
  
  The issue is seen after several days of running the flow.
  It is seen after a large number of creation and destroy of
  condition variables over that period of time.
  

Local fix

Problem summary

• ****************************************************************
  USERS AFFECTED:
  All Users of IBM App Connect Enterprise V11 and IBM Integration
  Bus V10 using CallableFlowInvoke node.
  
  
  Platforms affected:
  z/OS, MultiPlatform
  
  ****************************************************************
  PROBLEM DESCRIPTION:
  <span style="background-color:rgb(255, 255, 255)">A hang is
  observed after significant executions of CallableFlowInvoke node
  calling a CallableInput flow </span><span
  style="background-color:rgb(255, 255, 255)">deployed in the same
  integration server. </span><span
  style="background-color:rgb(255, 255, 255)">The  stack similar
  to below is observed on the hung thread of CallableFlowInvoke
  node :</span>
  
  <span style="background-color:rgb(255, 255,
  255)">pthread_cond_timedwait@</span>@GLIBC_2.3.2<span
  style="background-color:rgb(255, 255, 255)"> ()   </span>
  <span style="background-color:rgb(255, 255,
  255)">ImbConditionVariable::wait(ImbSimpleMutex::Mutex&, int) ()
      </span>
  <span style="background-color:rgb(255, 255,
  255)">ImbCallableFlowManager::invokeFlow</span><span
  style="background-color:rgb(255, 255, 255)">
                                   </span>
  <span style="background-color:rgb(255, 255,
  255)">ImbCalla</span><span style="background-color:rgb(255, 255,
  255)">bleFlowInvokeNode::requestData</span>
  
  <span style="background-color:rgb(255, 255, 255)">The
  corresponding stack of </span><span
  style="background-color:rgb(255, 255, 255)">CallableInput node
  is observed as below :</span>
  
  <span style="background-color:rgb(255, 255,
  255)">__lll_lock_wait () </span>
  <span style="background-color:rgb(255, 255,
  255)">pthread_cond_broadcast@</span>@GLIBC_2.3.2<span
  style="background-color:rgb(255, 255, 255)">   </span>
  <span style="background-color:rgb(255, 255,
  255)">ImbConditionVariable::signalAll()     </span>
  <span style="background-color:rgb(255, 255,
  255)">ImbCallableFlowManager::FlowReplyCa</span><span
  style="background-color:rgb(255, 255,
  255)">llbackLocal::setResponse</span>
  
  <span style="background-color:rgb(255, 255, 255)">The issue is
  typically seen to occur after several days of running the
  message  flow.  </span>
  

Problem conclusion

• The product now implements an improved logic at handling
  internal objects responsible for Callable flows and thus
  avoiding the hang situation.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
  Version    Maintenance Level
  v10.0      10.0.0.22
  v11.0      11.0.0.11
  
  The latest available maintenance can be obtained from:
  http://www-01.ibm.com/support/docview.wss?rs=849&uid=swg27006041
  
  If the maintenance level is not yet available,information on
  its planned availability can be found on:
  http://www-1.ibm.com/support/docview.wss?rs=849&uid=swg27006308
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  INTEGRATION BUS

• Fixed component ID

  5724J0540

Applicable component levels