JR59242: THE BPEL INDEXER STARTS INDEXING BEFORE THE CHANGE LOG TABLE HAS BEEN REINITIALIZED

pfs.18001.delta.repository

APAR status

• Closed as program error.

Error description

• When the Elastic Search index for a BPEL federated system is
  re-created by multiple IBM Process Federation Server nodes
  (after stopping the nodes, deleting the Elastic Search directory
   of each node, and then restarting the nodes), all the nodes
  coordinate to rebuild the index in the following steps:
  
  1. One of the nodes re-creates the Elastic Search index and adds
  the following message to its logs: "CWMFS1001I: Successfully
  created Elastic Search index name = INDEX_NAME".
  2. All the nodes reset the CONSUMER_XXX column of the
  PFS_BPEL_CHANGE_LOG table in batches of 50000 rows.
  
  When these two operations are completed, the nodes start
  indexing the BPEL data.
  
  In rare situations, one of the node starts indexing the BPEL
  data before the two operations are completed, which causes index
  creation to take more time than usual and might cause exceptions
  similar to the following exception:
  
  CWMFS3007E: An error has occurred with BPEL Indexer:
  federatedSystem: FEDERATED_SYSTEM_NAME, caused by
  com.ibm.db2.jcc.am.SqlTransactionRollbackException: The current
  transaction has been rolled back because of a deadlock or
  timeout.  Reason code "2".. SQLCODE=-911, SQLSTATE=40001,
  DRIVER=4.21.29 (UPDATE SCHEMA.PFS_BPEL_CHANGE_LOG BPELCL SET
  BPELCL.CONSUMER_XXX=? WHERE (BPELCL.CONSUMER_XXX=?) AND
  (BPELCL.PARTITION BETWEEN ? AND ? )).
  
  If too many of these errors occur, the BPEL indexer might stop
  at the faulty node.
  
  PRODUCTS AFFECTED
  Process Federation Server
  

Local fix

• Stop all the federated servers, delete the Elastic Search
  directory on each server (reconstruction of the index), and then
   restart all the federated servers.
  

Problem summary

• No additional information is available.
  

Problem conclusion

• A fix is provided for Process Federation Server V8.5.7.0
  CF2017.06 that ensures other Process Federation Server nodes are
   notified when a BPEL indexer starts creating an Elastic Search
  index to prevent one node from skipping the index creation
  process and starting to index BPEL data, potentially causing
  deadlocks in the federated BPEL database.
  

Temporary fix

• Not applicable.
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  BPM EXPRESS

• Fixed component ID

  5725C9600

Applicable component levels