IBM Support

JR48886: POOR PERFORMANCE AND HIGH DATABASE CPU USAGE WHEN A LARGE NUMBER OF TASKS ARE PRESENT

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • The TWSearch() API takes 10s of minutes to complete when there
    are a large number of tasks in the system. Overall, system
    performance could be reduced and hung thread errors are
    recorded in the server log files.
    
    Your database administrator might see the following query,
    which uses most of the CPU time:
    
    SELECT DISTINCT v.ALIAS AS ALIAS, v.VARIABLE_TYPE AS
    VARIABLE_TYPE,
    i.SNAPSHOT_ID AS SNAPSHOT_ID
    FROM LSW_BPD_INSTANCE_VARIABLES v
    JOIN LSW_BPD_INSTANCE i ON v.BPD_INSTANCE_ID = i.BPD_INSTANCE_ID
    ORDER BY ALIAS, VARIABLE_TYPE
    
    You see the following stack trace in the server log file:
    
    [12/11/13 1:40:57:674 NZDT] 0000006c BPMInspectorR E   Error
    communicating with server
    
    .
    com.ibm.processinspector.rest.ProcessAdminRestException: Error
    communicating with server
    at
    com.ibm.processinspector.rest.ProcessAdminRestClient.doPostQuery
    (ProcessAdminRestClient.java:201)
    at
    com.ibm.processinspector.rest.ProcessAdminRestClient.doQuery(Pro
    cessAdminRestClient.java:303)
    ......
    Caused by: java.net.ConnectException: Connection timed out
      at
    java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:412)
      at
    java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:2
    71)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:258)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:376)
       at java.net.Socket.connect(Socket.java:537)
    ......
    
    .-----------------------------------------------------
       Here is an example of the WSVR0605W message that might be
    logged when the problem occurs.
    
    
    [10/9/14 8:41:18:169 PDT] 000000a4 ThreadMonitor W   WSVR0605W:
    Thread "WebContainer : 9" (0000018b) has been active for 631295
    milliseconds and may be hung.  There is/are 1 thread(s) in
    total in the server that may be hung.
    ...
    at
    com.lombardisoftware.server.ejb.psapi.SearchAPIFacadeBean$24.exe
    cute(SearchAPIFacadeBean.java:358)
        at
    com.lombardisoftware.server.ejb.psapi.SearchAPIFacadeBean$24.exe
    cute(SearchAPIFacadeBean.java:356)
        at
    com.lombardisoftware.server.ejb.EjbServerUtils.handleEjbCall(Ejb
    ServerUtils.java:86)
        at
    com.lombardisoftware.server.ejb.psapi.SearchAPIFacadeBean.handle
    EjbCall(SearchAPIFacadeBean.java:50)
        at
    com.lombardisoftware.server.ejb.psapi.SearchAPIFacadeBean.findSe
    archable
    BusinessDataAliases(SearchAPIFacadeBean.java:356)
        at
    com.lombardisoftware.server.ejb.psapi.EJSRemoteStatelessSearchAP
    IFacade_
    9724b32b.findSearchableBusinessDataAliases(Unknown Source)
        at
    com.lombardisoftware.server.ejb.psapi._SearchAPIFacadeInterface_
    Stub.fin
    dSearchableBusinessDataAliases(_SearchAPIFacadeInterface_Stub.ja
    va:1446)
        at
    com.lombardisoftware.client.delegate.SearchAPIFacadeDelegateDefa
    ult$24.run(SearchAPIFacadeDelegateDefault.java:708)
        at
    java.security.AccessController.doPrivileged(AccessController.jav
    a:327)
        at
    com.lombardisoftware.client.delegate.common.WebsphereDelegateHel
    per.doAs
    CurrentSubjectContextSensitive(WebsphereDelegateHelper.java:197)
        at
    com.lombardisoftware.client.delegate.SearchAPIFacadeDelegateDefa
    ult.find
    SearchableBusinessDataAliases(SearchAPIFacadeDelegateDefault.jav
    a:705)
        at
    com.lombardisoftware.core.script.js.helpers.TypeConverter.getBus
    inessDataAliases(TypeConverter.java:1152)
        at
    com.lombardisoftware.core.script.js.helpers.TypeConverter.fromIn
    ternalQueryResult(TypeConverter.java:592)
        at
    com.lombardisoftware.core.script.js.TWSearchScriptable.executeSe
    archInternal(TWSearchScriptable.java:362)
        at
    com.lombardisoftware.core.script.js.TWSearchScriptable.jsFunctio
    n_execut e(TWSearchScriptable.java:172)
    

Local fix

  • No IFix.
    
    temporary workaround->
    disable the new behavior:
    ----------------------------------------------------------------
    <use-business-aliases-for-process-instances>false</use-business-
    aliases-for-process-instances>.
    ----------------------------------------------------------------
    -
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  IBM BPM Advanced, Standard, and Express     *
    ****************************************************************
    * PROBLEM DESCRIPTION: Searchable business data aliases are    *
    *                      the columns that you can use in saved   *
    *                      searches, the TWSearch API, and the     *
    *                      REST Search API. Before the             *
    *                      <use-business-aliases-for-process-      *
    *                      instances> property was introduced,     *
    *                      business data aliases that were         *
    *                      defined in any snapshot were            *
    *                      available and reloaded after each       *
    *                      snapshot deployment, which slowed       *
    *                      down performance, especially as the     *
    *                      number of snapshots increased, such     *
    *                      as in a Process Center environment      *
    *                      with many developers.                   *
    *                      When the                                *
    *                      <use-business-aliases-for-process-      *
    *                      instances> property is set to 'true',   *
    *                      the list of searchable business data    *
    *                      aliases are returned only when they     *
    *                      are used in actual process instances.   *
    *                      Because business data aliases are       *
    *                      loaded as needed, performance might     *
    *                      slow down as the number of instances    *
    *                      increases (such as in a high-volume     *
    *                      Process Server environment), but the    *
    *                      number will not climb unnecessarily.    *
    *                      Business data aliases might not be      *
    *                      available immediately after instances   *
    *                      are created.                            *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    You experience poor performance and high database CPU usage
    when a large number of tasks are present.
    

Problem conclusion

  • A fix is available for IBM BPM V8.0.1.2 and V8.5.0.1 that
    ensures the Business Data Alias cache is built by default
    when the server starts, improving loading performance.
    The <use-business-aliases-for-process-instances> property is set
    to "false" so that the list of searchable business aliases
    are returned based on snapshot definitions.
    
    On Fix Central (http://www.ibm.com/support/fixcentral), and
    search for JR49354 for V8.0.1.2, or JR48886 for V8.5.0.1:
    
    1. Select the product group, product, installed version, and
    platform, and click Continue.
    2. Select APAR or SPR, enter JR49354 or JR48886, and click
    Continue.
    
    When downloading fix packages, ensure that you also download
    the readme file for each fix. Review each readme file for
    additional installation instructions and information about the
    fix.
    

Temporary fix

  • Yes. In the 100Custom.xml file, change the
     <use-business-aliases-for-process-instances> property value to
    "false":
    
    In stand-alone profiles, go to
    Profile_home\config\cells\<~cellname~>\nodes\<~nodename~>\server
    s\<~servername~>\<~process-center/server~>\config\100Custom.xml
    
    In cluster profiles, go to
    DMGR_home\config\cells\<~cellname~>\nodes\<~nodename~>\servers\<
    ~servername~>\<~process-center/server~>\config\100Custom.xml
    
    In the file, insert the following XML code under the
    <properties> tag:
    
    <server>
     <portal merge="mergeChildren">
      <use-business-aliases-for-process-instances
    merge="replace">false</use-business-aliases-for-process-instance
    s>
     </portal>
    </server>
    

Comments

APAR Information

  • APAR number

    JR48886

  • Reported component name

    BPM STANDARD

  • Reported component ID

    5725C9500

  • Reported release

    850

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2013-12-19

  • Closed date

    2014-02-13

  • Last modified date

    2016-01-04

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    BPM STANDARD

  • Fixed component ID

    5725C9500

Applicable component levels

  • R800 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSFTDH","label":"IBM Business Process Manager Standard"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.5","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
04 January 2016