IBM Support

FileNet Content Engine /Reporting high CPU utilization on CE JVM with SQL as DataSource provider

Troubleshooting


Problem

Content Engine using SQL Server as datasource provider on Windows 2003. Reports high CPU % utilization between 85-100 % on the CE JVM process.

Symptom

During non-peak usage periods on the CE server /Windows 2003 high CPU utilization is seen on CE JVM

Cause

Current issue with SQL JDBC DataSource version 2.0.1803.100. Issue is addressed in Microsoft HF(HotFix) update :

http://support.microsoft.com/kb/977924

Environment

Content Engine 4.5.x

Windows 2003

SQL Server

Diagnosing The Problem

The following actions were used in determining root cause of issue reported:

  1. Enable verboseGC data. Below are the instructions for doing so in the WebSphere Admin Console:


    Enabling verbose garbage collection (verbosegc) in WebSphere Application Server http://www-304.ibm.com/support/docview.wss?uid=swg21114927

  2. Please follow the instructions for installing TPROF for Windows, a mentioned in the MustGather technote as being our preferred way to collect CPU data.


    MustGather: Performance, hang, or high CPU issues on Windows http://www-304.ibm.com/support/docview.wss?uid=swg21111364

  3. Once the high CPU issue is seen again, recollect the High CPU MustGather data using TPOF and running the windows_hang.py script at the same time.

    Once the data has been collected, review the three javacores, TPROF CPU data, and server logs (SystemOut, SystemErr, native_stdout, and native_stderr.log)

  4. Identification of the current version SQL JBDC driver being used by CE can be view in the CE JVM logs systemout.log

    [6/5/12 12:38:31:964 CDT] 00000022 DSConfigurati I DSRA8203I: Database
    product name : Microsoft SQL Server
    [6/5/12 12:38:31:995 CDT] 00000022 DSConfigurati I DSRA8204I: Database
    product version : 9.00.3042
    [6/5/12 12:38:31:995 CDT] 00000022 DSConfigurati I DSRA8205I: JDBC
    driver name : Microsoft SQL Server JDBC Driver 2.0
    [6/5/12 12:38:31:995 CDT] 00000022 DSConfigurati I DSRA8206I: JDBC
    driver version : 2.0.1803.100

Resolving The Problem

Review of collected Javacore files showed evidence as follows:

Found 3 threads consuming over 18% each of the cpu. They are perform code from FileNet to persist data to the SQL database and the threads are reading/performing steps to the microsoft/sqlserver:


  • LAB TKS %%% NAMES

    PID 61876 88.09 java.exe_0d9c

    )) Process_Thread

  • LAB TKS %%% NAMES

    PID 61876 88.09 java.exe_0d9c
    TID 13270 18.89 tid_170c
    TID 12985 18.49 tid_1778
    TID 12979 18.48 tid_1738
    TID 3722 5.30 tid_1034

  • Each java/callstack referered to SQL DB calls being issue by FileNet:

    Java callstack:
    at com/microsoft/sqlserver/jdbc/SQLServerConnection.terminate(
    at com/microsoft/sqlserver/jdbc/SQLServerConnection.terminate(
    at com/microsoft/sqlserver/jdbc/TDSChannel.read(IOBu
    at com/microsoft/sqlserver/jdbc/TDSReader.readPacket(IOBuffe
    at com/microsoft/sqlserver/jdbc/TDSReader.nextPacket(IOB
    <...com/mircorsoft/sqlserver/jdbc code>

  • The above pointed to SQL JDBC as culprit and hence applying the following MS HF fpr SQL JBDC driver update to version 2.0.1803.206 on CE server in question.
    ===
    http://support.microsoft.com/kb/977924

[{"Product":{"code":"SSNW2F","label":"FileNet P8 Platform"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Content Engine","Platform":[{"code":"PF033","label":"Windows"},{"code":"PF027","label":"Solaris"},{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"}],"Version":"4.5.1;5.0;5.1","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
17 June 2018

UID

swg21599191

Page Feedback