Troubleshooting
Problem
You might encounter a slow-running operation during operation of Engineering Lifecycle Management (ELM). For example, you might be running a repotools operation that takes many days. Or you might be running a specific operation such as a datasource refresh that times out before it completes.
In cases where the database performs most of the work, it might be relevant to optimize the SQL execution plan of the Oracle database.
Note: Do not attempt this optimization to speed up an upgrade. Raise a case to IBM support. This optimization is best attempted for specific operations. Is your database well-resourced? Are you implementing the recommended settings? Are the Oracle table metrics manually updated (sometimes not all tables are updated)? If so and your database is encountering CPU 100% due to sudden spikes in load, the issue might be a defect. Consider raising a case to IBM support. You are asked to provide Oracle MustGather information to diagnose the cause.
Symptom
You generate Java cores against the application server JVM at least 5 minutes apart. The operation being performed in the thread with the deepest stack depth is the same one in both Java cores. The operation is processing an Oracle JDBC statement.
For example, the thread Default Executor-thread-102729 continued the same processing for more than 5 minutes and was the deepest stack depth of all threads at 124.
The operation is oracle/jdbc/driver/OracleStatement.executeQuery
Runnable
Java Stack
at sun/nio/ch/FileDispatcherImpl.read0(Native Method)
at sun/nio/ch/SocketDispatcher.read(SocketDispatcher.java:51(Compiled Code))
at sun/nio/ch/IOUtil.readIntoNativeBuffer(IOUtil.java:235(Compiled Code))
at sun/nio/ch/IOUtil.read(IOUtil.java:209(Compiled Code))
at sun/nio/ch/SocketChannelImpl.read(SocketChannelImpl.java:394(Compiled Code))
at oracle/net/nt/TimeoutSocketChannel.read(TimeoutSocketChannel.java:174(Compiled Code))
at oracle/net/ns/NSProtocolNIO.doSocketRead(NSProtocolNIO.java:555(Compiled Code))
at oracle/net/ns/NIOPacket.readHeader(NIOPacket.java:258(Compiled Code))
at oracle/net/ns/NIOPacket.readPacketFromSocketChannel(NIOPacket.java:190(Compiled Code))
at oracle/net/ns/NIOPacket.readFromSocketChannel(NIOPacket.java:132(Compiled Code))
at oracle/net/ns/NIOPacket.readFromSocketChannel(NIOPacket.java:105(Compiled Code))
at oracle/net/ns/NIONSDataChannel.readDataFromSocketChannel(NIONSDataChannel.java:91(Compiled Code))
at oracle/jdbc/driver/T4CMAREngineNIO.prepareForUnmarshall(T4CMAREngineNIO.java:784(Compiled Code))
at oracle/jdbc/driver/T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:449(Compiled Code))
at oracle/jdbc/driver/T4CTTIfun.receive(T4CTTIfun.java:410(Compiled Code))
at oracle/jdbc/driver/T4CTTIfun.doRPC(T4CTTIfun.java:269(Compiled Code))
at oracle/jdbc/driver/T4C8Oall.doOALL(T4C8Oall.java:655(Compiled Code))
at oracle/jdbc/driver/T4CStatement.doOall8(T4CStatement.java:229(Compiled Code))
at oracle/jdbc/driver/T4CStatement.doOall8(T4CStatement.java:41)
at oracle/jdbc/driver/T4CStatement.executeForDescribe(T4CStatement.java:765)
at oracle/jdbc/driver/OracleStatement.executeMaybeDescribe(OracleStatement.java:983)
at oracle/jdbc/driver/OracleStatement.doExecuteWithTimeout(OracleStatement.java:1168(Compiled Code))
at oracle/jdbc/driver/OracleStatement.executeQuery(OracleStatement.java:1362)
at oracle/jdbc/driver/OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:369)
at com/ibm/team/integration/reporting/db/connection/wrapper/DbStatement.executeQuery(DbStatement.java:132)Note: The Java cores might help to narrow down the cause of poor performance to the Oracle database. They can provide additional information to narrow down the operation causing the long-running SQL. But you need the Oracle AWR report to diagnose a problem. If the duration of the SQL is short and run many times, rather than one long SQL statement, the Java cores might not contain the problematic database operation. It might be necessary to raise a case to IBM support and obtain more Oracle performance MustGather information to narrow down the cause of poor performance to the database.
To generate Java cores on UNIX platform, run the command kill -3 against the application server JVM. See this document for Windows platform. You can also use repodebug. We recommend opening the Java core by using this free tool, which helps to visualize the different threads.
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB77","label":"Automation Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSPRJQ","label":"IBM Engineering Lifecycle Management Base"},"ARM Category":[{"code":"a8m0z000000CbQqAAK","label":"Jazz Team Server-\u003EDatabase-\u003EOracle"}],"ARM Case Number":"TS009801405","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Log InLog in to view more of this document
This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.
Was this topic helpful?
Document Information
More support for:
IBM Engineering Lifecycle Management Base
Component:
Jazz Team Server->Database->Oracle
Software version:
All Versions
Document number:
6604985
Modified date:
25 July 2022
UID
ibm16604985
Manage My Notification Subscriptions