Troubleshooting
Problem
Deadlock error occurs in FileNet Content Engine while creating documents under a heavy load.
Symptom
When the load gets increased, the P8_server_error.log is giving the deadlock error resulting hung threads. The error shown in the P8_server_error.log is similar as below:
2011-12-26T06:45:26.000Z 18611861 CSTG FNRCE0259E - ERROR failed to execute content queue update statement com.filenet.api.exception.EngineRuntimeException: E_DEADLOCK_ERROR: The operation could not be completed due to a deadlock error. A retry may be appropriate. SQL: "UPDATE ContentQueue SET session_id = ?, process_attempts = process_attempts + 1, lease_expiry_date = GetDate() + DateAdd(s,420,0) FROM ContentQueue fsq1 WHERE session_id = ? AND GetDate() > create_date AND NOT EXISTS (SELECT 1 FROM ContentQueue fsq2 WHERE fsq2.object_id = fsq1.object_id AND fsq2.session_id != ?)" In-bindings: ({45797457-8A5A-48D7-9C8A-482AD1F90398},{6A3189F2-EAFF-4E53-B1F5-D1BEB30ACE59},{6A3189F2-EAFF-4E53-B1F5-D1BEB30ACE59})
at com.filenet.engine.dbpersist.DBMSSQLContext.throwEngineException(DBMSSQLContext.java:302)
at com.filenet.engine.dbpersist.DBExecutionElement.execute(DBExecutionElement.java:266)
at com.filenet.engine.dbpersist.DBExecutionContext.getNextResult(DBExecutionContext.java:106)
at com.filenet.engine.dbpersist.DBStatementBase.execute(DBStatementBase.java:249)
at com.filenet.engine.queueitem.ContentQueueDispatcher.appendToBatch(ContentQueueDispatcher.java:1050)
at com.filenet.engine.queueitem.ContentQueueDispatcher._execute(ContentQueueDispatcher.java:453)
at com.filenet.engine.queueitem.ContentQueueDispatcher.execute(ContentQueueDispatcher.java:145)
at com.filenet.engine.tasks.BackgroundTask.safeExecute(BackgroundTask.java:210)
at com.filenet.engine.tasks.BackgroundTask.access$000(BackgroundTask.java:42)
at com.filenet.engine.tasks.BackgroundTask$BackgroundTaskPriviledgedExceptionAction.run(BackgroundTask.java:821)
at com.filenet.engine.context.CallState.doAsSystem(CallState.java:360)
at com.filenet.engine.tasks.BackgroundTask.run(BackgroundTask.java:166)
at com.filenet.engine.jca.workmgr.ThreadPool$WorkerThread.run(ThreadPool.java:85)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 55) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:196)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1454)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:388)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:338)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4026)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1416)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:185)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:160)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.execute(SQLServerPreparedStatement.java:320)
at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecute(WSJdbcPreparedStatement.java:955)
at com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.execute(WSJdbcPreparedStatement.java:631)
at com.filenet.engine.dbpersist.DBExecutionElement.execute(DBExecutionElement.java:218)
... 11 more
Cause
"webSphereDefaultIsolationLevel" custom property is not created or it is set to a wrong value. Normally the Configuration Manager tool should have been used to create the data sources for the object stores. If that had been done, the isolation level would have been correctly set to 2. So perhaps the data sources were created manually.
Diagnosing The Problem
1. Check if the webSphereDefaultIsolationLevel custom property has been created for non-XA data source and has the value 2 in the Websphere administrative console.
2. Alternatively, we can check this value in the P8 trace file. For example:
2012-01-03T00:18:59.937Z 4E3A4E3A CSTG FNRCE0000D - DEBUG ContentQueueDispatcher running for ObjectStore=[OSTEST]
...
2012-01-03T00:18:59.937Z 4E3A4E3A CSTG FNRCE0000D - DEBUG DB Connection isolation level = 2
Resolving The Problem
Added the property in Non-XA data source custom property:
Name="webSphereDefaultIsolationLevel"
Type="java.lang.String"
value="2"
Was this topic helpful?
Document Information
Modified date:
17 June 2018
UID
swg21584702