JR53531: MISSING JOB LOG ENTRIES FROM OPS CONSOLE DUE TO FAILED INSERTS (UNIQUE CONSTRAINT ERRORS) TO THE JOBRUNLOG TABLE

InfoSphere Information Server, Version 11.5 for Linux
InfoSphere Information Server, Version 11.5 for Windows
InfoSphere Information Server, Version 11.5.0.2 for Linux
InfoSphere Information Server, Version 11.5.0.2 for AIX
InfoSphere Information Server, Version 11.5.0.2 for Windows
InfoSphere Information Server, Version 11.5.0.1 for AIX

APAR status

• Closed as program error.

Error description

• Missing job log entries from Ops Console due to failed inserts
  (unique constraint errors) to the JobRunLog table
  
  The following error is seen in JobMonApp*.log
  2014-10-26 16:39:11,414 ERROR
  com.ibm.datastage.runtime.common.db.DBManager.insertRecordCmd(DB
  Manager.java:819) - Error executing insert statement: INSERT
  INTO OPCON.JobRunLog
  (RUNID,EventId,LogTimestamp,LogType,MessageId,ContentType,Messag
  eText) VALUES (?,?,?,?,?,?,?). Values: [1894048741526078777, 3,
  Sun Oct 26 16:39:05 GMT-05:00 2014, WAR, IIS-DSTAGE-RUN-E-0441,
  , Job control process (pid 26091) has failed]
  com.ibm.db2.jcc.am.uo: DB2 SQL Error: SQLCODE=-803,
  SQLSTATE=23505, SQLERRMC=1;OPCON.JOBRUNLOG, DRIVER=3.58.135
      at com.ibm.db2.jcc.am.ed.a(ed.java:672)
      at com.ibm.db2.jcc.am.ed.a(ed.java:60)
      at com.ibm.db2.jcc.am.ed.a(ed.java:127)
      at com.ibm.db2.jcc.am.zm.b(zm.java:2193)
      at com.ibm.db2.jcc.am.zm.c(zm.java:2176)
      at com.ibm.db2.jcc.t4.fb.k(fb.java:355)
      at com.ibm.db2.jcc.t4.fb.a(fb.java:61)
      at com.ibm.db2.jcc.t4.t.a(t.java:50)
      at com.ibm.db2.jcc.t4.vb.b(vb.java:218)
      at com.ibm.db2.jcc.am.an.jc(an.java:2901)
      at com.ibm.db2.jcc.am.an.b(an.java:3845)
      at com.ibm.db2.jcc.am.an.bc(an.java:715)
      at com.ibm.db2.jcc.am.an.executeUpdate(an.java:698)
      at
  com.ibm.datastage.runtime.common.db.DBManager.insertRecordCmd(DB
  Manager.java:807)
      at
  com.ibm.datastage.runtime.common.db.DBManager.insertRecord(DBMan
  ager.java:753)
      at
  com.ibm.datastage.runtime.engmonapp.FileManager.insertRecord(Fil
  eManager.java:1716)
      at
  com.ibm.datastage.runtime.engmonapp.FileManager.processJobLog(Fi
  leManager.java:1642)
      at
  com.ibm.datastage.runtime.engmonapp.FileManager.processFile(File
  Manager.java:818)
      at
  com.ibm.datastage.runtime.engmonapp.FileManager.processDirectory
  (FileManager.java:527)
      at
  com.ibm.datastage.runtime.engmonapp.FileManager.watchForDirector
  ies(FileManager.java:398)
      at
  com.ibm.datastage.runtime.engmonapp.EngMonApp.run(EngMonApp.java
  :102)
      at
  com.ibm.datastage.runtime.engmonapp.EngMonApp.main(EngMonApp.jav
  a:133)
   2014-10-26 16:39:11,425 ERROR
  com.ibm.datastage.runtime.engmonapp.FileManager.discardEventFile
  (FileManager.java:2530) - Content of event file
  3905_001_4595_00001_6.dsv:
  ODBEVENT=6
  AT=2014-10-26 21:39:05+0000
  J.ProjectName=MyProject
  J.JobName=MyJob
  J.CompilationTimestamp=2014-07-25 06:54:02
  R.InvocationId=InvocationID
  R.CreationTimestamp=2014-10-26 21:38:54.701+0000
  R.RunMajorStatus=STA
  R.RunMinorStatus=RNW
  L.EventId=3
  L.LogTimestamp=2014-10-26 21:39:05+0000
  L.LogType=WAR
  L.MessageId=IIS-DSTAGE-RUN-E-0441
  L.ContentType=
  L.MessageText=Job control process (pid 26091) has failed
  
  Missing job log entries from Ops Console due to failed inserts
  (unique constraint errors) to the JobRunLog table
  

Local fix

Problem summary

• User affected
    Missing job log entries from Ops Console due to unique
  constraint errors
  
  Problem Description
    Job log entries can be lost sometimes due to failed inserts
  (unique constraint errors)
    to the JobRunLog table. For example, the error below in the
  EngMonApp log:
  
    2015-07-16 07:02:42,527 ERROR
  com.ibm.datastage.runtime.common.db.DBManager.insertRecordCmd
  
  (DBManager.java:812)
    - Error executing insert statement: INSERT INTO
  DSODB.JobRunLog
  
  (RUNID,EventId,LogTimestamp,LogType,MessageId,
    ContentType,MessageText) VALUES (?,?,?,?,?,?,?). Values:
  [9004028666441980075, 1, Tue Feb 03 14:51:28 EST
  
  2015,
    CTL, IIS-DSTAGE-RUN-I-0077, , Finished Job test.]
    com.ibm.db2.jcc.am.uo: DB2 SQL Error: SQLCODE=-803,
  SQLSTATE=23505, SQLERRMC=1;DSODB.JOBRUNLOG,
  
  DRIVER=3.58.135
          at com.ibm.db2.jcc.am.ed.a(ed.java:672)
          ...
  

Problem conclusion

• Fix the missing job log entries by retry insert with different
  event IDs. Theese log entries will appear at the end of job run.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WIS DATASTAGE

• Fixed component ID

  5724Q36DS

Applicable component levels

• R870 PSY

     UP

• R910 PSY

     UP

• R912 PSY

     UP

• RB30 PSY

     UP

• RB31 PSY

     UP

• RB50 PSY

     UP