IT12657: HEALTHCHECK UTILITY CAUSED IBM STERLING B2B INTEGRATOR NODE TO CRASH

APAR status

• Closed as program error.

Error description

• When running the Healtcheck utility in an IBM Sterling B2B
  Integrator environment, it was noticed that a particular
  scanner, in this case, the Waiting on IO test in the Dynamic
  Tracking Data Scanner, produced over 20,000 records. While the
  scanner completed, the system locked up and eventually crashed
  (killed the noapp JVM) while attempting to compile the results
  into a PDF report.
  
  The thread causing the crash:
  
  1XMCURTHDINFO  Current thread
  NULL           ----------------------
  3XMTHREADINFO      "TaskThread-11"
  J9VMThread:0x000001002F8C0C00, j9thread_t:0x00000100221068C0,
  java/lang/Thread:0x0A0000002A45C0D0, state:R, prio=5
  3XMJAVALTHREAD            (java/lang/Thread getId:0x155C5B,
  isDaemon:false)
  3XMTHREADINFO1            (native thread ID:0x143003D, native
  priority:0x5, native policy:UNKNOWN)
  3XMTHREADINFO3           Java callstack:
  4XESTACKTRACE                at
  java/util/ArrayList.newElementArray(ArrayList.java:94(Compiled
  Code))
  4XESTACKTRACE                at
  java/util/ArrayList.growAtEnd(ArrayList.java:375(Compiled Code))
  4XESTACKTRACE                at
  java/util/ArrayList.add(ArrayList.java:158(Compiled Code))
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/planning/PathNode.notIncluding(PathNode.j
  ava:145(Compiled Code))
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/planning/PathNode.expand(PathNode.java:99
  )
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/planning/PathNode.next(PathNode.java:213)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/planning/PathChooser.choose(PathChooser.j
  ava:219)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/pdf/PdfReportGenerator.getSuggestion(PdfR
  eportGenerator.java:1368)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/pdf/PdfReportGenerator.makeSuggestion(Pdf
  ReportGenerator.java:1313)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/Report.makePdfReport(Report.java:406)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/Report.makeReport(Report.java:362)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/PdfScanResults.endPrintMultiple(PdfScanRe
  sults.java:407)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/PdfScanResults.EndPrint(PdfScanResults.ja
  va:314)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/controller/HealthCheckController.BeginTes
  ts(HealthCheckController.java:251)
  4XESTACKTRACE                at
  com/ibm/si/healthcheck/ui/run/TestExecutor.run(TestExecutor.java
  :51)
  4XESTACKTRACE                at
  com/ibm/tenx/ui/page/TaskThread.run(TaskThread.java:42)
  

Local fix

• STRRTC - 481480
  RW / RW
  Circumvention:
  Isolate the scanner that reported too many errors and disable
  the scanner from running.
  

Problem summary

• Problem Description:
  If the HealthCheck tests produce a large number of
  failures (thousands), HealthCheck will hang on the PDF report
  generation step and eventually cause the JVM to crash.
  
  Platforms Affected:
  All
  
  Users Affected:
  All
  

Problem conclusion

• Resolution Summary:
  Added a new property, testErrorLimit, to limit the number of
  failures a test can report. Once the limit is reached,
  execution of the test is immediately terminated.
  
  Delivered In:
  5020601 (HealthCheck 1.0.3.1)
  

Temporary fix

• Disable tests and/or scanners that produce a large number of
  failures.
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  STR B2B INTEGRA

• Fixed component ID

  5725D0600

Applicable component levels

• R526 PSY

     UP