Topic
  • No replies
SystemAdmin
SystemAdmin
2736 Posts

Pinned topic JVM crashes randomly due to the "inside compiled method" error

‏2012-04-06T09:55:50Z |
Hi,

We have a java server ran on AIX 6.1. Somehow, the server always crashes randomly. The core dump file shows the crash is a JIT crash and occurred when executing the compiled method(java/lang/Class.throwNoSuchMethodException.(Ljava/lang/String;[Ljava/lang/Class;)V).

We are not sure why the issue happens. We upgraded the JRE to the 6.0 SR9 FP1 version but the issue still is there.

The IBM website provides a workaround to resolve this problem. That is to add the following JVM option. But
unfortunately, the workaround doesn't work too. So for now, we have no any ideas to resolve it.
-Xjit:verbose={compileStart|compileEnd},vlog=CL_JIT.log,exclude={java/lang/Class.throwNoSuc#hMethodException.(Ljava/lang/String;[Ljava/lang/Class;)V}

Do you have any suggestions? Thank you very much.

FYI. The following is the key sections of the core dump file.

0SECTION TITLE subcomponent dump routine
NULL ===============================
1TISIGINFO Dump Event "gpf" (00002000) received
1TIDATETIME Date: 2012/03/12 at 08:27:36
1TIFILENAME Javacore filename: /healthvisionp/cis5.8/integrator/javacore.20120312.082425.57540822.0002.txt
1TIREQFLAGS Request Flags: 0x81 (exclusive+preempt)
1TIPREPSTATE Prep State: 0x0
1TIPREPINFO Exclusive VM access not taken: data may not be consistent across javacore sections
NULL
0SECTION GPINFO subcomponent dump routine
NULL ================================
2XHOSLEVEL OS Level : AIX 6.1
2XHCPUS Processors -
3XHCPUARCH Architecture : ppc
3XHNUMCPUS How Many : 16
3XHNUMASUP NUMA is either not supported or has been disabled by user
NULL
1XHEXCPCODE J9Generic_Signal_Number: 00000004
1XHEXCPCODE Signal_Number: 0000000B
1XHEXCPCODE Error_Value: 00000000
1XHEXCPCODE Signal_Code: 00000033
1XHEXCPCODE Handler1: F0B66208
1XHEXCPCODE Handler2: F0A6BF1C
NULL
NULL
1XHREGISTERS Registers:
2XHREGISTER R0: 002CB872
2XHREGISTER R1: 35DFF480
2XHREGISTER R2: F0B67C6C
2XHREGISTER R3: B0DE8C30
2XHREGISTER R4: 333C0000
2XHREGISTER R5: B0DE8C30
2XHREGISTER R6: 00000000
2XHREGISTER R7: 00000000
2XHREGISTER R8: 00000000
2XHREGISTER R9: F13ACDA8
2XHREGISTER R10: 331AB2C4
2XHREGISTER R11: 00000000
2XHREGISTER R12: D3A9C6E4
2XHREGISTER R13: 331AB100
2XHREGISTER R14: 33780A70
2XHREGISTER R15: 00000000
2XHREGISTER R16: D2477BAC
2XHREGISTER R17: B0055D30
2XHREGISTER R18: B0DE87E8
2XHREGISTER R19: B0D50030
2XHREGISTER R20: B0D50118
2XHREGISTER R21: B0D50108
2XHREGISTER R22: B00E8B78
2XHREGISTER R23: B0003510
2XHREGISTER R24: B0DE8AD8
2XHREGISTER R25: B0DE8AF0
2XHREGISTER R26: 00000001
2XHREGISTER R27: B0DE8AF0
2XHREGISTER R28: B00E8B78
2XHREGISTER R29: B0DE8AF0
2XHREGISTER R30: B0DE8AF0
2XHREGISTER R31: B0DE8C30
2XHREGISTER FPR0: 3FE8000000000000
2XHREGISTER FPR1: 0000000000000000
2XHREGISTER FPR2: 0000000000000000
2XHREGISTER FPR3: 0000000000000000
2XHREGISTER FPR4: 0000000000000000
2XHREGISTER FPR5: 0000000C0000000C
2XHREGISTER FPR6: 4028000000000000
2XHREGISTER FPR7: 4030000000000000
2XHREGISTER FPR8: 006A006100760061
2XHREGISTER FPR9: 002E006C0061006E
2XHREGISTER FPR10: 0067002E0043006C
2XHREGISTER FPR11: 006100730073004C
2XHREGISTER FPR12: 0000000000000000
2XHREGISTER FPR13: 0000000000000000
2XHREGISTER FPR14: 0000000000000000
2XHREGISTER FPR15: 0000000000000000
2XHREGISTER FPR16: 0000000000000000
2XHREGISTER FPR17: 0000000000000000
2XHREGISTER FPR18: 0000000000000000
2XHREGISTER FPR19: 0000000000000000
2XHREGISTER FPR20: 0000000000000000
2XHREGISTER FPR21: 0000000000000000
2XHREGISTER FPR22: 0000000000000000
2XHREGISTER FPR23: 0000000000000000
2XHREGISTER FPR24: 0000000000000000
2XHREGISTER FPR25: 0000000000000000
2XHREGISTER FPR26: 0000000000000000
2XHREGISTER FPR27: 0000000000000000
2XHREGISTER FPR28: 0000000000000000
2XHREGISTER FPR29: 0000000000000000
2XHREGISTER FPR30: 0000000000000000
2XHREGISTER FPR31: 0000000000000000
2XHREGISTER IAR: 30C36FE8
2XHREGISTER LR: 30C36FD4
2XHREGISTER MSR: 0000D032
2XHREGISTER CTR: D245D8BC
2XHREGISTER CR: 2800044C
2XHREGISTER FPSCR: 82000000
2XHREGISTER XER: 0000000C
2XHREGISTER TID: 00000000
2XHREGISTER MQ: 00000000
1XHEXCPMODULE Inside compiled method: java/lang/Class.throwNoSuchMethodException(Ljava/lang/String;[Ljava/lang/Class;)V
NULL
1XHFLAGS VM flags:00000000
NULL
NULL
0SECTION ENVINFO subcomponent dump routine
NULL =================================
1CIJAVAVERSION JRE 1.6.0 IBM J9 2.4 AIX ppc-32 build jvmap3260sr9-20110203_74623
1CIVMVERSION VM build 20110203_074623
1CIJITVERSION JIT enabled, AOT enabled - r9_20101028_17488ifx3
1CIGCVERSION GC - 20101027_AA
1CIRUNNINGAS Running as a standalone JVM
.....

0SECTION THREADS subcomponent dump routine
NULL =================================
NULL
1XMCURTHDINFO Current thread
NULL
3XMTHREADINFO "Thread-2466375" J9VMThread:0x331AB100, j9thread_t:0x33A61A4C, java/lang/Thread:0xB0D50030, state:R, prio=5
3XMTHREADINFO1 (native thread ID:0x34013F, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at java/lang/Class.throwNoSuchMethodException(Class.java:286(Compiled Code))
4XESTACKTRACE at java/lang/Class.getDeclaredMethod(Class.java:616(Compiled Code))
4XESTACKTRACE at java/lang/Thread$1.run(Thread.java:306(Compiled Code))
4XESTACKTRACE at java/security/AccessController.doPrivileged(AccessController.java(Compiled Code))
4XESTACKTRACE at java/lang/Thread.initialize(Thread.java:290(Compiled Code))
4XESTACKTRACE at java/lang/Thread.<init>(Thread.java:267(Compiled Code))
4XESTACKTRACE at java/lang/Thread.<init>(Thread.java:179(Compiled Code))
4XESTACKTRACE at java/lang/UNIXProcess$2$1.<init>(UNIXProcess.java:84(Compiled Code))
4XESTACKTRACE at java/lang/UNIXProcess$2.run(UNIXProcess.java:84(Compiled Code))
4XESTACKTRACE at java/security/AccessController.doPrivileged(AccessController.java:202(Compiled Code))
4XESTACKTRACE at java/lang/UNIXProcess.<init>(UNIXProcess.java:49(Compiled Code))
4XESTACKTRACE at java/lang/ProcessImpl.start(ProcessImpl.java:112(Compiled Code))
4XESTACKTRACE at java/lang/ProcessBuilder.start(ProcessBuilder.java:464(Compiled Code))
4XESTACKTRACE at java/lang/Runtime.exec(Runtime.java:604(Compiled Code))
4XESTACKTRACE at com/hie/util/process/BasicProcessMonitor.execProcess(Bytecode PC:396(Compiled Code))
4XESTACKTRACE at com/hie/util/process/BasicProcessMonitor.run(Bytecode PC:67(Compiled Code))
4XESTACKTRACE at java/lang/Thread.run(Thread.java:736(Compiled Code))
3XMTHREADINFO3 Native callstack:
4XENATIVESTACK (0xD2415910 http://libj9vm24.so+0xb910)
4XENATIVESTACK (0xD248BBAC http://libj9prt24.so+0x1bac)
4XENATIVESTACK (0xD2415830 http://libj9vm24.so+0xb830)
4XENATIVESTACK (0xD1EC7CC0 http://libj9thr24.so+0x1cc0)
4XENATIVESTACK _pthread_body+0xec (0xD04DEC50 http://libpthreads.a+0x3c50)
NULL
NULL

.....