IC5Notice: We have upgraded developerWorks Community to the latest version of IBM Connections. For more information, read our upgrade FAQ.
Topic
  • 11 replies
  • Latest Post - ‏2012-11-20T06:55:20Z by SystemAdmin
SystemAdmin
SystemAdmin
2736 Posts

Pinned topic UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

‏2012-10-30T14:54:26Z |
We have a program that uses JNI for the communication with a legacy system. It's working fine on several platforms, but unfortunately not on AIX with a Java version > 1.6 (SR8 FP1).

I've written a small test program that only tries to load the library (System.loadLibrary("jnilib")) to ensure that the problem isn't caused by any other part of our program. This test program successfully loads the library with Java 1.6 SR8 FP1 (fullversion: "JRE 1.6.0 IBM AIX build pap3260sr8fp1-20100624_01 (SR8 FP1)"), but it fails with Java 1.6 SR9 FP2 (fullversion: "JRE 1.6.0 IBM AIX build pap3260sr9fp2-20110627_03 (SR9 FP2)") or any newer version with an UnsatisfiedLinkError:

java.lang.UnsatisfiedLinkError: jnilib (No such file or directory)
at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1018)
at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:982)
at java.lang.System.loadLibrary(System.java:506)
at com.betasystems.jdmc.jni.JDMConnector.<clinit>(JDMConnector.java:626)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:200)
at com.betasystems.jdmc.jni.JDMC.<init>(JDMC.java:80)
at com.betasystems.jdmc.rmi.JDMCServer.main(JDMCServer.java:813)

I've used the same command to start the test program for all tested Java versions, e.g. the Java version is the only difference.

I've already tried to change the library path from a relative to an absolute path, without success.
I've also recompiled the c-library as suggested here: http://www.ibm.com/developerworks/forums/thread.jspa?messageID=14878763&tstart=0, without success.

After searching the internet for a while I've learned that the message 'No such file or directory' doesn't necessarily mean that Java couldn't find the library itself. I might be caused by some other libraries that are used by our c-library that couldn't be loaded. Is there any way to the 'real reason' for this error?
Updated on 2012-11-20T06:55:20Z at 2012-11-20T06:55:20Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-10-30T18:01:40Z  
    Can you try by pointing the environment variable LIBPATH to libjnilib.so and running the testcase?
    export LIBPATH=/path/to/libjnilib/:$PATH

    I remember coming across some code changes in this area around the Java6 SR9FP1 time frame - unfortunately i cant seem to get hold of the APAR's for you. I think it was something to do with MANDATING the use of LIBPATH while working with native libraries inside Java code.
    Hope this helps.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-10-31T07:35:08Z  
    Can you try by pointing the environment variable LIBPATH to libjnilib.so and running the testcase?
    export LIBPATH=/path/to/libjnilib/:$PATH

    I remember coming across some code changes in this area around the Java6 SR9FP1 time frame - unfortunately i cant seem to get hold of the APAR's for you. I think it was something to do with MANDATING the use of LIBPATH while working with native libraries inside Java code.
    Hope this helps.
    Setting the LIBPATH doesn't helps, I'm still getting the same error. But Thanks for your help anyway.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-05T09:58:20Z  
    Setting the LIBPATH doesn't helps, I'm still getting the same error. But Thanks for your help anyway.
    The problem could be related to JNI shared library built with runtime linking (-brtl). You can try with a shared library without runtime linking enabled.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-06T13:10:32Z  
    The problem could be related to JNI shared library built with runtime linking (-brtl). You can try with a shared library without runtime linking enabled.
    Sorry for the late reply, I was busy with some other tasks.

    Runtime linking isn't enabled. The only option that is used for linking is '-G'.

    I'm not really a *nix expert. Isn't there any way to find out whats going on when Java tries to load the library?
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-07T08:38:12Z  
    Sorry for the late reply, I was busy with some other tasks.

    Runtime linking isn't enabled. The only option that is used for linking is '-G'.

    I'm not really a *nix expert. Isn't there any way to find out whats going on when Java tries to load the library?
    To find out the shared library dependencies, use command "ldd"
    for example: ldd libc.so

    We can use the command "truss" to trace the system calls invoked by the JVM. In the truss output, we will be interested in "load" system call returns code.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-07T08:40:39Z  
    To find out the shared library dependencies, use command "ldd"
    for example: ldd libc.so

    We can use the command "truss" to trace the system calls invoked by the JVM. In the truss output, we will be interested in "load" system call returns code.
    Also, you can try by building the library as suggested in the link,

    http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/topic/com.ibm.java.doc.user.aix64.60/user/jni.html?resultof=%22%72%75%6e%74%69%6d%65%22%20%22%72%75%6e%74%69%6d%22%20%22%6c%69%6e%6b%69%6e%67%22%20%22%6c%69%6e%6b%22%20

    >>>>>>text from the above link<<<<<<
    You are recommended to compile your native methods (C or C++ functions called by Java) into AIX shared objects (dynamically loaded libraries). For example, if your native methods are stored in the file nm.c, you could create the shared object with the following command:
    cc_r -qmkshrobj -q64 -I /usr/java6_64/include -o libnm.a nm.c
    >>>>>>>end<<<<<
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-07T14:31:43Z  
    Also, you can try by building the library as suggested in the link,

    http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/topic/com.ibm.java.doc.user.aix64.60/user/jni.html?resultof=%22%72%75%6e%74%69%6d%65%22%20%22%72%75%6e%74%69%6d%22%20%22%6c%69%6e%6b%69%6e%67%22%20%22%6c%69%6e%6b%22%20

    >>>>>>text from the above link<<<<<<
    You are recommended to compile your native methods (C or C++ functions called by Java) into AIX shared objects (dynamically loaded libraries). For example, if your native methods are stored in the file nm.c, you could create the shared object with the following command:
    cc_r -qmkshrobj -q64 -I /usr/java6_64/include -o libnm.a nm.c
    >>>>>>>end<<<<<
    Thanks for your suggestions. It's still not working, but I've some more information that might help to understand the problem.

    At the moment I've two versions of our library (which is actually named DMConnector):
    One build with JDK 5 (fullversion: "J2RE 1.5.0 IBM AIX build pap32dev-20070511(SR5)") and one build with JDK 6 (fullversion: "JRE 1.6.0 IBM AIX build pap3260sr9fp2-20110627_03 (SR9 FP2)").

    My test program successfully loads both versions with Java up to version 1.6 SR8 FP1 and it fails since 1.6 SR9 FP2.

    ldd lists the following dependencies for the version build with JDK 5:
    /usr/lib/libc.a(shr.o)
    /unix
    /usr/lib/libcrypt.a(shr.o)

    and the following for the version build with JDK 6:
    /usr/lib/libpthreads.a(shr_xpg5.o)
    /usr/lib/libc.a(shr.o)
    /usr/lib/libpthreads.a(shr_comm.o)
    /unix
    /usr/lib/libcrypt.a(shr.o)

    All files are available, so this shouldn't be a problem.

    I've started the test program with truss. I've attached the output, cause I don't really understand it's meaning.
    The location of both libraries is '/IMPORT/company.all/QA/DM/b93web/AIX64/WE18QA/Application/connector/lib/AIX/'. The one build with JDK 5 is name libDMConnector.so, the one build with JDK 6 is named libDMConnector.1.6.so.
    I've also tried to build the library as suggested in the given link (using -qmkshrobj, but omitting the option -q64 since we have to use 32 bit Java). With this option the linker fails with the following errors:
    ld: 0711-317 ERROR: Undefined symbol: .iconv
    ld: 0711-317 ERROR: Undefined symbol: .iconv_close
    ld: 0711-317 ERROR: Undefined symbol: .iconv_open
    ld: 0711-317 ERROR: Undefined symbol: bqs_rprod
    ld: 0711-317 ERROR: Undefined symbol: bqs_rfile
    ld: 0711-317 ERROR: Undefined symbol: bqs_rtabl
    ld: 0711-317 ERROR: Undefined symbol: bqs_rkey
    ld: 0711-317 ERROR: Undefined symbol: bqs_rfld
    ld: 0711-317 ERROR: Undefined symbol: bqs_rchk
    ld: 0711-317 ERROR: Undefined symbol: bqs_rstrc
    ld: 0711-317 ERROR: Undefined symbol: print
    ld: 0711-317 ERROR: Undefined symbol: sysin
    ld: 0711-317 ERROR: Undefined symbol: sysprint

    I get these errors with the option '-qmkshrobj' independently from the used JDK, e.g. these symbols are also missing in the older version.
    Are these undefined symbols the reason for the whole problem? And if they are: Why aren't we getting the same problem with Java 1.6 SR8 FP1?

    Thanks in advance!

    Attachments

  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-08T09:34:28Z  
    Thanks for your suggestions. It's still not working, but I've some more information that might help to understand the problem.

    At the moment I've two versions of our library (which is actually named DMConnector):
    One build with JDK 5 (fullversion: "J2RE 1.5.0 IBM AIX build pap32dev-20070511(SR5)") and one build with JDK 6 (fullversion: "JRE 1.6.0 IBM AIX build pap3260sr9fp2-20110627_03 (SR9 FP2)").

    My test program successfully loads both versions with Java up to version 1.6 SR8 FP1 and it fails since 1.6 SR9 FP2.

    ldd lists the following dependencies for the version build with JDK 5:
    /usr/lib/libc.a(shr.o)
    /unix
    /usr/lib/libcrypt.a(shr.o)

    and the following for the version build with JDK 6:
    /usr/lib/libpthreads.a(shr_xpg5.o)
    /usr/lib/libc.a(shr.o)
    /usr/lib/libpthreads.a(shr_comm.o)
    /unix
    /usr/lib/libcrypt.a(shr.o)

    All files are available, so this shouldn't be a problem.

    I've started the test program with truss. I've attached the output, cause I don't really understand it's meaning.
    The location of both libraries is '/IMPORT/company.all/QA/DM/b93web/AIX64/WE18QA/Application/connector/lib/AIX/'. The one build with JDK 5 is name libDMConnector.so, the one build with JDK 6 is named libDMConnector.1.6.so.
    I've also tried to build the library as suggested in the given link (using -qmkshrobj, but omitting the option -q64 since we have to use 32 bit Java). With this option the linker fails with the following errors:
    ld: 0711-317 ERROR: Undefined symbol: .iconv
    ld: 0711-317 ERROR: Undefined symbol: .iconv_close
    ld: 0711-317 ERROR: Undefined symbol: .iconv_open
    ld: 0711-317 ERROR: Undefined symbol: bqs_rprod
    ld: 0711-317 ERROR: Undefined symbol: bqs_rfile
    ld: 0711-317 ERROR: Undefined symbol: bqs_rtabl
    ld: 0711-317 ERROR: Undefined symbol: bqs_rkey
    ld: 0711-317 ERROR: Undefined symbol: bqs_rfld
    ld: 0711-317 ERROR: Undefined symbol: bqs_rchk
    ld: 0711-317 ERROR: Undefined symbol: bqs_rstrc
    ld: 0711-317 ERROR: Undefined symbol: print
    ld: 0711-317 ERROR: Undefined symbol: sysin
    ld: 0711-317 ERROR: Undefined symbol: sysprint

    I get these errors with the option '-qmkshrobj' independently from the used JDK, e.g. these symbols are also missing in the older version.
    Are these undefined symbols the reason for the whole problem? And if they are: Why aren't we getting the same problem with Java 1.6 SR8 FP1?

    Thanks in advance!
    For linker error, we need to specify the libraries in which those symbols defined.
    For example,

    ld: 0711-317 ERROR: Undefined symbol: .iconv
    ld: 0711-317 ERROR: Undefined symbol: .iconv_close
    ld: 0711-317 ERROR: Undefined symbol: .iconv_open

    iconv, iconv_close & iconv_open defined in libiconv.a

    from man page,
    Library The iconv Library (libiconv.a)
    Syntax #include <iconv.h>
    size_t iconv (CD, InBuf, InBytesLeft, OutBuf, OutBytesLeft)

    like that, we need to find out the libraries for other symbols & need to pass the libraries to compiler/linker

    cc -qmkshrobj -q32 ... -liconv <<other libraries>>

    The reason for asking to build the library is to veryify whether "-G" option has any effect on the issue.

    As per the definition of "-G" option, it enables runtime linking. http://pic.dhe.ibm.com/infocenter/comphelp/v121v141/topic/com.ibm.xlf141.aix.doc/compiler_ref/opt_g_upper.html?resultof=%22%2d%47%22%20%22%67%22%20

    I have gone through the truss.log, I can see that the JVM is able to locate the library

    statx("/IMPORT/company.all/QA/DM/b93web/AIX64/WE18QA/Application/connector/lib/AIX/libDMConnector.so", 0x3013E360, 128, 010) = 0 , not clear from log why loadx failed to load it.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-08T13:26:51Z  
    For linker error, we need to specify the libraries in which those symbols defined.
    For example,

    ld: 0711-317 ERROR: Undefined symbol: .iconv
    ld: 0711-317 ERROR: Undefined symbol: .iconv_close
    ld: 0711-317 ERROR: Undefined symbol: .iconv_open

    iconv, iconv_close & iconv_open defined in libiconv.a

    from man page,
    Library The iconv Library (libiconv.a)
    Syntax #include <iconv.h>
    size_t iconv (CD, InBuf, InBytesLeft, OutBuf, OutBytesLeft)

    like that, we need to find out the libraries for other symbols & need to pass the libraries to compiler/linker

    cc -qmkshrobj -q32 ... -liconv <<other libraries>>

    The reason for asking to build the library is to veryify whether "-G" option has any effect on the issue.

    As per the definition of "-G" option, it enables runtime linking. http://pic.dhe.ibm.com/infocenter/comphelp/v121v141/topic/com.ibm.xlf141.aix.doc/compiler_ref/opt_g_upper.html?resultof=%22%2d%47%22%20%22%67%22%20

    I have gone through the truss.log, I can see that the JVM is able to locate the library

    statx("/IMPORT/company.all/QA/DM/b93web/AIX64/WE18QA/Application/connector/lib/AIX/libDMConnector.so", 0x3013E360, 128, 010) = 0 , not clear from log why loadx failed to load it.
    Thanks for your input. I will try to find out which libraries are missing and try to run my program again.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-19T13:19:28Z  
    For linker error, we need to specify the libraries in which those symbols defined.
    For example,

    ld: 0711-317 ERROR: Undefined symbol: .iconv
    ld: 0711-317 ERROR: Undefined symbol: .iconv_close
    ld: 0711-317 ERROR: Undefined symbol: .iconv_open

    iconv, iconv_close & iconv_open defined in libiconv.a

    from man page,
    Library The iconv Library (libiconv.a)
    Syntax #include <iconv.h>
    size_t iconv (CD, InBuf, InBytesLeft, OutBuf, OutBytesLeft)

    like that, we need to find out the libraries for other symbols & need to pass the libraries to compiler/linker

    cc -qmkshrobj -q32 ... -liconv <<other libraries>>

    The reason for asking to build the library is to veryify whether "-G" option has any effect on the issue.

    As per the definition of "-G" option, it enables runtime linking. http://pic.dhe.ibm.com/infocenter/comphelp/v121v141/topic/com.ibm.xlf141.aix.doc/compiler_ref/opt_g_upper.html?resultof=%22%2d%47%22%20%22%67%22%20

    I have gone through the truss.log, I can see that the JVM is able to locate the library

    statx("/IMPORT/company.all/QA/DM/b93web/AIX64/WE18QA/Application/connector/lib/AIX/libDMConnector.so", 0x3013E360, 128, 010) = 0 , not clear from log why loadx failed to load it.
    Today I was able to take a look at this problem again and I found all missing symbols. I've added these symbols, build our library and now everything is working again!
    It was neither necessary to build the library with a newer JDK, nor to set the LIBPATH. The new version of the library works with Java 1.5 as well as with Java 1.6 > SR8 FP1.

    Many thanks for your help!
    My problem might be solved, but I still don't really understand whats going on. Out library is just a jni-wrapper for another c-library. The missing symbols where caused by this c-library. Two functions have defined some global variables as 'extern' and everyone who uses one of these functions has to define these global variables. We aren't using these functions and therefore we haven't defined the global variables, what has caused the linker errors. To solve our problem I just had to define these global variables.
    Why do we have to define these global variables for Java 1.6 > SR8 FP1? The corresponding functions are definitely not used and I always thought that the linker removes unused code.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: UnsatisfiedLinkError on AIX since Java 1.6 SR8 FP1

    ‏2012-11-20T06:55:20Z  
    Today I was able to take a look at this problem again and I found all missing symbols. I've added these symbols, build our library and now everything is working again!
    It was neither necessary to build the library with a newer JDK, nor to set the LIBPATH. The new version of the library works with Java 1.5 as well as with Java 1.6 > SR8 FP1.

    Many thanks for your help!
    My problem might be solved, but I still don't really understand whats going on. Out library is just a jni-wrapper for another c-library. The missing symbols where caused by this c-library. Two functions have defined some global variables as 'extern' and everyone who uses one of these functions has to define these global variables. We aren't using these functions and therefore we haven't defined the global variables, what has caused the linker errors. To solve our problem I just had to define these global variables.
    Why do we have to define these global variables for Java 1.6 > SR8 FP1? The corresponding functions are definitely not used and I always thought that the linker removes unused code.
    Happy to hear that your problem got resolved.

    Regarding your question on behavior change,

    Because of Apar: IZ76085, the JVM/Java launcher has enabled with runtime linking from Java 1.6 SR9. As per the runtime linking, when an application(in our case JVM) & the module(your shared library) is enabled with the runtime linking(application using -brtl & the module using -G option), it expects to have all the symbols get resolved at the load time itself.

    Previously, it was not the case as the application(JVM) not enabled with runtime linking & so no such constraint.

    To know more about runtime linking. Refer
    http://www.ibm.com/developerworks/aix/tutorials/linking102/section3.html