IC5Notice: We have upgraded developerWorks Community to the latest version of IBM Connections. For more information, read our upgrade FAQ.
Topic
  • No replies
SystemAdmin
SystemAdmin
196 Posts

Pinned topic $ORIGIN feature for C++ linking on AIX power?

‏2009-06-19T19:09:31Z |
Hello everyone!

I have an C++ application that uses a shared object. I would like to be able to build my program and then run it from another directory. Fine so far. My problem, is that I don't know where the shared library and executable will be installed - my software is provided to another group.

I think that typically the shared object would be installed at /usr/local/lib, or some application-specific path, like /opt/foo1/lib. But in my case, the shared object is side-by-side with the executable. Another group uses our objects and may choose to deploy this in a different directory.

Really, what I want is the $ORIGIN feature that is available on ELF systems. I suppose there is some way to do this with XCOFF, but I can't figure it out.

Here is an example to show the problem - the source files are taken form the "AIX Linking and Loading" guide. Assume 2 libraries (share1.c share2.c) and one executable (main.c)

bb04wrk31:/u/bioliv> cat share1.c
/************
  • share1.c: shared library source.
*************/
#include <stdio.h>
void func1 (){
printf("func1 called\n");
}
void func2 (){
printf("func2 called\n");
}

bb04wrk31:/u/bioliv> cat share2.c
/************
  • share2.c: shared library source.
*************/
void func3 (){
printf("func3 called\n");
}

bb04wrk31:/u/bioliv>
bb04wrk31:/u/bioliv>
bb04wrk31:/u/bioliv> cat main.c
#include <stdio.h>
extern void func1 (),
func2 (),
func3 ();
main (){
func1 ();
func2 ();
func3 ();
}

Build the shared libaries...

bb04wrk31:/u/bioliv> xlC -G -q64 share2.c -o share2.so
bb04wrk31:/u/bioliv> xlC -G -q64 share1.c -o share1.so
Build the main executable...
bb04wrk31:/u/bioliv> xlC -q64 main.c -o main share1.so share2.so

Works fine from current directory...
bb04wrk31:/u/bioliv> ./main
func1 called
func2 called
func3 called

Does not work when called from another directory...
bb04wrk31:/u/bioliv> mkdir foodir
bb04wrk31:/u/bioliv> cd foodir
bb04wrk31:/u/bioliv/foodir> ../main
Could not load program ../main:
Dependent module share1.so could not be loaded.
Could not load module share1.so.
System error: No such file or directory

Here is why...
bb04wrk31:/u/bioliv/foodir> ldd ../main
../main needs:
/usr/lib/libc.a(shr_64.o)
/usr/vacpp/lib/libC.a(shr_64.o)
/usr/vacpp/lib/libC.a(ansi_64.o)
Cannot find share1.so
Cannot find share2.so
/unix
/usr/lib/libcrypt.a(shr_64.o)
/usr/vacpp/lib/libC.a(ansicore_64.o)
/usr/vacpp/lib/libC.a(shrcore_64.o)
/usr/vacpp/lib/libC.a(shr3_64.o)
/usr/vacpp/lib/libC.a(shr2_64.o)
bb04wrk31:/u/bioliv/foodir> dump -H -X64 ../main

../main:

***Loader Section***
Loader Header Information
VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000010 0x00000027 0x00000078

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000006 0x00000428 0x000000ee 0x000004a0
***Import File Strings***
INDEX PATH BASE MEMBER
0 /usr/vac/lib:/usr/vacpp/lib:/usr/lib:/lib
1 libc.a shr_64.o
2 libC.a shr_64.o
3 libC.a ansi_64.o
4 share1.so
5 share2.so
If I knew where the application would be deployed, I could write the install directory into the executable, using -blibpath, like this:
bb04wrk31:/u/bioliv> xlC -q64 main.c -o main share1.so share2.so -blibpath:/u/bioliv/:/usr/local/lib:/usr/lib:/lib
bb04wrk31:/u/bioliv> cd foodir
bb04wrk31:/u/bioliv/foodir> ../main
func1 called
func2 called
func3 called
bb04wrk31:/u/bioliv/foodir> !dump
dump -H -X64 ../main

../main:

***Loader Section***
Loader Header Information
VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x00000010 0x00000027 0x00000076

#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000006 0x00000428 0x000000ee 0x0000049e
***Import File Strings***
INDEX PATH BASE MEMBER
0 /u/bioliv/:/usr/local/lib:/usr/lib:/lib
1 libc.a shr_64.o
2 libC.a shr_64.o
3 libC.a ansi_64.o
4 share1.so
5 share2.so

I could also fix this at run-time using $LIBPATH, but that is not what I want. :-(
Really, what I want is something like this:

bb04wrk31:/u/bioliv> xlC -q64 main.c -o main share1.so share2.so -blibpath:\$ORIGIN:/usr/local/lib:/usr/lib:/lib
But that doesn't seem to work.
Help!

-bill
Updated on 2012-06-19T18:23:16Z at 2012-06-19T18:23:16Z by sandek
  • vijayka
    vijayka
    2 Posts

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2009-06-19T22:08:07Z  

    Have you looked at the possibility of using LD_LIBRARY_PATH environment variable that users of your shared objects may use to find the shared lib?
    Vijay
  • SystemAdmin
    SystemAdmin
    196 Posts

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2009-06-20T00:44:00Z  
    • vijayka
    • ‏2009-06-19T22:08:07Z

    Have you looked at the possibility of using LD_LIBRARY_PATH environment variable that users of your shared objects may use to find the shared lib?
    Vijay

    Hi Vijay!
    On AIX, the environment variable is $LIBPATH, not $LD_LIBRARY_PATH, but I take your meaning.
    Setting $LIBPATH at runtime would cause "main" to work, since additional search directories can be provided. But, this is something that would have to be done by the user.
    I was hoping for a process that I could use at link time, so that the user would not have to set an environment variable.
    -b
  • SystemAdmin
    SystemAdmin
    196 Posts

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2009-06-22T12:49:29Z  

    Hi Vijay!
    On AIX, the environment variable is $LIBPATH, not $LD_LIBRARY_PATH, but I take your meaning.
    Setting $LIBPATH at runtime would cause "main" to work, since additional search directories can be provided. But, this is something that would have to be done by the user.
    I was hoping for a process that I could use at link time, so that the user would not have to set an environment variable.
    -b

    Hi Bill,

    You have a few choices.

    If you know the exact path that your shared libraries will be in while you're building your application, then you're on the right track with the -blibpath compiler option. Just remember that the search path you provide is taken literally (with no variable expansion).

    If you're not sure where your shared libraries will be until runtime, you're also on the right track with LIBPATH. If you would rather not have your users specify this themselves, you could write a shell script to set this just prior to running your application.

    For example, if your program, main, ends up getting installed to the directory, /opt/bill/, you could use the following shell script.
    #!/bin/sh
    export LIBPATH=/opt/bill/:$LIBPATH
    exec /opt/bill/main "$@"

    If your shared libraries end up getting installed to a different path (e.g. /opt/bill/lib) just replace the /opt/bill in the LIBPATH and you should be set.
    #!/bin/sh
    export LIBPATH=/opt/bill/lib:$LIBPATH
    exec /opt/bill/main "$@"

    Now if you really, really want to use your own environment variable (say, $ORIGIN) and have your users specify it, just replace the /opt/bill in your LIBPATH with $ORIGIN.
    #!/bin/sh
    export LIBPATH=$ORIGIN:$LIBPATH
    exec /opt/bill/main "$@"

    Hope this helps,

    • chris
  • Michael_Wong
    Michael_Wong
    29 Posts

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2009-06-23T12:47:17Z  

    Hi Bill,

    You have a few choices.

    If you know the exact path that your shared libraries will be in while you're building your application, then you're on the right track with the -blibpath compiler option. Just remember that the search path you provide is taken literally (with no variable expansion).

    If you're not sure where your shared libraries will be until runtime, you're also on the right track with LIBPATH. If you would rather not have your users specify this themselves, you could write a shell script to set this just prior to running your application.

    For example, if your program, main, ends up getting installed to the directory, /opt/bill/, you could use the following shell script.
    #!/bin/sh
    export LIBPATH=/opt/bill/:$LIBPATH
    exec /opt/bill/main "$@"

    If your shared libraries end up getting installed to a different path (e.g. /opt/bill/lib) just replace the /opt/bill in the LIBPATH and you should be set.
    #!/bin/sh
    export LIBPATH=/opt/bill/lib:$LIBPATH
    exec /opt/bill/main "$@"

    Now if you really, really want to use your own environment variable (say, $ORIGIN) and have your users specify it, just replace the /opt/bill in your LIBPATH with $ORIGIN.
    #!/bin/sh
    export LIBPATH=$ORIGIN:$LIBPATH
    exec /opt/bill/main "$@"

    Hope this helps,

    • chris
    I think this last answer is the only possible way and is the best answer. That is using a wrapper as Chris suggested:
    #!/bin/sh
    export LIBPATH=$ORIGIN:$LIBPATH
    exec /opt/bill/main "$@"
    I spoke with the AIX OS folks, and they tell me there is no facility for a runtime interpreted macro in the linker's -blibpath, i.e. there's no way to get exec to read a user-named environment variable.
  • SystemAdmin
    SystemAdmin
    196 Posts

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2009-06-23T14:48:44Z  
    I think this last answer is the only possible way and is the best answer. That is using a wrapper as Chris suggested:
    #!/bin/sh
    export LIBPATH=$ORIGIN:$LIBPATH
    exec /opt/bill/main "$@"
    I spoke with the AIX OS folks, and they tell me there is no facility for a runtime interpreted macro in the linker's -blibpath, i.e. there's no way to get exec to read a user-named environment variable.

    Too bad there's no $ORIGIN like Solaris. It makes some use cases a lot easier, such as a .so intended to be dynamically loaded into some other process. If one doesn't have control over how that process starts, one can't ensure LD_LIBRARY_PATH/LIBPATH is set effectively.
    Also for setuid/setgid programs, there are security issues with LD_LIBRARY_PATH/LIBPATH that the system enforces, such that the search path may be ignored. $ORIGIN has similar problems, though, so there's no perfect solution here.

  • Michael_Wong
    Michael_Wong
    29 Posts

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2009-06-24T10:29:46Z  

    Too bad there's no $ORIGIN like Solaris. It makes some use cases a lot easier, such as a .so intended to be dynamically loaded into some other process. If one doesn't have control over how that process starts, one can't ensure LD_LIBRARY_PATH/LIBPATH is set effectively.
    Also for setuid/setgid programs, there are security issues with LD_LIBRARY_PATH/LIBPATH that the system enforces, such that the search path may be ignored. $ORIGIN has similar problems, though, so there's no perfect solution here.

    Yes, I agree that all the libpath type environment setting are really hacks anyway for temporarily testing a shared library.

    Proposing something like this as a feature for the AIX OS is a good thought, especially if will help the Solaris folks move to AIX.
  • SystemAdmin
    SystemAdmin
    196 Posts

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2009-06-25T12:28:15Z  
    Yes, I agree that all the libpath type environment setting are really hacks anyway for temporarily testing a shared library.

    Proposing something like this as a feature for the AIX OS is a good thought, especially if will help the Solaris folks move to AIX.
    Thanks for all the posts! This is what I thought, but it is great to know I wasn't missing something.

    How would I propose this to AIX folks? I am an IBM Partner.

    Before posting, I had written a shell script to set necessary environment variables - mirroring approach proposed by Michael and Chris. I include the whole thing here, in case it is useful to others.

    This is for the Firebird open source database. The idea is to put this script "setfbenv.sh" in the directory that is to become the Firebird root. After the shell script is sourced, FIREBIRD is set to the root directory, and on AIX, LIBPATH will include the root.

    
    #! /bin/sh
    #
    <ol> <li>setfbenv.sh</li> </ol> #
    <ol> <li>source this script before running firebird client tools</li> </ol> #
    <ol> <li>directory where this script is located becomes firebird root</li> </ol> #

    <ol> <li>set FIREBIRD, this is the firebird root</li> </ol> FIREBIRD=`cd -P -- "$(dirname -- "$0")" && pwd -P`

    <ol> <li>set LIBPATH or LD_LIBRARY path to include firebird root</li> </ol> if test `uname` = "AIX"; then
    if test -z $LIBPATH; then
    # LIBPATH not set at all
    LIBPATH=$FIREBIRD
    else
    if test $LIBPATH = "${LIBPATH#$FIREBIRD}"; then
    # LIBPATH was set, but did not have FIREIBRD
    LIBPATH=$FIREBIRD:$LIBPATH
    fi
    fi
    else
    if test -z $LD_LIBRARY_PATH; then
    # LD_LIBRARY_PATH not set at all
    LD_LIBRARY_PATH=$FIREBIRD
    else
    if test $LD_LIBRARY_PATH = "${LD_LIBRARY_PATH#$FIREBIRD}"; then
    # LD_LIBRARY_PATH was set, but did not have FIREBIRD
    LD_LIBRARY_PATH=$FIREBIRD:$LD_LIBRARY_PATH
    fi
    fi
    fi

    <ol> <li>set path to include the firebird client tools</li> </ol> if test $PATH = "${PATH#$FIREBIRD}"; then
    PATH=$FIREBIRD:$PATH
    fi

    echo environment configured for firebird client tools on `uname`
    Updated on 2009-06-25T12:28:15Z at 2009-06-25T12:28:15Z by SystemAdmin
  • sandek
    sandek
    1 Post

    Re: $ORIGIN feature for C++ linking on AIX power?

    ‏2012-06-19T18:23:16Z  
    Thanks for all the posts! This is what I thought, but it is great to know I wasn't missing something.

    How would I propose this to AIX folks? I am an IBM Partner.

    Before posting, I had written a shell script to set necessary environment variables - mirroring approach proposed by Michael and Chris. I include the whole thing here, in case it is useful to others.

    This is for the Firebird open source database. The idea is to put this script "setfbenv.sh" in the directory that is to become the Firebird root. After the shell script is sourced, FIREBIRD is set to the root directory, and on AIX, LIBPATH will include the root.

    <pre class="jive-pre"> #! /bin/sh
    #
    <ol> <li>setfbenv.sh</li> </ol> #
    <ol> <li>source this script before running firebird client tools</li> </ol> #
    <ol> <li>directory where this script is located becomes firebird root</li> </ol> #

    <ol> <li>set FIREBIRD, this is the firebird root</li> </ol> FIREBIRD=`cd -P -- "$(dirname -- "$0")" && pwd -P`

    <ol> <li>set LIBPATH or LD_LIBRARY path to include firebird root</li> </ol> if test `uname` = "AIX"; then
    if test -z $LIBPATH; then
    # LIBPATH not set at all
    LIBPATH=$FIREBIRD
    else
    if test $LIBPATH = "${LIBPATH#$FIREBIRD}"; then
    # LIBPATH was set, but did not have FIREIBRD
    LIBPATH=$FIREBIRD:$LIBPATH
    fi
    fi
    else
    if test -z $LD_LIBRARY_PATH; then
    # LD_LIBRARY_PATH not set at all
    LD_LIBRARY_PATH=$FIREBIRD
    else
    if test $LD_LIBRARY_PATH = "${LD_LIBRARY_PATH#$FIREBIRD}"; then
    # LD_LIBRARY_PATH was set, but did not have FIREBIRD
    LD_LIBRARY_PATH=$FIREBIRD:$LD_LIBRARY_PATH
    fi
    fi
    fi

    <ol> <li>set path to include the firebird client tools</li> </ol> if test $PATH = "${PATH#$FIREBIRD}"; then
    PATH=$FIREBIRD:$PATH
    fi

    echo environment configured for firebird client tools on `uname`
    </pre>
    Hi,

    I want to know if IBM has incorporated some mechanism similar to $ORIGIN in the latest release of AIX....
    Thanks
    Sandeep