IBM Advance Toolchain for PowerLinux Documentation

4 likes Updated 7/15/15, 2:13 AM by RaphaelMoreiraZinslyTags:

Table of Contents

  1. Introduction
  2. Installation
  3. Usage
  4. Advance Toolchain Limitations and known issues
  5. Frequently Asked Questions
  6. Support

 

Introduction

 

The IBM Advance Toolchain for PowerLinux is a set of open source development tools and runtime libraries which allows users to take leading edge advantage of IBM's latest POWER hardware features on Linux:

  • POWER8 enablement
  • POWER8 optimized scheduler
  • POWER8 Transactional Memory enablement
  • POWER8 Crypto Operations enablement
  • POWER8 Fusion enablement

Advance Toolchain 8.0 features:

  • Support for big endian (ppc64) and little endian (ppc64le)
  • POWER7 and POWER8 optimized runtime libraries

Advance Toolchain 7.1 features:

  • Support for little endian (ppc64le)
  • POWER7 and POWER8 optimized runtime libraries

Advance Toolchain 7.0 features:

  • POWER7 and POWER8 optimized runtime libraries

Advance Toolchain 6.0 features:

  • POWER7 optimized runtime libraries

Advance Toolchain releases up to version 5.0 feature:

  • PPC970, POWER4, POWER5, POWER5+, POWER6, POWER6x and POWER7 optimized runtime libraries

This self-contained toolchain does not rely on the system toolchain and requires minimum dependencies. Nor does it override the default Linux distribution toolchain (it is installed in /opt). The latest release includes current stable versions of the following packages:

  • GNU Compiler Collection (gcc, g++ and gfortran), plus individually optimized gcc runtime libraries for supported POWER processors
  • GNU C library (glibc), individually optimized for supported POWER processors
  • GNU Binary Utilities (binutils)
  • Decimal Floating Point Library (libdfp), optimized with hardware DFP support for POWER6, POWER6x and POWER7 but also available in emulation for PPC970, POWER4, POWER5, and POWER5+
  • AUXV Library (libauxv)
  • GNU Debugger (gdb)
  • Performance analysis tools (oprofile, valgrind, itrace)
  • Muti-core exploitation libraries (Intel TBB, Userspace RCU, SPHDE), starting in version 5.0-3
  • Plus several support libraries (libhugetlbfs, Boost, zlib, etc)

For more information about AUXV and Optimized Libraries, see this technical report.

 

Supported Linux Distributions

 

Supported Linux distributions according to the Advance Toolchain version and their respective links for manual download.

  Advance Toolchain 8.0 ³ Advance Toolchain 7.1 ² Advance Toolchain 7.0 ¹ Advance Toolchain 6.0 ¹
Fedora 18   Cross-compiler Native & Cross-compiler Native & Cross-compiler
Fedora 19 Native & Cross-compiler Native & Cross-compiler Native & Cross-compiler  
RHEL 5     Runtime compatibility Runtime compatibility
RHEL 6 Cross-compiler Cross-compiler Native & Cross-compiler Native & Cross-compiler
RHEL 7 Native & Cross-compiler Native & Cross-compiler Native & Cross-compiler  
SLES 10     Runtime compatibility Runtime compatibility
SLES 11   Cross-compiler Native & Cross-compiler Native & Cross-compiler
SLES 12 Native & Cross-compiler      
Ubuntu 12.04     Cross-compiler  
Ubuntu 14.04 Native & Cross-compiler Native & Cross-compiler    
Ubuntu 14.10 Native & Cross-compiler    

¹ All versions of Advance Toolchain until 7.0 (inclusive) support big endian only.

² Advance Toolchain 7.1 supports little endian only.

³ Advance Toolchain 8.0 supports both big and little endian.

This includes all the service packs and update releases for the mentioned Linux distributions.

In addition, we provide a runtime compatibility package for older Linux distributions, so binaries built by the Advance Toolchain can be run on those.

For more information about each individual release, please refer to the release notes file available at the Advance Toolchain file repositories below.

 

Installation

 

The gpg public keys gpg-pubkey-6976a827-5164221b (for RHEL6/7, SLES 12 and Ubuntu) and gpg-pubkey-3052930d-5175955a (for RHEL6, RHEL5, SLES 10 and SLES 11) are provided in the repositories. These public keys can be used to verify the authenticity of both the Advance Toolchain packages and the repository contents. Download the gpg public key for your Linux distribution and import it using the following commands:

RPM based distributions:
wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL7/gpg-pubkey-6976a827-5164221b
rpm --import gpg-pubkey-6976a827-5164221b
Ubuntu:
wget ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu/dists/precise/6976a827.gpg.key
sudo apt-key add 6976a827.gpg.key

Warnings:

  • RHEL6 requires both gpg public keys.
  • On SLES10 and RHEL5, please install the ncurses package (64-bit) before installing the Advance Toolchain.
  • atX.X in the instructions below should be replaced by the version you are using (for example, at6.0, at7.0, etc)

 

Using YaST or Zypper (SLES10/11/12)

 

YaST

To install using YaST, execute yast2 as root. Then follow these steps:

  1. Select Add-on Product.
  2. Select the FTP Protocol:
    (x) FTP...
  3. Under Server Name:
    ftp.unicamp.br
  4. Under Directory on Server:
    /pub/linuxpatch/toolchain/at/suse/SLES_12 or /pub/linuxpatch/toolchain/at/suse/SLES_11 or /pub/linuxpatch/toolchain/at/suse/SLES_10
  5. You will get a warning about there being no product information available at the given location. This is because the repomd based repository does not contain the YaST product information. This is not a bug. Select [Continue].
  6. Under the Software Management interface, search for "advance toolchain" and mark the advance-toolchain-atX.X-runtime, advance-toolchain-atX.X-devel, advance-toolchain-atX.X-perf and advance-toolchain-atX.X-mcore-libs packages for installation as necessary and click [Accept]. If you are installing a cross compiler, select advance-toolchain-atX.X-cross-ppc64 or advance-toolchain-atX.X-cross-ppc64le.

Zypper

To install using Zypper, you first need (as root) to add the Unicamp FTP to the repository list:

zypper addrepo ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/suse/SLES_12 "Advance Toolchain"

This will create a new repository entry called "Advance Toolchain" pointing to the Unicamp FTP site. Then, execute the following to install:

zypper install advance-toolchain-atX.X-runtime \
               advance-toolchain-atX.X-devel \
               advance-toolchain-atX.X-perf \
               advance-toolchain-atX.X-mcore-libs

Or to install the cross compiler, run:

zypper install advance-toolchain-atX.X-cross-ppc64

You will be prompted to trust the repository key (AdvanceToolchain05). Hit "a" for always trusting this key, then "y" to confirm packages installation. Optionally, you may also install the advance-toolchain-atX.X-mcore-libs package, which contains the multi-core exploitation libraries (versions 5.0-3 and above), and the advance-toolchain-atX.X-selinux package (versions 5.0-7 and above), which enables compatibility with SELinux.

Warning: Do not try to install the advance-toolchain-atX.X-selinux package if you do not have SELinux installed and enabled.

YaST and Zypper support package upgrades for new revision releases (i.e. 6.0-0 to 6.0-1). For new major releases (i.e. 5.0-8 to 6.0-1), please proceed as in a normal installation.

Warning: When upgrading using Zypper on SLES 10, use the following command:

zypper update -t package advance-toolchain-at5.0-runtime

The -t package option is required on SLES 10.

 

Using YUM (RHEL5/6/7 and Fedora 18/19)

 

First, if you have never created an Advance Toolchain repository configuration file, you need to do so. Create the following file using the text editor of your choice as root:

/etc/yum.repos.d/atX.X.repo

and add the following content:

# Beginning of configuration file
[atX.X]
name=Advance Toolchain Unicamp FTP
baseurl=ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL7
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL7/gpg-pubkey-6976a827-5164221b
# End of configuration file

Notes

  • When installing on RHEL6 or Fedora 18, point to the RHEL6 baseurl, and include both gpgkeys:
    baseurl=ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL6
    gpgkey=ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL6/gpg-pubkey-6976a827-5164221b
           ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL6/gpg-pubkey-3052930d-5175955a
    
  • When installing on RHEL5, remember to point both baseurl and gpgkey to RHEL5 repository:
    baseurl=ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL5
    gpgkey=ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/redhat/RHEL5/gpg-pubkey-3052930d-5175955a
    
  • Fedora 19 uses the same repository of RHEL7.

You may check if the repository was added correctly by running yum repolist. You will see your new atX.X repository listed if the configuration file is correct.

To install, execute yum install as root:

yum install advance-toolchain-atX.X-runtime \
            advance-toolchain-atX.X-devel \
            advance-toolchain-atX.X-perf \
            advance-toolchain-atX.X-mcore-libs

Or to install a cross compiler:

yum install advance-toolchain-atX.X-cross-ppc64

Answer "y" when YUM prompts for confirmation. Optionally, you may also install the advance-toolchain-atX.X-mcore-libs package, which contains the multi-core exploitation libraries (versions 5.0-3 and above), and the advance-toolchain-atX.X-selinux package (versions 5.0-7 and above), which enables compatibility with SELinux.

Warning: Do not try to install the advance-toolchain-atX.X-selinux package if you do not have SELinux installed and enabled.

YUM supports package upgrades for new revision releases (i.e. 5.0-0 to 5.0-1). For new major releases (i.e. 5.0-8 to 6.0-1), please proceed as in a normal installation.

 

Manual RPM installation

 

If you are installing the rpms manually, you will need to download them from our repositories and install in the following order (due to prerequisites):

advance-toolchain-atX.X-runtime-X.X-X

advance-toolchain-atX.X-devel-X.X-X

advance-toolchain-atX.X-perf-X.X-X

advance-toolchain-atX.X-mcore-libs-X.X-X

where X.X-X is the actual version of the package (i.e. 6.0-1). Optionally, you may also install the advance-toolchain-atX.X-mcore-libs package, which contains the multi-core exploitation libraries (versions 5.0-3 and above), and the advance-toolchain-atX.X-selinux package (versions 5.0-7 and above), which enables compatibility with SELinux. These packages depend on advance-toolchain-atX.X-runtime.

Warning: Do not try to install the advance-toolchain-atX.X-selinux package if you do not have SELinux installed and enabled.

If you are updating your current installation to a new revision release (i.e. 7.0-0 to 7.0-1), you need to update as a batch, so rpm can solve the dependencies:

rpm -Uvh advance-toolchain-atX.X-runtime-X.X-X.ppc64.rpm \
         advance-toolchain-atX.X-devel-X.X-X.ppc64.rpm \
         advance-toolchain-atX.X-perf-X.X-X.ppc64.rpm \
         advance-toolchain-atX.X-mcore-libs-X.X-X.ppc64.rpm

The rpm command should always be run as root. Failure to do so will result in an error similar to this:

error: can't create transaction lock on /var/lib/rpm/__db.000

In case you installed or updated the Advance Toolchain incorrectly, you might end up with an inconsistent rpm database (i.e. showing that the rpms are installed, but no files present in /opt/atX.X). In that case, you will have to force-remove the rpms and install again. To force-remove all the Advance Toolchain rpms:

rpm -e --force advance-toolchain-*

Then reinstall using any of the procedures described above.

 

Using Aptitude or apt (Ubuntu 12.04/14.04/14.10)

 

Configure the repositories

If you have never configured Advance Toolchain repositories in this system, you need to do so. Add the following line to /etc/apt/sources.list using your editor of choice as root.

On i386 or amd64:

deb [arch=i386] ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty at8.0

On ppc64el:

deb ftp://ftp.unicamp.br/pub/linuxpatch/toolchain/at/ubuntu trusty at8.0

Aptitude

After configuring the repository, refresh the aptitude cache by running:

sudo aptitude update

Finally, install the Advance Toolchain package running the following command:

sudo aptitude install advance-toolchain-atX.X-runtime \
                      advance-toolchain-atX.X-devel \
                      advance-toolchain-atX.X-perf \
                      advance-toolchain-atX.X-mcore-libs

Or to install a cross compiler:

sudo aptitude install advance-toolchain-atX.X-cross-ppc64

Aptitude supports package upgrades for new revision releases (i.e. 7.1-0 to 7.1-1). For new major releases (i.e. 7.1-1 to 8.0-0), please proceed as in a normal installation.

apt

After configuring the repository, refresh the apt cache by running:

sudo apt-get update

Finally, install the Advance Toolchain package running the following command:

sudo apt-get install advance-toolchain-atX.X-runtime \
                     advance-toolchain-atX.X-devel \
                     advance-toolchain-atX.X-perf \
                     advance-toolchain-atX.X-mcore-libs

Or to install a cross compiler:

sudo apt-get install advance-toolchain-atX.X-cross-ppc64

APT supports package upgrades for new revision releases (i.e. 7.1-0 to 7.1-1). For new major releases (i.e. 7.1-1 to 8.0-0), please proceed as in a normal installation.

 

Packages Descriptions

 

In most cases, you do not need to install all of the packages that are provided with the Advance Toolchain. The following list describes when they are needed:

  • advance-toolchain-atX.X-runtime

    Provides base functionality to run Advance Toolchain applications. This package is always required.

  • advance-toolchain-atX.X-runtime-compat

    Substitutes the above package on previous distributions versions. More information about SLES9 and RHEL4 or SLES10 and RHEL5

  • advance-toolchain-atX.X-devel

    Provides development tools. This package is only needed when developing applications.

  • advance-toolchain-atX.X-perf

    Provides tools for measuring performance. It's only useful on some development environments.

  • advance-toolchain-atX.X-mcore-libs

    Provides libraries for multi-thread development, like Boost, SPHDE and Intel Threading Building Blocks. This package is also required on servers running the applications developed with those libraries.

  • advance-toolchain-atX.X-runtime-atZZ-compat

    Install this package only if you need to run an application built with the previous version of the Advance Toolchain on top of the current version. More information on Runtime Compatibility Between Advance Toolchain Versions

  • advance-toolchain-atX.X-selinux

    Provides SELinux settings. Only required when developing or running Advance Toolchain applications on a SELinux enabled environment.

  • advance-toolchain-atX.X-cross-ppc64 or advance-toolchain-atX.X-cross-ppc64le

    Provide a cross compiler for big endian (ppc64) or little endian (ppc64le). These packages are available for Intel-based machines (i386 or amd64) in order to generate binaries for POWER.

  • advance-toolchain-atX.X-cross-common

    Provides files common to both cross compiler packages (ppc64 and ppc64le). This package is mandatory for cross compiler installation starting from version 8.0.

Important Notes

  • advance-toolchain-atX.X-runtime-compat-X.X-X
    • Advance Toolchain 5.x: DO NOT INSTALL on SLES10 and beyond or RHEL5 and beyond. These are intended for runtime compatibility on SLES9 and RHEL4 only (see below). By doing so, you will corrupt your entire Advance Toolcahin installation and you will have to force-remove all the rpms and perform a new installation (see above for instructions on how to do it). If you are getting some strange "Illegal instruction" or "Segmentation fault" errors in runtime when you are absolutely sure those shouldn't be happening, you may want to verify if the compatibility rpm is installed:
      rpm -qa | grep advance-toolchain

      And check if it returns advance-toolchain-atX.X-runtime-compat-X.X-X.

    • Advance Toolchain 6.x and 7.x: DO NOT INSTALL on SLES11 and beyond or RHEL6 and beyond, for the same reasons cited above.
    • Advance Toolchain 8.x: DO NOT INSTALL on RHEL7 and beyond, for the same reasons cited above.
  • The Advance Toolchain rpms were designed so that you can install multiple versions of it in your machine if necessary. Different versions will install in different paths. (for example, /opt/at7.0 and /opt/at8.0 for Advance Toolchain 7.0-x and Advance Toolchain 8.0-y).

 

Installation on SLES9 and RHEL4 — Runtime Compatibility (version 5.x)

 

Users running applications on SLES9 or RHEL4 may install the compatibility rpm advance-toolchain-atX.X-runtime-compat-X.X-X in order to get the Advance Toolchain optimized runtime libraries. For SLES9 and RHEL4, optimized libraries are available for:

  • PPC970
  • POWER4
  • POWER5
  • POWER5+

 

Installation on SLES10, RHEL5/6 — Runtime Compatibility (version 6.x and beyond)

 

Users running applications on SLES10, RHEL5 or RHEL6 may install the compatibility rpm advance-toolchain-atX.X-runtime-compat-X.X-X in order to get the Advance Toolchain runtime libraries. There are no optimized libraries for the runtime compatibility rpm starting on version 6.0-0.

 

Runtime compatibility between Advance Toolchain versions

 

Users running applications built with an older version of the Advance Toolchain may install the compatibility rpm advance-toolchain-atX.X-runtime-atZ.Z-compat-X.X-X in order to run these applications on top of a newer version of the Advance Toolchain runtime, for example, using AT 7.0 to run applications built with AT 6.0.

After installing this package, please run:

/etc/rc.d/init.d/atX.X-runtime-atZ.Z-compat start

 

Using the Cross Compiler (Intel x86)

 

The cross compiler provides a way to compile programs for POWER using an Intel-based machine (i386 or amd64), helping teams to start developing for POWER without having access to a server in the early development phase. However it does not completely remove the requirement of a POWER server to run a complete test in a proper environment.

For more information about using the cross compiler, visit Building with Advance Toolchain cross compiler documentation.

 

Usage

 

After completing the installation steps, the Advance Toolchain is ready for usage. Just call the program directly, for example /opt/atX.X/bin/gcc.

However, some applications have complex build systems (for example: autotool, make, cmake) in which is necessary to correctly set the environment PATH, for example:

PATH=/opt/atX.X/bin:/opt/atX.X/sbin:$PATH make

 

Manual Pages

 

In order for the system man application to pick up Advance Toolchain installed manual pages, you must export the location of the AT manual pages in the MANPATH variable prior to invoking man. This is best accomplished with the following commands:

unset MANPATH
export MANPATH="/opt/atX.X/share/man:`manpath`"
man <topic>
  

Or you may override the current environment MANPATH as demonstrated in the following example:

MANPATH="/opt/atX.X/share/man:`manpath`" man lsauxv
  

 

Optimization Selection

 

Directing gcc to build an application for a particular CPU can take advantage of processor-specific instruction selection. In some cases, it can significantly improve performance. Building without selecting a particular CPU simply causes gcc to select the default (lowest common denominator) instruction set.

-mcpu=power4

-mcpu=970

-mcpu=power5

-mcpu=power5+

-mcpu=power6

-mcpu=power6x

-mcpu=power7

-mcpu=power8

Warnings

  • On Advance Toolchain 7.x and 8.0, the compiler defaults to -mcpu=power7 -mtune=power8
  • On Advance Toolchain 6.0, the compiler defaults to -mcpu=power6 -mtune=power7
  • When using -mcpu=power7, DO NOT disable Altivec (for example, -mno-altivec) without also disabling VSX (for example, -mno-vsx). The following combination is illegal:
    -mcpu=power7 -mno-altivec

 

Common GCC options

 

  • -fpeel-loops

    Peels the loops so that there is enough information that they do not roll much (from profile feedback). This value also turns on complete loop peeling; complete removal of loops with small constant number of iterations.

  • -funroll-loops<

    Unroll loops whose number of iterations can be determined at compile time or upon entry to the loop. The -funroll-loops value implies the use of the -frerun-cse-after-loop value. This option makes the code larger, and may or may not make it run faster.

  • -ftree-vectorize

    Perform loop vectorization on trees. This flag is enabled by default at -O3, starting at the GCC 4.3 time frame.

  • -ffast-math

    Sets the -fno-math-errno, -funsafe-math-optimizations, -ffinite-math-only, -fno-rounding-math, -fno-signaling-nans and -fcx-limited-range values. This option causes the preprocessor macro _ FAST_MATH _ to be defined. This option is not turned on by any -O option because it can result in incorrect output for programs which depend on an exact implementation of IEEE or ISO rules or specifications for math functions. It may, however, yield faster code for programs that do not require the guarantees of these specifications.

  • -mcmodel=medium

    Generate PowerPC64 code for the medium model: The TOC and other static data may be up to a total of 4G in size.

 

GCC options for older Linux distributions

 

The Advance Toolchain shows significant gains over the older (SLES10 SP3) Enterprise Linux Distributions, because these releases use an older GCC-4.1 compiler that is not enabled for POWER7. In addition, these releases are restricted to POWER6 compatibility mode and cannot leverage the new instructions enabled for POWER7 for applications or runtime libraries. There are, however, some best practices that you can use to maximize your performance.

Best practice for the -mcpu and -mtune values

  • If your program will be running on the same POWER systems for the foreseeable future, built with the matching -mcpu value.
  • If your program needs to run on multiple systems, the best strategy is to build with the -mcpu set for the oldest supported system. If your program mostly runs on a newer system with the older system for accommodation or backup, then use the -mcpu set for the oldest system combined with the -mtune value for system that you want to optimize for performance.
  • If your program in running in POWER6 compatibility mode on a POWER7 System, use the -mcpu=power6 and -mtune=power7 values.
  • If your code is organized in dynamic libraries, you can compile and build your libraries multiple times using the -mcpu value for the specific Power platform and then install those libraries into the matching /lib64/power directory. This allows the dynamic linker to automatically select the dynamic library optimized for the specific system.

Best practice for large programs

  • Avoid compiling with the -mminimal-toc value as this adds extra levels of indirection for static data accesses
  • Compile with the -mcmodel=medium value to optimize static data access and allow the linker to perform additional optimizations on the final program or library image.
  • For programs with static data exceeding 2 GB, you may need to use the -mcmodel=large value.

 

Platform and Hardware Capabilities Determination

 

The Advance Toolchain includes libauxv and lsauxv, a system library and application, respectively, which provide a mechanism for querying the system platform information from the kernel's auxiliary vector. The system hardware capabilities (hwcap) may be queried through the auxiliary vector as well. For example, the platform can be queried dynamically with the following function:

char * platform = (char *) query_auxv (AT_PLATFORM);

Information from the hwcap can be queried in the following manner:

unsigned long int hwcap_mask = (unsigned long int) query_auxv (AT_HWCAP);
if (hwcap_mask & PPC_FEATURE_HAS_FPU)
    printf(" HAS_FPU\n");
  

For more information about using libauxv and lsauxv can be found in the auxv and lsauxv manual pages provided by the Advance Toolchain (see Manual Pages above).

 

Relinking a Pre-built Application with the Advance Toolchain

 

Locate all of the application's .o files. You can also link .a files to pick them all up at once. These will be needed for the relink.

Locate the paths to all of the necessary linked shared-object files, for example:

/usr/X11R6/lib for libXrender

/opt/gnome/lib for libgtk-x11-2.0

Edit /opt/atX.X/etc/ld.so.conf and add the directories to all of the shared object files to the end of this file. Don't forget lib64 for the 64-bit equivalent libraries if applicable, for example:

/opt/gnome/lib/

/opt/gnome/lib64/

/usr/X11R6/lib

/usr/X11R6/lib64/

Run the Advance Toolchain's ldconfig application to regenerate /opt/atX.X/etc/ld.so.cache, for example:

sudo /opt/atX.X/sbin/ldconfig

The loader uses /opt/atX.X/etc/ld.so.cache to find the libraries the application was linked against.

Relink using the Advance Toolchain's compiler:

/opt/atX.X/bin/gcc -g -O2 -o <application_name> <list_of_dot_o_files> \
                   <list_of_dot_a_files> -L<path_to_libraries> \
                   -l<one_for_each_library_needed_for_the_link>
  

For example:

/opt/at5.0/bin/gcc -g -O2 -o mandelbrot callbacks.o interface.o \
    main.o quadmand.o support.o mandel_internals.a \
    -L/usr/X11R6/lib -L/usr/X11R6/lib64 -L/opt/gnome/lib -lgtk-x11-2.0
    -lgdk-x11-2.0 -latk-1.0 -lgdk_pixbuf-2.0 \
    -lpangocairo-1.0 -lpango-1.0 -lcairo -lgobject-2.0 -lgmodule-2.0 -ldl \
    -lglib-2.0 -lfreetype -lfontconfig \
    -lXrender -lX11 -lXext -lpng12 -lz -lglitz -lm -lstdc++ -lpthread \
    -lgthread-2.0
  

If ld gives an error like the following then you're missing the path to that library in the link stage. Add it with -L<path to library>, e.g.

/opt/at5.0/bin/ld: cannot find -lgtk-x11-2.0

Add -L/opt/gnome/lib/ to the gnome compilation line. You need to tell the linker where to find all of the libraries.

When running the relinked application, if you get an error like the following:

./mandelbrot: error while loading shared libraries: libglib-2.0.so.0: cannot open shared object file: No such file or directory.

You need to add the path to the library in question to /opt/atX.X/etc/ld.so.conf and re-run /opt/atX.X/sbin/ldconfig. The Advance Toolchain's loader needs to know where to find the libraries and uses the generated /opt/atX.X/etc/ld.so.cache to find them.

You can verify that the Advance Toolchain libraries were picked up by running the application prefaced with LD_DEBUG=libs, e.g.

LD_DEBUG=all ./mandelbrot

Warning: do NOT use LD_LIBRARY_PATH to point to the Advance Toolchain libraries if your applications are not relinked with the Advance Toolchain. Doing so can result in ld.so and libc.so version mismatch and cause runtime failures.

 

Library Search Paths

 

/opt/atX.X/etc/ld.so.conf already includes /etc/ld.so.conf in the search order, but you may need to re-run /opt/atX.X/sbin/ldconfig in order populate /opt/atX.X/etc/ld.so.cache with the specialized search paths you have added to /etc/ld.so.conf after an Advance Toolchain installation.

If you are running ldd against your binary and it is showing that some libraries are not found, you may need to re-run /opt/atX.X/sbin/ldconfig to fix that.

 

Using the Advance Toolchain with libhugetlbfs-2.0 (and later)

 

The Advance Toolchain provides its own 32-bit and 64-bit versions of libhugetlbfs 2.x. Please refer to the Advance Toolchain's libhugetlbfs man page for more information on using libhugetlbfs-2.0:

unset MANPATH
export MANPATH="/opt/atX.X/share/man:`manpath`"
man libhugetlbfs
  

 

Using the Advance Toolchain with libhugetlbfs-1.0

 

Warning: Libhugetlbfs 1.0 is deprecated for new users. Please use libhugetlbfs 2.0 (or later) that's provided directly by the Advance Toolchain. If you must use libhugetlbfs 1.0 follow these instructions.

The /opt/atX.X/scripts/createldhuge.sh script is provided (until Advance Toolchain 8.0), which copies /opt/atX.X/bin/ld to /opt/atX.X/bin/ld.orig and creates a wrapper script in /opt/atX.X/bin/ld. You only need to run this if you want the Advance Toolchain to work with libhugetlbfs.

The new /opt/atX.X/bin/ld is a wrapper script which detects whether the --hugetlbfs-link or --hugetlbsf-align switches have been passed to the linker. If so then it sets a script-local LD environment variable to /opt/atX.X/bin/ld.orig and invokes the system's ld.hugetlbfs, e.g.

LD="/opt/atX.X/bin/ld.orig" /usr/share/libhugetlbfs/ld.hugetlbfs *switches*

If it doesn't detect the hugetlbfs-link/hugetlbfs-align switch, then it simply forwards the linker invocation to /opt/atX.X/bin/ld.orig directly.

If libhugetlbfs support is desired, the first thing to do is backup the original Advance Toolchain linker just in case there are problems and you need to restore it manually.

cp -p /opt/atX.X/bin/ld /opt/atX.X/bin/ld.backup

The scripts in /opt/atX.X/scripts/ will do the rest of the work for you:

createldhuge.sh restoreld.sh

Invoke createldhuge.sh to create the wrapper ld:

sudo sh createldhuge.sh /<prefix-to-libhugetlbfs>/share/libhugetlbfs/ld.hugetlbfs /opt/atX.X

This MUST be executed as sudo (or root) for the ld wrapper script to be created properly.

When/If you want to restore the original Advance Toolchain linker, simply run:

sudo sh restoreld.sh.

The Advance Toolchain gcc always ignores the -B/<prefix-to-libhugetlbfs>/share/libhugetlbfs directive because it has been built to always invoke /opt/atX.X/bin/ld directly. You can use the gcc invocation you have always used, e.g.

/opt/atX.X/bin/gcc temp.c -v -o temp -B/<prefix-to-libhugetlbfs>/share/libhugetlbfs/ -Wl,--hugetlbfs-link=BDT

Note: If you invoke /opt/atX.X/bin/ld --hugetlbfs-link=BDT directly, you'll need to supply a -m* flag which is normally provided by gcc directly (see man ld for supported emulations).

 

Packaging an application built with the Advance Toolchain using RPM

 

Due to a limitation in the way that RPM generates dependency lists, the symbols from the Advance Toolchain may collide with the ones provided by the Linux distribution. There are two ways to prevent this:

  • Manually set your dependencies, using the steps below:
    • Change your spec file, adding the entry Autoreq: 0 to disable the auto requirements check;
    • Perform the build of your application as usual and then check every component manually for its dependencies (usually using ldd to find out shared library dependencies);
    • Use the shared library list obtained in the previous step and replace the Advance Toolchain entries found by a single advance-toolchain-atX.X-runtime, where X.X is the Advance Toolchain version being used for the build;

      Note: If any libraries from the Advance Toolchain mcore-libs package have being used, their entries must be replaced by a single advance-toolchain-atX.X-mcore-libs entry instead;
    • Add an entry of Requires: <list-found> on your spec file, replacing the "<list-found>" with the list assembled in the previous step;
    Note: These steps are only required for the first build to adjust your spec file for packaging, unless changes in the build system, or other packaging spec defined changes occur (in which case, it must be revised to guarantee its functionality).
  • Use personalized require dependency filters, using the steps below:
    • Since RPM version 4.8, you can replace the macro that checks for dependencies, so we must, at first, create an external script that will perform such search. This script must accept a list of files as input and provide a list of found dependencies as output. This script must be provided by you, and it should look for standard dependencies found inside the Advance Toolchain install path and replace them by a single output of advance-toolchain-atX.X-runtime, or advance-toolchain-atX.X-mcore-libs if the dependency found was a multi core library provided by Advance Toolchain. Remember that any other dependency found that are out of Advance Toolchain bounds should be passed through as it was (unchanged).
    • Once this script is working, change your spec file, redefining the macro required using the following entry:

      %__find_requires </full/path/to/script>

      RPM by default points the %__find_requires macro to its standard script (which we recommend review of as a reference) located on /usr/lib/rpm/find_requires. Take a look at this link for more information about how it could be done.

Note: The second method is best used with RPM version 4.8 or later. With earlier RPM versions, the first method is recommended.

 

Using the Advance Toolchain with IBM XLC and XLF

 

When compiling binaries using XLC or XLF, the user must add the -F <path_to_cfg_file> option to the compiler command line. The Advance Toolchain provides a script that creates those files in /opt/atX.X/scripts. This script is automatically run during installation. If you need to re-run it later (i.e. you installed XLC/XLF after the Advance Toolchain installation), execute the following command:

/opt/atX.X/scripts/at-create-ibmcmp-cfg.sh

Please notice the absolute path when calling the script. DO NOT call it using a relative path. The script creates the config files in /opt/atX.X/scripts.

This procedure does not affect the default XLC/XLF configuration.

 

Debugging

 

gdb can be asked to output _Decimal[32|64|128] formatted floating point registers by default, using its printf command.

When using objdump to inspect POWER6 code, make sure to use the -Mpower6 flag:

/opt/atX.X/bin/objdump -d -Mpower6 <your_file>

The same applies to POWER7 code.

 

Time zones

 

Since AT 5.0-7 and AT 6.0-1, the Advance Toolchain automatically sets its time zone to the same time zone used by the system. This is done with the symbolic link /opt/atX.X/etc/localtime to /etc/localtime. To replicate this behavior in the Advance Toolchain 5.0-6, 6.0-0 and earlier versions, it is necessary to replace the old symbolic link:

rm -f /opt/atX.X/etc/localtime && ln -s /etc/localtime /opt/atX.X/etc/localtime

By doing so, further changes of system's time zone will automatically reflects in Advance Toolchain's time zone.

If necessary, the AT can work with a different time zone. All the available time zones are listed in the directory /opt/atX.X/share/zoneinfo.

Firstly, remove the symbolic link /opt/atX.X/etc/localtime:

rm -f /opt/atX.X/etc/localtime

Then, create a symbolic link to the new time zone, i.e.:

ln -s /opt/atX.X/share/zoneinfo/Europe/Athens /opt/atX.X/etc/localtime

 

SELinux issue

 

There is a bug on RHEL6 that prevents restorecon to restore the context of symbolic links. This bug affects AT's time zone settings. While Red Hat works to solve this bug, a workaround is available.

Replace the symbolic link by a hard link:

rm -f /opt/atX.X/etc/localtime
ln /etc/localtime /opt/atX.X/etc/localtime
  

However, if /etc and /opt are stored in different file systems the only solution available is to copy the system's time zone:

rm -f /opt/atX.X/etc/localtime cp /etc/localtime /opt/atX.X/etc/localtime

 

Advance Toolchain Limitations and Known Issues

 

  • oprofile's operf tool running on SLES 11: if running operf as the root user, only --system-wide or --pid modes are supported. This is a permanent restriction.
  • gcc no longer has support for Graphite loop optimizations starting on Advance Toolchain 6.0-0.
  • libdfp has some known limitations as indicated in the libdfp README.user documentation.
  • oprofile currently doesn't support profiling using JVMPI.
  • oprofile Java profiling restriction: To use oprofile for profiling Java VMs 1.5 or greater, users generally have two options available when invoking the JVM: -agentlib or -agentpath. However, when using the Advance Toolchain's oprofile, you must use
    -agentpath:/opt/at5.0/lib64/oprofile/libjvmti_oprofile.so
    for 64-bit JVMs, or
    agentpath:/opt/at5.0/lib/oprofile/libjvmti_oprofile.so
    for 32-bit JVMs.
  • Upgrading from 5.0-0, 5.0-1 or 5.0-2 to a newer release is not working. The problem was fixed in 5.0-3 and does not affect that, nor newer releases. If you need to upgrade from 5.0-0, 5.0-1 or 5.0-2, please uninstall the Advance Toolchain and then install the new version.

 

Frequently Asked Questions

 

  1. How do I find which package provides a specific feature?
  2. What does ";undefined reference to sem_*" mean?
  3. Why doesn't the Advance Toolchain find my library?
  4. Why does my program fail with "No such file or directory"?
  5. Can I set RPATH/RUNPATH with the Advance Toolchain?
  6. Can I mix parts of different glibc versions?
  7. How to verify if the Advance Toolchain is corrupted?

 

How do I find which package provides a specific feature?

 

On RHEL based systems you can run:

yum provides <feature>

On SLES you need to run the following command:

zypper what-provides <feature>

 

What does "undefined reference to sem_*" mean?

 

These kind of errors usually appear when trying to link a multithreaded program without enabling multithreading support (i.e. specify -pthread when calling the linker), e.g.:

/opt/at7.0/bin/gcc main.c -pthread

 

Why doesn't the Advance Toolchain find my library?

 

The loader keeps a cache of all libraries available in the library path (/opt/atX.X/etc/ld.so.conf). Every time a new library is installed in one of the directories of the library path or when the library path is changed, it is necessary to update this cache by running /opt/atX.X/sbin/ldconfig as root.

 

Why does my program fail with "No such file or directory"

 

The symptom of this issue is: a program built with the Advance Toolchain fails to execute with the error message bash: ./program: No such file or directory.

This message is indicating the system was not able to run this program. Usually this happens because the system wasn't able to find the correct major version of the Advance Toolchain. In other words, Bash didn't find the loader specified in the .interp section of the executable.

You can identify which Advance Toolchain version built this program by running ldd <program>. Then, look for /opt/atX.X/lib64/ld64.so.2. Ensure to install the latest update of the major version you identified.

 

Can I set RPATH/RUNPATH with the Advance Toolchain?

 

Yes. The Advance Toolchain uses the RPATH and RUNPATH tags to identify where its libraries are located, but it is possible to add new directories using the linker paramenter -rpath=dir.

By default, the Advance Toolchain will prepend its own lib directory to the RPATH/RUNPATH. However, in case the user changes this behavior, he or she must ensure the Advance Toolchain lib directory will always appear before a directory with another libc, i.e. /usr/lib, /usr/lib64, /lib or /lib64. This will guarantee you won't mix parts of different glibc versions.

 

Can I mix parts of different glibc versions?

 

No. The GNU C Library has not been designed to support mixing parts of different glibc versions, e.g.: use the loader provided by the Linux distribution and glibc libraries provided by the Advance Toolchain.

When these libraries are mixed up, the program usually aborts complaining about missing symbols in one of the libraries provided by glibc.

To avoid these issues, the user has to:

  • Ensure the Advance Toolchain lib directories appear before a directory with another libc in the library search path, i.e. /usr/lib, /usr/lib64, /lib or /lib64. Which includes LD_LIBRARY_PATH, RPATH/RUNPATH and /opt/atX.X/etc/ld.so.conf.
  • Force to use the Advance Toolchain loader when forcing the preload of an Advance Toolchain library with a program compiled using the system's toolchain, e.g.: LD_PRELOAD=/opt/atX.X/lib64/libtcmalloc.so /opt/atX.X/lib64/ld-X.XX.so /usr/bin/printenv PATH

 

How to verify if the Advance Toolchain is corrupted?

 

The package manager provides .

On RPM-based systems (RHEL, SLES, Fedora):
Run rpm --verify <list of packages>.

Replace <list of packages> by the list of Advance Toolchain packages installed in the system, for example: rpm --verify advance-toolchain-atX.X-runtime advance-toolchain-atX.X-devel advance-toolchain-atX.X-perf advance-toolchain-atX.X-mcore-libs

If this command creates any output, some of the packages are corrupted. If it returns 0, the installation is fine.

On DEB-based systems (Ubuntu):
Run debsums <list of packages>.

Replace <list of packages> by the list of Advance Toolchain packages installed in the system, for example: debsums advance-toolchain-atX.X-runtime advance-toolchain-atX.X-devel advance-toolchain-atX.X-perf advance-toolchain-atX.X-mcore-libs

This command is going to print the status for each file. If it returns 2, there is at least one corrupted file. If it returns 0, the installation is fine.

In case the previous commands report any issue, uninstall and install the corrupted package again.

 

Support

 

Customer support for the Advance Toolchain is provided in one of three ways:

  • If you are using the Advance Toolchain as directed by an IBM product team (i.e. IBM XL Compiler or PowerVM Lx86), please report the suspected issues to the IBM Support using that product name and entitlement.
  • IBM's Support Line for Linux Offerings in the United States now provides support for the Advance Toolchain as well. If you have a contract with an United States Support Line for Linux, place a call to IBM Support:
    1. Dial 1-800-426-IBM-SERV.
    2. Option #2 (Other business products or solutions).
    3. Option #2 (Software).
    4. Option #7 (Other OS/Linux).
  • All other users can use an electronic forum that is monitored Monday through Friday. For questions regarding the use of the Advance Toolchain or to report a suspected defect, click here.
    1. Open the Advance Toolchain topic.
    2. Select "Post a New Reply".
    3. Enter and submit your question or problem.
    4. An initial response will be attempted within 2 business days.

The Advance Toolchain is supported on RHEL5, RHEL6, RHEL7, SLES10 and SLES11 (and their service packs/update releases). Runtime compatibility support only is also available for RHEL 5 and SLES 10 (Advance Toolchain 6.0 and 7.0) and RHEL 4 and SLES 9 (Advance Toolchain 5.0).