Building Boost libraries using the xlcpp toolset for IBM compilers on Linux

Using Boost Build b2/bjam with IBM XL compiler for Linux on little endian systems

This article introduces the procedures to build Boost libraries with Boost build engine, b2/bjam, using the new xlcpp toolset of IBM XL compilers for Linux on little endian systems. The target audience is software engineers interested in increasing productivity and quality of C++ applications by using the expertly designed Boost libraries. This is important for the development of high-quality applications, where the project requirements exceed the extent of features provided by the C++ standard libraries.

Share:

Anh Tuyen Tran (anhtuyen@ca.ibm.com), Software Engineer, IBM

Anh Tuyen Tran's photoAnh Tuyen Tran has a bachelor’s degree in Software Engineering, Computer Science from the University of Toronto, Canada, and a master’s degree in Financial Studies from Yamaguchi University, Japan. Since 2007, he has worked for IBM as a software developer on the debugger team and now on the compiler team. Anh was the team lead for the Test Servers Administration team in the compiler area from 2009 to 2013. He is one of the major contributors to the documentation of building Boost libraries with the new xlcpp toolset, inline assembly for Linux on z Systems, hardware transactional memory, and cryptography built-in functions for IBM POWER8 architecture. Anh has written several articles about the IBM compilers.



22 April 2016

Also available in Chinese

Introduction

Boost is a collection of C++ libraries, which are developed, reviewed, and published by the Boost community under the Boost Software License. Libraries in Boost use the advanced features and the latest innovations in C++ programming to enhance the functionality of the programming language. Over the years, many Boost libraries have been adapted to be included in the C++ Standard Library.

A Boost package consists of the code base to build the Boost libraries, the source code to bootstrap the build engine, and the JAM configuration files to drive the build process. Although some users still build Boost libraries in the same manner as building regular C++ applications using the make utility, a part of the industry now builds and installs the libraries with the Boost build engine, b2. Because all the JAM configuration files required for the build process are released together with each Boost version, building Boost libraries with b2 is a straightforward process.


Boost build tool b2/bjam

b2 is the official name of the build engine that was developed from an earlier build tool called bjam. In newer releases of Boost, b2 and bjam are, in fact, identical. This can be verified by running a checksum function on the two files. For the ease of discussion, this article addresses the Boost build tool as b2.

Figure 1: Checksum to verify that b2 and bjam are identical

b2 supports a wide range of compilers on a large number of platforms. Each compiler is represented as a toolset to the build engine. Instead of the traditional toolset, vacpp, specified for IBM XL compilers, Boost 1.59.0 introduces a new toolset, titled xlcpp, for newer IBM compilers. At the time of this writing, xlcpp toolset is supported by IBM XL compilers for Linux on little endian systems.

Table 1 displays the relationship between Boost versions, compiler versions, and the corresponding toolsets for Linux on little endian systems. For support of Boost xlcpp toolset by IBM XL compilers on other platforms, refer to the IBM C and C++ Compilers family website.

Table 1: Toolsets supported by IBM XL compilers for Linux on little endian systems

Compiler version for Linux (little endian) Boost version Toolset
XL C/C++ V13.1.2 1.55.0 vacpp
XL C/C++ V13.1.3 1.59.0 xlcpp

Details about the Boost.Jam language, the Boost build engine, the build configuration, and the installation are readily available on www.boost.org website. This article focuses on the specific configuration needed to build and install Boost libraries using the new xlcpp toolset for IBM XL compilers. The discussion uses version 1.59.0, the first version of Boost released with the xlcpp toolset. The contents, however, might well be applicable for the later releases of Boost.


Build process

The process of building Boost libraries, in general, consists of the following three steps:

  1. Preparing the source codes
  2. Bootstrapping b2, the Boost build engine
  3. Using b2 to build and install the Boost libraries

After being built and installed at the specified locations, Boost libraries and Boost header files are ready for software engineers to use in their applications.

Prepare the source code

You can download the Boost source code from the www.boost.org website. You need to decompress the downloaded package before it can be used. Listing 1 shows an example of how to acquire and decompress the source code for Boost version 1.59.0.

Listing 1: Obtain and decompress Boost code base

$ wget http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz 
$ gunzip < boost_1_59_0.tar.gz | tar –xvf-

Instead of using gunzip and tar separately as shown in Listing 1, users can use the z flag along with the tar utility to decompress the .gz file using gunzip. It is, nevertheless, important to include f as the last flag in the command. The f flag specifies the path and the name of the compressed file.

The decompressing process creates a directory called boost_1_59_0. This directory is regarded to as BOOST_ROOT during discussion about the build and the installation processes.

Patching is required for building Boost libraries with the xlcpp toolset. Users must obtain and apply the corresponding patch file to the decompressed source codes of Boost before proceeding further. During the patching process, the Boost header files and the JAM configuration files will be modified properly to ensure the successful creation of the Boost libraries. Refer to Boost library regression test summaries page of IBM XL compilers for a list of available patch files. The patch files can be obtained with an FTP client or system tools such as wget. Users must avoid copying the contents of the patch file rendered on a web browser because it might result in missing certain parts of the file. Listing 2 shows an example of how to acquire and apply the patch file for the source code of Boost 1.59.0.

Listing 2: Obtain and patch boost source codes

$ wget ftp://public.dhe.ibm.com/software/xlcpp/boost/bst1590/boost_modfile_le_1313.txt 
$ cd <the-directory-where-boost_1_59_0-resides>  
$ patch –p0 < <path-to-the-pathfile>/boost_modfile_le_1313.txt

Note: Some users will not be able to apply the patch file to the Boost source code due to certain constraints, such as the existence of the legacy codes. This will be addressed in a companion article that follows this one.

Although the compressed package is relatively small in size, decompressing, building, installing, and, especially, testing the built product of the entire collection of Boost libraries require multiple GB of disk space. You must be prepared to have ample disk space if you intend to test Boost libraries.

Bootstrap to create the Boost build engine

Boost provides a script named bootstrap.sh to build b2, the boost build engine needed for building, installing, and testing Boost libraries. You can find the bootstrap.sh script within the BOOST_ROOT directory, boost_1_59_0.

To bootstrap b2 with the xlcpp toolset, the user must make sure that the path to the IBM xlC compiler is included in the $PATH environment variable. You can verify this using the which command as shown in Listing 3.

Listing 3: Preparing the $PATH environment variable for bootstrapping

$ which xlC                                           #xlC is not on $PATH
 which: 0652-141 There is no xlC in  …	
$ PATH=/full-path-to-xlC:${PATH}                           #Add path of xlC to $PATH 
export PATH	
$ which xlC                                           #xlC is on the $PATH
/full-path-to-xlCxlC

After preparing the $PATH variable to enable the build engine to find the xlC compiler, the bootstrap.sh script will be called from within the BOOST_ROOT directory with the option, --with-toolset=xlcpp.

Listing 4: Calling bootstrap.sh to build b2/bjam with xlcpp toolset

$ cd boost_1_59_0 
$ ./bootstrap.sh --with-toolset=xlcpp 
Building Boost.Build engine with toolset xlcpp... 
… 
Generating Boost.Build configuration in project-config.jam... 
Bootstrapping is done. To build, run: 
    ./b2 
To adjust configuration, edit 'project-config.jam'. 
…

If the bootstrap.sh script runs successfully, two executable files named b2 and bjam will be created within the directory boost_1_59_0. As indicated in Figure 1, b2 and bjam are in fact two identical files.

If the bootstrapping fails, examining the log file bootstrap.log, stored in the directory boost_1_59_0, can reveal the causes of the failure. Note that even if the information in the bootstrap.log describes problems in the current working directory ".", in some places, the directory "." actually points to boost_1_59_0/tools/build/src/engine. This is because during the build process, the script bootstrap.sh issues a cd command to navigate to the boost_1_59_0/tools/build/src/engine directory.

If no toolset is specified, the bootstrap.sh script will attempt to use some other compiler available on the system. All activities carried out by the script will be recorded in the log file, bootstrap.log, in the BOOST_ROOT directory, boost_1_59_0.

In the unlikely case that the bootstrap.sh script fails to create the executable files b2 and bjam with xlcpp toolset, user can try to call the script with a different compiler available on the system. The build engine b2/bjam created by the other compiler might be used as an alternative to continue the processes of building, installing, and testing Boost with the xlcpp toolset in the next steps.

Build and install boost with xlcpp toolset

After being created using bootstrapping, b2/bjam can be used to build, install, and test the Boost libraries. When b2/bjam is called without any command-line option, the default configuration will be used to build Boost. For the majority of software projects, a build with a default configuration might not be sufficient. The next section in this article discusses the steps required to perform a complete Boost build with the command-line options of b2 using the xlcpp toolset. You can find the documentation about all build options at the www.boost.org website; this article focusses only to the major ones used for xlcpp toolset.

Set up a directory to start the Boost build
For the ease of discussion, let's assume the build process will be done from within a working directory called mybuild. The path to the boost source code, also called BOOST_ROOT, is <path>/boost_1_59_0. The setup of the working directory can be done as shown in Listing 5.

Listing 5: Setting up the environment for the Boost build

$ mkdir mybuild                                         #Create working directory 
$ cd mybuild 
$ BOOST_ROOT=<path>/boost_1_59_0                         #Set $BOOST_ROOT 
$ export BOOST_ROOT 

$ mkdir boost_build                                     #Create build directory 
$ BOOST_BUILD=<path>/boost_build                         #Set $BOOST_BUILD 
$ export BOOST_BUILD

You need to properly set the permission for the created directories so that the build processes can access them.

Specifying the BOOST_ROOT path is critical to the build process; it informs the build engine the location to find the source codes on the system. In addition to specifying BOOST_ROOT as an environment variable as shown in Listing 5, user can also provide the value directly as a command-line option when invoking b2 using the –s option.

Listing 6: Specify BOOST_ROOT on the command line

<path>/b2 -s BOOST_ROOT=<path>/boost_1_59_0 …

Specify the compiler to be used for the Boost build process
Without modification to the xlcpp.jam and the user_config.jam files, b2 will use the xlC compiler on the $PATH to build the Boost libraries with toolset xlcpp. You can find details about setting the xlC compiler on $PATH in Listing 3.

User can modify the configuration files to gain the flexibility to specify the compiler as a parameter to the build. The details will be discussed in the next article.

The Boost build engine, b2, accepts three kinds of command-line tokens: options, properties, and targets. The command-line tokens can be specified in any order and are distinguished as follows.

  • Options, which stipulate the behavior of the build engine b2, start with either one or two dashes such as –a, --build-dir.
  • Properties, which specify details of the libraries to be built, are tokens with an equal sign of the form feature=value such as link=static.
  • Targets, which are all tokens that are neither options nor properties. Target tokens inform the build engine about the targets to be built.

Major command-line options
Table 2 lists some of the most important options frequently used for building Boost libraries. For an exhaustive list of supported options, refer to Boost.Build User Manual published by www.boost.org.

Table 2: Important options

OptionDescription
-a Rebuilds everything, even if the target already exists
-n Does not run the commands, instead, prints them to the standard output
-d0 Enables debugging level zero (that is, no informational message will be displayed)
-dN Enables cumulative debugging levels from 1 to N. –d1 is the default value, which displays the information about the action taken to build each target when they are run
-d+N Enables only debugging level N
-jN Runs up to N commands in parallel
-s var=value Overrides the value of variable var, which was imported from the environment, with the specified value
--debug-configuration Displays debug information about the loading of Boost.Build and toolset files
--clean Cleans all targets in the current directory and any subprojects
--help Displays the help information

Common properties
The most common properties, which users might frequently specify on the command line of the form of feature=value, are listed in Table 3. For a complete list of acceptable properties, refer to the Builtin features section published by www.boost.org.

Table 3: Common properties in the form of feature=value

FeatureAcceptable valuesDescription
variantdebug, release Determines whether the build will be a debug build or a release one
linkshared, static Determines if the created libraries will be shared objects or static ones
threadingsingle , multiple Causes the produced binaries to be thread-safe if multiple is specified. This requires proper support in the source code.
cxxflags(arbitrary string) Specifies the set of options to be passed to the C++ compiler
cflags(arbitrary string) Specifies the set of options to be passed to the C compiler
linkflags(arbitrary string) Specifies the set of options to be passed to the linker

For certain features, such as link and threading, multiple values can be specified in the command line for each of them. In this case, the target will be built several times, one for each specified value of a feature. For example, if the command-line properties are link=static link=shared threading=single threading=multi, or equivalently link=static,shared threading=single,multi, a total of four builds will be carried out.

Table 4: Combined properties with multiple specified values

Combined propertiesBuildLinkThreading



link=static,shared threading=single,multi
1staticsingle
2staticmultiple
3sharedsingle
4sharedmultiple

Build and install Boost
After preparing the Boost source code and setting up the environment, users can start to build and install the Boost libraries from the mybuild directory. Refer to Listing 7 for the commands.

Listing 7: Build and install Boost libraries

$ mkdir mybuild 
$ cd mybuild 
$ BOOST_ROOT=<path>/boost_1_59_0 
$ export BOOST_ROOT 
$ mkdir <path>/boost_build 
$ BOOST_INSTALL=<path>/boost_install 
$ export BOOST_ INSTALL 
$ BOOST_BUILD=<path>/boost_build 
$ export BOOST_BUILD 

$ cd $BOOST_ROOT 
$ ./b2 install --prefix=$BOOST_ INSTALL --build-dir=$BOOST_BUILD -j8 
-l1200 toolset=xlcpp variant=release target-os=linux -s OS=LINUX -d2 -a

The command, b2 install performs a configuration check and then builds and installs all the libraries based on the results of the checking.

Architecture-independent files (for example, header files) will be installed in the subdirectory specified by --includedir within the location dictated by the value of --prefix. If the options are not provided, the default values will be used.

Table 5: Options for installing architecture-independent files

OptionDefault valueDescription
--prefix=<PREFIX>/usr/local Specifies the location to install the architecture-independent files
--includedir=<HDRDIR><PREFIX>/include Specifies the location to install the header files

In the example shown in Listing 7, value of --prefix is the explicitly specified location <path>/boost_install. Also, because --includedir is absent on the command line, its default value <PREFIX>/include is assumed. Thus, the header files of Boost in this case will be copied to <path>/boost_install/include.

Figure 2: Header files installed in <path>/boost_install/include

The compiled libraries will be created based on the corresponding JAM configuration files. They will then be copied to the subdirectory specified by --libdir within the location stipulated by the value of --exec-prefix. In the absence of the options, the default values will be used.

Table 6: Options for installing architecture-dependent files

OptionDefault valueDescription
--exec-prefix=<EPREFIX><PREFIX> Specifies the location to install architecture dependent files
--libdir=<DIR><EPREFIX>/lib Specifies the location to install library files

In the example shown in Listing 7, value of --exec-prefix is the default value <PREFIX>, that is, <path>/boost_install because it is not explicitly specified. Likewise, as --libdir is also absent on the command line, its default value <EPREFIX>/lib is assumed. Accordingly, the libraries will be installed in <path>/boost_install/lib.

Figure 3: Libraries installed in <path>/boost_install/lib

Click to see larger image

The --build-dir=$BOOST_BUILD property stipulates that the build must be carried out within the $BOOST_BUILD directory instead of within the distribution tree. This is essential when the distribution tree is in a read-only location.

The –j8 option specifies that up to eight commands can be run in parallel. The –l1200 option dictates that the Boost build engine timeout in 1200 seconds. The –d2 option enables debugging level 2, which displays all action text to the standard output as they are run. The toolset=xlcpp property informs the build engine that the IBM XL compiler will be used for compilations. The target-osproperty tells the build engine to create libraries for Linux operation systems.

Because the vast majority of Boost libraries are header files only, the execution to build and install Boost libraries is relatively fast when no optimization is involved. At higher level of optimizations such as –O2 and –qipa (interprocedural analysis optimization), the compilation of Boost libraries might take significantly more time.

In order to pass extra options to the compiler, you can use cxxflags, cflags, or linkflags. For example, passing the -qnoxlcompatmacros option to the compiler can be achieved as shown in Listing 8. The next article about further customization of Boost configuration will discuss the other ways to pass the extra options to the compiler.

Listing 8: Passing extra options to the compiler

$ ./b2 install cxxflags="-qnoxlcompatmacros" cflags="-qnoxlcompatmacros" 
--prefix=$BOOST_ INSTALL --build-dir=$BOOST_BUILD -j8 -l1200 toolset=xlcpp 
variant=release target-os=linux -s OS=LINUX -d2 -a

Display the names of the libraries to be built in a Boost release
The number of libraries in a Boost release might vary from version to version. To display the exact names of all libraries to be built within a certain Boost release, user can invoke b2 with the --show-libraries option.

Listing 9: Displaying the libraries to be built

$ BOOST_ROOT/b2 --show-libraries 
The following libraries require building: 
    - atomic 
    - chrono 
    - container 
    - context 
< … >

The exact names of libraries acquired with this option are to be used together with options, such as --with-<library> and --without-<library>. These are explained in the next section.

Build and install only specific libraries
When being invoked with only the install command, the Boost build engine b2 will build and install all libraries included in the project. To dictate that only certain libraries should be built, the user can use the --with-<library> command line option. When called with this option, Boost build engine b2 will still install all header files, but will only build libraries in the following two groups:

  • Explicit libraries (which are libraries explicitly specified with an option)
  • Implicit libraries (which are libraries needed for building the explicit libraries)

For example, the following invocation stipulates that only the chrono library will be built. Although the system library is not specified, it is also built because it is needed for the explicitly specified library, chrono.

Listing 10: Building only the chrono library

$BOOST_ROOT/bjam install --with-chrono -j4 -l1200 toolset=xlcpp 
variant=release target-os=linux -s OS=LINUX -d2 -a

Figure 4 shows that both the explicit library, chrono, and the implicit one, system, are built.

Figure 4: Both explicit and implicit libraries are built

Excluding specific libraries
When certain libraries are not needed, the user can inform b2 of the fact by using the option, --without-<library>. Note that, however, if the specified library is needed by at least one other library, it will still be built. For example, because the system library is required by the chrono library, even if the system library is explicitly mentioned to not be built, it will still be created as an implicit library for the chrono library.

Listing 11: Exclude only the system library

$BOOST_ROOT/bjam install --without-system -j4 -l1200 toolset=xlcpp 
variant=release target-os=linux -s OS=LINUX -d2 -a

Conclusion

Starting with version 1.59.0, Boost supports the xlcpp toolset for IBM XL compilers. Building Boost libraries with b2/bjam using the xlcpp toolset is straightforward, especially when the project allows the use of a patching file to modify the Boost headers and the JAM configuration files. Under circumstances where using the patch file is not desirable, or when a patch file is not readily available, more works have to be done before the Boost libraries can be built.

This article introduced the basic procedures to build and install Boost libraries from the Boost source code after the Boost headers and the JAM configuration files are properly patched. A follow-up article further describes the customization to the Boost configuration to facilitate the building of Boost libraries even when no patch file is allowed.


Acknowledgment

I would like to thank Ms. Nha-Vy Tran for her advice and suggestions while creating this article.


Resources


References

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Linux on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux
ArticleID=1030281
ArticleTitle=Building Boost libraries using the xlcpp toolset for IBM compilers on Linux
publish-date=04222016