Contents


How to build IBM PowerLinux applications on Intel x86 servers

Using IBM Advance Toolchain for PowerLinux cross compiler

Comments

Porting Linux® applications running on x86 servers to another platform such as PowerLinux requires setting up a separate build environment to compile the application, right? Well, not necessarily. One can use a cross compiler on an x86 server and build application binaries that target a different platform architecture, such as IBM Power®. Cross compilers have been around for a long time. They are mostly used for building applications for embedded systems that have limited resources to run an operating system capable of supporting a native compiler. The first time I was introduced to cross compilers was when I worked on a project for a hand-held device. I wrote and compiled my source code on an x86 workstation and loaded the binaries into the device through a serial communications connection. More recently I've been using a cross compiler to build simple applications to control servo motors for a robot I've been working on. But enough of my personal history and let's get down to business.

This article describes the cross compiler included in the IBM Advance Toolchain for PowerLinux. From hereon, I will refer to the IBM Advance Toolchain for PowerLinux as "the toolchain". The toolchain is a set of open source development tools and runtime libraries which allows users to take advantage of IBM's latest Power hardware features on Linux. The toolchain itself is self-contained and does not rely on the system toolchain. It is installed in /opt so as not to override the default Linux distribution toolchain. As of this writing, the latest release includes current stable versions of the following packages:

  • GNU Compiler Collection (gcc, g++ and gfortran), plus individually optimized gcc runtime libraries for supported IBM POWER® processors
  • GNU C library (glibc), individually optimized for supported POWER processors
  • GNU binary utilities (binutils)
  • Decimal floating point library (libdfp), optimized with hardware DFP support for IBM POWER7® and IBM POWER8™ processor-based servers
  • AUXV library (libauxv)
  • GNU debugger (gdb)
  • Performance analysis tools (oprofile, valgrind, itrace)
  • Muticore exploitation libraries (Intel® TBB, Userspace RCU), starting in version 5.0-3
  • And, several support libraries (such as libhugetlbfs, zlib, and so on)

For information about downloading the toolchain, refer to the resources section at the end of this article.

For the purpose of this article, the focus will mainly be on the toolchain cross compiler. This article explains how to build binary executables for PowerLinux on an x86 server using the toolchain cross compiler. It can also answer the following questions.

  • Will cross-compiled applications built on an x86 platform perform as good as binaries built natively on PowerLinux?
  • Will cross-compiled applications and libraries built on an x86 platform have the same application binary functionality as applications built on PowerLinux?
  • Will the cross-compiled applications built on an x86 platform produce binaries that can be debugged natively on PowerLinux?

To answer these questions, I used two popular open source applications, PostgreSQL and Apache httpd server. Both the PostgreSQL and Apache httpd source distribution were built using the toolchain cross compiler on an x86 server and natively on a PowerLinux server. The applications were then installed and ran on a PowerLinux server in different install directories. I ran a couple of benchmark tools against these applications and compared their performance and functionality with each other.

The results of the tests are described in the next sections. For readers interested in re-creating the tests, information about the environment configuration and how the applications were built are described in the sections titled Environment and configuration and Building the applications using the toolchain respectively.

Results

To test the applications, I used a few publicly available benchmark tools. For PostgreSQL, I used the pgbench-tool available from GitHub. It uses the pgbench utility that comes with the PostgreSQL source distribution. For Apache httpd, I used the apache bench (ab) tool that came with the Apache httpd source distribution. Please see the Resources section at the end of this paper for information on where to download pgbench-tools.

PostgreSQL test setup and results

After building the PostgreSQL application with the cross compiler on an x86 server, the complete set of binaries were transferred onto a PowerLinux server. The following modifications were done to the default configuration files:

Postgresql.conf
max_connections = 4096
shared_buffers = 2048MB
synchronous_commit = off
checkpoint_segments = 3
checkpoint_timeout = 5min

Modify the kernel.sem value of the PowerLinux system settings.

sysctl.conf
kernel.sem = 250     32000   32   12288

Modify the config file of pgbench-tools.

MAX_WORKERS="32".

As documented in the pgbench-tools website, the PostgreSQL tests were ran according to the following steps:

  1. Create the pgbench and results database.
  2. Initialize the results database.
  3. Create a test set with description.
  4. Run the test using the runset script.
  5. Compile and post the test results.

For specific information on commands used to perform the steps, refer to the pgbench-tools README.

Postgresql results (cross-compiled)

Here are the results of running pgbench-tools against the cross-compiled binaries on the Power server.

Figure 1. Scaling factor for cross-compiled PostgreSQL binaries
Figure 1. Scaling factor for cross-compiled PostgreSQL binaries
Figure 1. Scaling factor for cross-compiled PostgreSQL binaries

The scaling factor shows a plot of the number of transactions per second as the database size grows. The green line represents that database size in megabytes (MB) and the red line represents the number of transactions per second. The graph shows that as the database size increases, the number of transactions per second decreases.

Figure 2. Scalability diagram of cross-compiled PostgreSQL binaries
Figure 2. Scalability diagram of cross-compiled PostgreSQL binaries
Figure 2. Scalability diagram of cross-compiled PostgreSQL binaries

Figure 2 shows the scalability diagram. It shows that as the number of clients increases, the number of transactions per second increases as well.

PostgreSQL results (natively compiled)

When testing the natively compiled binaries, the same exact configuration settings and the database location (/tmp/usr/local/pgsql/data) as the cross-compiled PostgreSQL were used. To make sure that the databases are clean, drop both the results and pgbench databases before starting the tests.

Postgresql# drop database results;
DROP DATABASE

postgresql# drop database pgbench;
DROP DATABASE

Both the scaling factor and the scalability diagrams are shown in Figure 3 and Figure 4. As mentioned earlier, the scaling factor shows a plot of the number of transactions per second as the database size grows. The green line represents the database size in MB and the red line represents the number of transactions per second.

Figure 3. Scaling factor for natively built PostgreSQL binaries
Figure 3. Scaling factor for natively built PostgreSQL binaries

The scalability diagram in Figure 4 shows the increase in the number of transactions per second as the number of clients increase.

Figure 4. Scalability diagram of natively built PostgreSQL binaries
Scalability diagram of natively built PostgreSQL binaries
Scalability diagram of natively built PostgreSQL binaries

Notice that the graphs for the cross-compiled binaries are very much the same as the graphs of the natively compiled binaries. This simple test shows that cross compiling on x86 servers can yield binaries that can have the same performance characteristics as a natively compiled code.

Apache httpd test setup and results

To test Apache httpd, I used the benchmark tool called ab that came with the Apache httpd source distribution. Similar to the PostgreSQL, the Apache httpd was built using the toolchain cross compiler and the binaries transferred to a PowerLinux server. I modified the default httpd.conf file to change the htdocs directory to point to the local documents directory. Everything else was left at the default values.

The command line and arguments I used to run the ab tool was as follows:

# ./ab -n 500000 -c 100 -g out1.data http://9.3.4.241/

I ran the command three times writing the output to out1.data, out2.data, and out3.data and plotted the files using the gnuplot utility with the parameters in the benchmark.tpl file as shown below.

# gnuplot benchmark.tpl
benchmark.tpl # output as png image set terminal png
# save file to "benchmark.png" set output "benchmark.png"
# graph title set title "Benchmark for 9.3.4.241"
# aspect ratio for image size set size 1,1
# enable grid on y-axis set grid y
# x-axis label set xlabel "Request"
# y-axis label set ylabel "Response Time (ms)"
# plot data using column 10 with smooth sbezier lines plot "out1.data" using 10 smooth sbezier with lines title "Benchmark 1:", \ "out2.data" using 10 smooth sbezier with lines title "Benchmark 2:", \ "out3.data" using 10 smooth sbezier with lines title "Benchmark 3:"

For more information about the Apache benchmarking tool, refer to the Resources section at the end of this article.

Apache httpd results (natively compiled and cross compiled)

The gnuplot utility produces a file called benchmark.png for each of the tests. The images can be viewed by any image viewer. Both images are show in Figure 5 and Figure 6.

Figure 5 shows the graph of the natively compiled httpd.

Figure 5. Graph of response time of natively compiled httpd
Figure 5. Graph of response time of natively compiled httpd

Figure 6 shows the graph of the cross-compiled httpd.

Figure 6. Graph of response time of cross-compiled httpd
Figure 6. Graph of response time of cross-compiled httpd

Notice that both of these graphs show very similar performance characteristics. I encourage readers to do more tests with varying web document and data.

Functional and debugging findings

Besides performance, developers will have other questions regarding the use of the toolchain cross compiler. This section answers questions relating to functionality and the ability to debug cross-compiled binaries. The questions are:

  • Are cross-compiler built libraries functionally compatible with natively built executables?
  • Can cross-compiled binaries be debugged on a PowerLinux server?

The two simple use cases that follow can help answer these questions.

  • Use case 1: Replace one or more shared libraries of a natively built httpd with a shared library that is built using the toolchain cross compiler. See if the natively built httpd still functions correctly.
  • Use case 2: On a PowerLinux server, attach a debugger to a running process. The process is a cross-compiled binary executable that is built from an x86 server. Set a breakpoint, set the source directory, and do a listing of a function.

Use case 1: Functional testing

Here are the instructions that I followed for use case 1:

  1. Install the natively built httpd in /home/usr/local.
  2. Run the ldd utility against the /home/use/local/bin/httpd binary to show its shared library dependencies. Note that both libaprutil-1.so.0 and libapr-1.so.0 are located in /home/usr/local/lib. These libraries are part of the natively compiled libraries required by httpd.
[root@localhost lib]# ldd ../../bin/httpd
    linux-vdso64.so.1 => (0x00000fffa1140000)
    libaprutil-1.so.0 => /home/usr/local/apr/lib/libaprutil-1.so.0 (0x00000fffa10c0000)
    libapr-1.so.0 => /home/usr/local/apr/lib/libapr-1.so.0 (0x00000fffa1060000)
    librt.so.1 => /opt/at7.0-5-rc1/lib64/power7/librt.so.1 (0x00000fffa1040000)
    libcrypt.so.1 => /opt/at7.0-5-rc1/lib64/power7/libcrypt.so.1 (0x00000fffa0ff0000)
    libpthread.so.0 => /opt/at7.0-5-rc1/lib64/power7/libpthread.so.0 (0x00000fffa0fb0000)
    libdl.so.2 => /opt/at7.0-5-rc1/lib64/power7/libdl.so.2 (0x00000fffa0f90000)
    libc.so.6 => /opt/at7.0-5-rc1/lib64/power7/libc.so.6 (0x00000fffa0da0000)
    /opt/at7.0-5-rc1/lib64/ld64.so.1 => /lib64/ld64.so.1 (0x00000000267a0000)
  1. Remove the shared libraries libaprutil-1.so.0 and libapr-1.so.0.
  2. Copy the cross-compiled libraries, libaprutil-1.so.0 and libapr-1.so.0, in /tmp/usr/local/lib.
  3. Include /tmp/usr/local/lib in the LD_LIBRARY_PATH.
# cd /home/usr/local/apr/lib 
# rm libaprutil-1.so.0 
# rm libapr-1.so.0 
# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/tmp/usr/local/lib
  1. Run the ldd utility again and see that the httpd is now using the libraries in /tmp/usr/local/lib.
[root@localhost lib]# ldd ../../bin/httpd
    linux-vdso64.so.1 => (0x00000fff951a0000)
    libaprutil-1.so.0 => /tmp/usr/local/apr/lib/libaprutil-1.so.0 (0x00000fff95150000)
    libapr-1.so.0 => /tmp/usr/local/apr/lib/libapr-1.so.0 (0x00000fff950f0000)
    librt.so.1 => /opt/at7.0-5-rc1/lib64/power7/librt.so.1 (0x00000fff950d0000)
    libcrypt.so.1 => /opt/at7.0-5-rc1/lib64/power7/libcrypt.so.1 (0x00000fff95080000)
    libpthread.so.0 =>/opt/at7.0-5-rc1/lib64/power7/libpthread.so.0 (0x00000fff95040000)
    libdl.so.2 => /opt/at7.0-5-rc1/lib64/power7/libdl.so.2 (0x00000fff95020000)
    libc.so.6 => /opt/at7.0-5-rc1/lib64/power7/libc.so.6 (0x00000fff94e30000)
    libexpat.so.0 => /tmp/usr/local/apr/lib/libexpat.so.0 (0x00000fff94df0000)
    /opt/at7.0-5-rc1/lib64/ld64.so.1 => /lib64/ld64.so.1 (0x0000000022730000)
  1. Start the application to see if it runs and functions as a normal application.
# cd /home/usr/local/bin 
# ./apachectl start 
# ps –ef | grep http
root 11022 1 0 15:41 ? 00:00:00 /home/usr/local/bin/httpd -k start
daemon 11023 11022 0 15:41 ? 00:00:00 /home/usr/local/bin/httpd -k start
daemon 11024 11022 0 15:41 ? 00:00:00 /home/usr/local/bin/httpd -k start
daemon 11025 11022 0 15:41 ? 00:00:00 /home/usr/local/bin/httpd -k start
daemon 11111 11022 0 15:41 ? 00:00:00 /home/usr/local/bin/httpd -k start
  1. Point the browser to the PowerLinux server running the httpd process and see if the following message is displayed on the browser:

It works!

In this scenario, we mixed a natively compiled binary with cross-compiled libraries. The cross-compiled libraries were built with the ppc64 architecture option. From this use case, we proved that we can mix cross-compiled libraries with natively compiled executables on a PowerLinux server.

Use case 2: Debugging

In the second use case, we try to prove that cross-compiled executables can be debugged on a PowerLinux server. Here are the instructions for use case 2:

  1. Using gdb, attach to the httpd process. Make sure that the httpd version being used is the one produced by the toolchain cross compiler built on an x86 platform.
[root@localhost ~]# gdb attach 25412 
GNU gdb (GDB) 7.6.50.20130722-cvs
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "powerpc64-linux". 
…
(gdb)
  1. Set the directory to the source distribution.
(gdb) set directories /home/ProjectToolchainPLCompile/apr-1.5.1
(gdb) where #0 0x00000fff7cb676f8 in ___newselect_nocancel () at ../sysdeps/unix/syscall-template.S:81 #1 0x00000fff7cd38274 in apr_sleep (t=<optimized out>) at time/unix/time.c:246 #2 0x0000000010034220 in ap_wait_or_timeout (status=0xfffeaa1e704, exitcode=0xfffeaa1e700, ret=0xfffeaa1e6e0, p=0x1003d0c6138, s=<optimized out>) at mpm_common.c:195 #3 0x0000000010079238 in server_main_loop (remaining_children_to_start=<optimized out>) at worker.c:1654 #4 worker_run (_pconf=<optimized out>, plog=<optimized out>, s=<optimized out>) at worker.c:1823 #5 0x0000000010033430 in ap_run_mpm (pconf=0x1003d0c6138, plog=0x1003d0f3378, s=0x1003d0ef4c0) at mpm_common.c:96 #6 0x000000001002a5d8 in main (argc=3, argv=0xfffeaa1ee28) at main.c:777
  1. Move up a function to apr_sleep.
(gdb) up 
#1 0x00000fff7cd38274 in apr_sleep (t=<optimized out>) at time/unix/time.c:246
246        select(0, NULL, NULL, NULL, &tv);
  1. List the source code of apr_sleep.
(gdb) list
241       delay(t/1000); 
242    #else 
243       struct timeval tv; 
244       tv.tv_usec = t % APR_USEC_PER_SEC; 
245       tv.tv_sec = t / APR_USEC_PER_SEC; 
246       select(0, NULL, NULL, NULL, &tv); 
247    #endif 
248    } 
249 
250    #ifdef OS2
  1. Detach from the process.
(gdb) detach 
Detaching from program: /tmp/usr/local/bin/httpd, process 25412
(gdb) quit

Remote debugging

The toolchain also includes the gdbserver program. It allows the developer to connect to their application from a remote gdb session. Make sure that when you invoke the gdbserver program on the PowerLinux server, you are using the one from the toolchain. In my case, I make sure to check that it is in my path.

# type gdbserver
gdbserver is hashed (/opt/at7.0-5-rc1/bin/gdbserver)

Here is an example on how to use the gdbserver program on the PowerLinux server. We will attach to a running httpd process:

[root@stgisv241 bin]# gdbserver --attach 9.3.4.240:2345 22589 
Attached; pid = 22589
Listening on port 2345

On the x86 server, invoke the powerpc64-linux-gdb program against the httpd binary on the local server. Notice that the gdb version from the toolchain is named powerpc64-linux-gdb. The httpd binary used is the same cross-compiled binary version that is the same as the one running on the PowerLinux server.

# pwd 
/tmp/usr/local/bin 
[root@stgisv240 bin]# powerpc64-linux-gdb ./httpd 
GNU gdb (GDB) 7.6.50.20130722-cvs 
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. 
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details. 
This GDB was configured as "--host=i686-linux --target=powerpc64-linux".
Type "show configuration" for configuration details. 
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
.. 
Reading symbols from /tmp/usr/local/bin/httpd...done. 
(gdb)

Still on the x86 gdb session, invoke the target command within gdb as in the following example.

(gdb) target remote 9.3.4.241:2345
Remote debugging using 9.3.4.241:2345
warning: Could not load shared library symbols for 7 libraries,
    e.g. /opt/at7.0-5-rc1/lib64/power7/librt.so.1. 
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?
Reading symbols from /tmp/usr/local/apr/lib/libaprutil-1.so.0...done.
Loaded symbols for /tmp/usr/local/apr/lib/libaprutil-1.so.0
Reading symbols from /tmp/usr/local/apr/lib/libexpat.so.0...done.
Loaded symbols for /tmp/usr/local/apr/lib/libexpat.so.0
Reading symbols from /tmp/usr/local/apr/lib/libapr-1.so.0...done.
Loaded symbols for /tmp/usr/local/apr/lib/libapr-1.so.0 
….
Reading symbols from /tmp/usr/local/modules/mod_alias.so...done. 
Loaded symbols for /tmp/usr/local/modules/mod_alias.so
warning: Unable to find dynamic linker breakpoint function. 
GDB will be unable to debug shared library initializers 
and track explicitly loaded dynamic code.
0x00000fffa8fe76f8 in ?? ()
(gdb)

Notice some warning messages that relate to gdb not being able to find library symbols. The gdb command info sharedlibrary shows you the libraries that gdb failed to read symbols from. The example output that follows shows the libraries that gdb failed to read symbols from. Notice that the libraries are part of the system and unless they were compiled with symbols turned on, you do not have to worry about them.

(gdb) info sharedlibrary 
From               To                 Syms Read Shared Object Library 
                                      No        linux-vdso64.so.1 
0x00000fffa924ce80 0x00000fffa927e8a8 Yes       /tmp/usr/local/apr/lib/libaprutil-1.s
0x00000fffa91f7940 0x00000fffa922aa18 Yes       /tmp/usr/local/apr/lib/libexpat.so.0
0x00000fffa9192e60 0x00000fffa91ca0b8 Yes       /tmp/usr/local/apr/lib/libapr-1.so.0 
                                      No        /opt/at7.0-5-rc1/lib64/power7/librt.s
                                      No        /opt/at7.0-5-rc1/lib64/power7/libcrypt
                                      No        /opt/at7.0-5-rc1/lib64/power7/libdl.s
                                      No        /opt/at7.0-5-rc1/lib64/power7/libpthre 
                                      No        /opt/at7.0-5-rc1/lib64/power7/libc.so
                                      No        /opt/at7.0-5-rc1/lib64/ld64.so.1
                                      No        /opt/at7.0-5-rc1/lib64/power7/libnss_f
0x00000fffa8ea0c00 0x00000fffa8ea1878 Yes       /tmp/usr/local/modules/mod_authn_file
0x00000fffa8e810a0 0x00000fffa8e82108 Yes       /tmp/usr/local/modules/mod_authn_core.
0x00000fffa8e60da0 0x00000fffa8e61bb0 Yes       /tmp/usr/local/modules/mod_authz_host. 
0x00000fffa8e40f40 0x00000fffa8e41ed0 Yes       /tmp/usr/local/modules/mod_authz_group
0x00000fffa8e208e0 0x00000fffa8e20f18 Yes       /tmp/usr/local/modules/mod_authz_user.
0x00000fffa8e019e0 0x00000fffa8e046d8 Yes       /tmp/usr/local/modules/mod_authz_core.
...

As use case 2 shows, it is possible to locally or remotely debug a cross-compiled executable on a PowerLinux server as one would debug a natively compiled executable. Also, note that the binaries were compiled with the –g option to produce debugging information.

Environment and configuration

The testing environment is made up of an IBM Flex System® with several IBM Flex System x240 compute nodes and two IBM POWER7 processor-based nodes (one Flex System p260 and one Flex System p460). For the purpose of this project, I used one of the Flex system x240 compute nodes for cross compiling Apache httpd and PostgreSQL. After building the binaries, I moved them over to the Flex System p460 compute node. Figure 7 shows a diagram of the IBM Flex System test environment.

Figure 7. IBM Flex System test environment
IBM Flex System test environment
IBM Flex System test environment

The following are my system configurations, operating system levels, and other information pertaining to system setup.

Flex System x240 compute node

The Intel node used to cross compile the binaries only need enough processing power and memory to be able to build the Power binaries. The configuration is as follows:

  • Kernel-based virtual machine (KVM) hypervisor Red Hat 6.5
    • 32 processors
    • 64 GB memory
    • IBM Storwize® V7000 storage area network (SAN) connected disk
  • KVM guest
    • Red Hat Enterprise Linux 6.5
    • Four processors
    • 16 GB memory
    • One virtio disk : 100 GB
    • One virtual network interface card (NIC) configured as br0

All other KVM guests were turned off to eliminate any chance of disrupting the environment.

Flex System p440 compute node

The Flex System p440 compute node consists of several logical partitions (LPARs), one of which is configured as the test server. The PowerLinux server configuration is as follows:

  • IBM PowerVM® guest
    • Red Hat Enterprise Linux 6.5
    • Eight processors (dedicated)
    • 32 GB memory
    • Storwize V7000 SAN connected disk (50 GB)
    • Shared Ethernet Adapter through Virtual I/O Server (VIOS)
  • LPAR configuration
    • Processing mode: Dedicated
      • Minimum processors: 8
      • Desired processors: 8
      • Maximum processors: 32
    • Dedicated memory
      • Minimum memory: 256 MB
      • Desired memory: 32 GB
      • Maximum memory: 64 GB
  • Virtual Ethernet Adapter (through VIOS)
    • Adapter ID: 2
    • VLAN ID: 1
    • This adapter is required for virtual server activation: Selected

Here are some other miscellaneous commands that I ran:

# ppc64_cpu --frequency 
min:    3.56 GHz (cpu 28) 
max:    3.56 GHz (cpu 4) 
avg:    3.56 GHz 
# ppc64_cpu --cores-present Number of cores present = 8
# sysctl.conf kernel.sem = 250     32000   32    12288

Building the applications using the toolchain

In general the process of building open source distribution is an iterative process. If you are lucky, it might build right out of the box. Unfortunately, since most open source distributions are built on x86 servers, you may encounter problems when you try to build them for a different architecture such as the IBM Power architecture.

In case you encounter problems building open source packages on Power, all you need to do is search the Internet for similar occurrences and see if these reported problems are closely related to what you are experiencing. If they are related, in most cases, there will be advice given on how to resolve the problem.

In my case, while postgresql built fine out of the box after choosing the correct parameters for the configure script, it was not the same experience for apache httpd. In the next sections I describe the fixes I needed to do to get apache httpd built for the Power architecture.

Cross-compiler experience

At the time of the writing this article, the cross-compiler version that I was using was an IBM internal only release specifically built to fix an error I encountered during testing. Readers are encouraged to download version AT 7.0-5, which is the latest cross-compiler (at the time of this publishing) and has the fix for the error described in the next few paragraphs.

When I tried to cross compile the apr-1.5.1 source distribution, I encountered the following compiler message.

"sorry - this program has been built without plugin support"

Following my advice on searching the Internet for possible help and solutions, I found several links that mentioned that the compiler needed to be built with "plugin enabled". I reported this to the toolchain support group and the maintainers of the toolchain responded quickly and provided me with a modified version built with plugin enabled. Note that both the cross compiler and the native compiler version of the toolchain need to be built with plugin enabled. I was able to proceed with the work after getting a version of the toolchain that had the fix.

This was my first experience with the PowerLinux toolchain.

Cross compiling httpd 2.4.3

To cross compile Apache httpd, three other packages were required. They are: apr, apr-util, and pcre. After finding the locations to download the latest packages, start to build them on the x86 server using the toolchain cross compiler.

The problem I encountered when building apr-1.5.1 was building the gen_test_char.o module. Refer to ASF Bugzilla – Bug 51257 for more details. I modified the Makefile.in file as shown below and ran the buildconf script to include the changes to the configure script.

The modifications to Makefile.in are shown here:

[root@stgisv240 apr-1.5.1]# diff -u Makefile.in ../../apr-1.5.1/Makefile.in
--- Makefile.in 2014-03-17 10:10:26.000000000 -0500
+++ ../../apr-1.5.1/Makefile.in 2014-07-03 13:36:11.125013781 -0500
@@ -46,7 +46,6 @@ 
CLEAN_TARGETS = apr-config.out apr.exp exports.c export_vars.c .make.dirs \ build/apr_rules.out tools/gen_test_char@EXEEXT@ \ - tools/gen_test_char.o tools/gen_test_char.lo \ include/private/apr_escape_test_char.h DISTCLEAN_TARGETS = config.cache config.log config.status \ include/apr.h include/arch/unix/apr_private.h \ @@ -132,10 +131,9 @@ make_tools_dir: $(APR_MKDIR) tools
-OBJECTS_gen_test_char = tools/gen_test_char.lo $(LOCAL_LIBS) -tools/gen_test_char.lo: make_tools_dir -tools/gen_test_char@EXEEXT@: $(OBJECTS_gen_test_char) - $(LINK_PROG) $(OBJECTS_gen_test_char) $(ALL_LIBS) +tools/gen_test_char@EXEEXT@: make_tools_dir +tools/gen_test_char@EXEEXT@: tools/gen_test_char.c + $(BUILD_CC) $(CFLAGS_FOR_BUILD) $< -o $@ include/private/apr_escape_test_char.h: tools/gen_test_char@EXEEXT@ $(APR_MKDIR) include/private

After modifying the Makefile.in file, I ran buildconf that created a new configure script. The next paragraph shows the arguments and variables used for the configure script. By the way, I chose the arguments to the config script as suggested by other developers who tried to cross compile Apache httpd and its components on other platforms, such as ARM.

The following are the arguments and environment variables I used when running the configure script for the apr-1.5.1 component on the x86 server.

# Config script using Power Linux toolchain on x86 
BUILD_CC=gcc
CC_FOR_BUILD=gcc
CC=powerpc64-linux-gcc
CPP=powerpc64-linux-cpp
AS=powerpc64-linux-as
AR=powerpc64-linux-ar 
RANLIB=powerpc64-linux-gcc-ranlib
CXX=powerpc64-linux-c++
LD=powerpc64-linux-ld
STRIP=powerpc64-linux-strip 
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/tmp/usr/local/apr --host=powerpc64-linux ac_cv_file__dev_zero=no ac_cv_func_setpgrp_void=no apr_cv_tcp_nodelay_with_cork=no ac_cv_sizeof_struct_iovec=1
BUILD_CC=gcc make install

I used the same distribution with the modifications on the Power server. Note that the PowerLinux toolchain for the Power server uses similarly named executables such as cpp and ld as installed from the native cpp and binutils packages. Modify the CPP and LD variables in the script as shown below. Make sure that the path for the Power toolchain comes first in your PATH environment.

The following are the arguments and environment variables I used when running the configure script for the apr-1.5.1 on the Power server.

# Config script using PowerLinux toolchain on Power 
BUILD_CC=gcc
CC_FOR_BUILD=gcc 
CC=powerpc64-linux-gcc 
CPP=cpp                                   # Note the difference
AS=powerpc64-linux-as
AR=powerpc64-linux-ar 
RANLIB=powerpc64-linux-gcc-ranlib 
CXX=powerpc64-linux-c++ 
LD=ld                                     # Note the difference 
STRIP=powerpc64-linux-strip 
#CFLAGS="-mcpu=440fp -mtune=440fp --sysroot $SYSROOT"
#LDFLAGS=-L$SYSROOT/lib
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/home/usr/local/apr ac_cv_file__dev_zero=no ac_cv_func_setpgrp_void=no apr_cv_tcp_nodelay_with_cork=no ac_cv_sizeof_struct_iovec=1 ac_cv_struct_rlimit=yes BUILD_CC=gcc make install

The following are the arguments and environment variables I used when running the configure script for the apr-util-1.5.3 component on the x86 server.

# Configure script for apr-util-1.5.3 on x86 
BUILD_CC=gcc 
CC_FOR_BUILD=gcc
CC=powerpc64-linux-gcc 
CPP=powerpc64-linux-cpp 
AS=powerpc64-linux-as 
AR=powerpc64-linux-ar
RANLIB=powerpc64-linux-gcc-ranlib
CXX=powerpc64-linux-c++ 
LD=powerpc64-linux-ld
STRIP=powerpc64-linux-strip 
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/tmp/usr/local/apr --host=powerpc64-linux --with- apr=/tmp/usr/local/apr ac_cv_file__dev_zero=no ac_cv_func_setpgrp_void=no apr_cv_tcp_nodelay_with_cork=no ac_cv_sizeof_struct_iovec=1

The following are the arguments and environment variables I used for running the configure script for the apr-util-1.5.3 component on the Power server.

# Configure script for apr-util-1.5.3 on Power
BUILD_CC=gcc 
CC_FOR_BUILD=gcc
CC=powerpc64-linux-gcc 
CPP=cpp 
AS=powerpc64-linux-as
AR=powerpc64-linux-ar
RANLIB=powerpc64-linux-gcc-ranlib
CXX=powerpc64-linux-c++
LD=ld 
STRIP=powerpc64-linux-strip 
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/home/usr/local/apr --host=powerpc64-linux --with- apr=/tmp/usr/local/apr ac_cv_file__dev_zero=no ac_cv_func_setpgrp_void=no apr_cv_tcp_nodelay_with_cork=no ac_cv_sizeof_struct_iovec=1

The following are the arguments and environment variables I used for running the configure script for the httpd component on the x86 server.

# Configure script for httpd 2.4.3 on x86 
CC_FOR_BUILD=gcc
CC=powerpc64-linux-gcc 
CPP=powerpc64-linux-cpp 
AS=powerpc64-linux-as 
AR=powerpc64-linux-ar
RANLIB=powerpc64-linux-gcc-ranlib 
CXX=powerpc64-linux-c++ 
LD=powerpc64-linux-ld
STRIP=powerpc64-linux-strip 
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/tmp/usr/local --host=ppc64 ap_cv_void_ptr_lt_long=no --with- pcre=/tmp/usr/local/bin/pcre-config --with-apr=/tmp/usr/local/apr --with-mpm=worker-- with-apr-util=/tmp/usr/local/apr/bin/apu-1-config

The following are the arguments and environment variables I used for running the configure script for the httpd component on the Power server.

# Configure script for httpd 2.4.3 on Power
CC_FOR_BUILD=gcc
CC=powerpc64-linux-gcc
CPP=cpp 
AS=powerpc64-linux-as 
#ASCPP=powerpc-apm-linux-gnu-as
AR=powerpc64-linux-ar 
RANLIB=powerpc64-linux-gcc-ranlib
#CXXCPP=powerpc-apm-linux-gnu-cpp
CXX=powerpc64-linux-c++ 
LD=/opt/at7.0-5-rc1/bin/ld
STRIP=powerpc64-linux-strip
#CFLAGS="-mcpu=440fp -mtune=440fp --sysroot $SYSROOT" 
#LDFLAGS=-L$SYSROOT/lib
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/home/usr/local --host=ppc64 ap_cv_void_ptr_lt_long=no--with- pcre=/home/usr/local/bin/pcre-config --with-apr=/home/usr/local/apr --with-mpm=worker-- with-apr-util=/home/usr/local/apr/bin/apu-1-config

Cross compiling PostgreSQL 9.4.3

Unlike the previous source distributions needed to build Apache httpd, I had no problems cross compiling PostgreSQL. PostgreSQL is built as is with the following config script.

The following are the arguments and environment variables I used for running the configure script for PostgresSQL on the x86 server.

# Configure script for postgresql-9.3.4 on x86 
CC=powerpc64-linux-gcc
CPP=powerpc64-linux-cpp 
AS=powerpc64-linux-as
AR=powerpc64-linux-ar
RANLIB=powerpc64-linux-gcc-ranlib
CXX=powerpc64-linux-c++ 
D=powerpc64-linux-ld
STRIP=powerpc64-linux-strip 
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/tmp/usr/local --host=powerpc64-linux --without-readline --without-zlib

As with the Apache config scripts, note that the toolchain for the Power server uses similarly named executables such as cpp and ld as installed from the native cpp and binutils packages. Modify the CPP and LD variables in the script as shown below. Make sure that the path for the toolchain comes first in your PATH environment.

The following are the arguments and environment variables I used for running the configure script for PostgresSQL on the Power server.

# Configure script for postgresql-9.3.4 on Power
CC=powerpc64-linux-gcc
CPP=cpp 
AS=powerpc64-linux-as 
AR=powerpc64-linux-ar
RANLIB=powerpc64-linux-gcc-ranlib
CXX=powerpc64-linux-c++
LD=powerpc64-linux-ld 
STRIP=powerpc64-linux-strip 
export CC CPP AS ASCPP AR RANLIB CXXCPP CXX LD STRIP CFLAGS LDFLAGS CC_FOR_BUILD
./configure --prefix=/home/usr/local --host=powerpc64-linux --without-readline --without-zlib

Conclusion

The real value of the toolchain cross compiler is that developers can compile and build applications for the Power Architecture on a development platform they are already familiar with which, in most cases, is x86. As our use cases showed, the toolchain can produce binary executables and libraries that can perform as good as natively compiled applications. Further, cross-compiled libraries are compatible with natively compiled applications. Last but not least, cross-compiled applications can be debugged natively as well as remotely using the gdb debugger.

The PowerLinux toolchain is no different than using the native Linux toolchain on the x86 platform. I hope this article enable readers to get a good understanding of the features and capabilities of the PowerLinux toolchain.

Resources

Acknowledgements

I'd like to thank the following IBMers for their support in writing this article: Tulio Filho, Steve Munroe, and Steve Pratt.


Downloadable resources


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux
ArticleID=982834
ArticleTitle=How to build IBM PowerLinux applications on Intel x86 servers
publish-date=09182014