New to PowerLinux tuning and optimizations
New to PowerLinux tuning?
Follow these steps to get up and running with tunings, optimizations, and efficient code on Power Systems servers running Linux.
1. Join the PowerLinux Community
First, you should join this community. We will strongly encourage you to take advantage of the information and tools available on the community.
Ask questions. Follow the community. Read the articles.
2. Update to the latest distro release
Get the latest distro service pack installed on your system.
From Red Hat, RHEL 6.4 is the latest and is recommended. This release has good enhancements and fixes for network drivers, CPU utilization metrics, etc. RHEL 6.3 can be used, but be sure you have a newer kernel than
From SUSE, SLES 11 SP2 is the latest and is recommended.
3. Set up service and productivity tools from the IBM PowerLinux Tools Repository
Download the yum/zypper repo setup rpm package, and then install the service and productivity packages, including the IBM Advance Toolchain for PowerLinux and the Powerlinux SDK.
Go to the IBM Power Linux Tools Repository and download the configuration repository setup rpm.
Install the latest Advance Toolchain and tools.
These steps download and install basic packages that will be helpful going forward. The Advance Toolchain is a "tuck-aside" package which can be leveraged later. Installing it is "safe"; it will not change how your basic system operates, and more importantly, it will install some of the latest tools in the /opt directory.
For more details on the Advance Toolchain, see this page.
If you have gpgcheck problems with the various keys installing Advance Toolchain, you can alternatively run with --nogpgcheck
Note that the Advance Toolchain installs a newer version of oprofile. For example:
# export PATH=/opt/at6.0/bin:/$PATH # ls /opt/at6.0/bin/o* /opt/at6.0/bin/objcopy /opt/at6.0/bin/opgprof /opt/at6.0/bin/objdump /opt/at6.0/bin/ophelp /opt/at6.0/bin/opannotate /opt/at6.0/bin/opimport /opt/at6.0/bin/oparchive /opt/at6.0/bin/opjitconv /opt/at6.0/bin/op-check-perfevents /opt/at6.0/bin/opreport /opt/at6.0/bin/opcontrol /opt/at6.0/bin/oprofiled /opt/at6.0/bin/openssl /opt/at6.0/bin/oprofile_map_events.pl /opt/at6.0/bin/operf /opt/at6.0/bin/oprofile_start.sh
4. Install lpcpu for profiler problem determination
Download the latest script , unpack it, and begin the process of confirming you have the packages installed for basic performance metrics.
You will need packages like sysstat, oprofile, and others installed.
If that works fine, add oprofile to the mix
./lpcpu.sh duration=30 profilers="oprofile top sar mpstat iostat meminfo vmstat"
lpcpu creates a tar ball in /tmp (by default). All of the profile data is available in that tar ball. You can copy the tar ball to an x86 system with the lpcpu package and format all of the data.
5. Looking for a specific tool?
Classic Linux tools
GNU Compiler Collection
Tools and bundles from IBM
Additional open source tools
Tools comparable to AIX
AIX kernel trace. Not available
Specialized tools for POWER
ppc64_cpu (in powerpc-utils)
Tools for porting from x86
- The recommended Java tools are available here: http://www.ibm.com/developerworks/java/jdk/tools/index.html
- pmcount is no longer available on Linux for systems running POWER7.
- To gather hardware counter data, we recommend the following article - Evaluate performance for Linux on POWER.
- If you have tools that were dependent on the format from pmcount, you might take advantage of the scripts in the article above to modify them to fit your needs.
In this context, a new capability of oprofile is being added called "ocount". Patches are being worked and the functionality is being actively tested.
- For an example, see the oprofile mailing list https://lists.sourceforge.net/lists/listinfo/oprofile-list.
- The RFC was posted here: http://sourceforge.net/mailarchive/message.php?msg_id=30915156
AIX-like system tracing
- Many application analysts and software product teams coming from the AIX operating system on Power are looking for the easy built-in system tracing facilities. Given the cross-platform nature of the Linux system and kernel, the kernel has not been implemented with the extensive trace hooks like AIX.
- The focus for new PowerLinux applications is also typically much more strongly focused on coding practices and techniques which should be improved first, long before worrying about possibly optimizing the Linux kernel for that particular workload. In this case, we would more strongly recommend the IBM SDK for application level tools and analysis.
Profiling - kernel and application
- The recommended system profiling tool on Linux is oprofile. Oprofile has two flavors, an older "opcontrol" based approach, and a newer strategic "operf" interface. For more details, see Getting started with Oprofile in the IBM Infocenter library.
- Recently, based on a continuing questions from some AIX customers moving to Linux, a technology release of the beginnings of the lparstat command was released in the powerpc-utils package.
- A new wiki page has been created to track and discuss the activities around the emerging command. See lparstat - an emerging Linux command
- tprof is an AIX profiler with a number of options available for profiling the amount of processor usage in different modes. On Linux, we recommend oprofile, operf, and perf. There are no known plans for making tprof available on Linux platforms.
6. Compiling with GCC?
If you are building your application with GCC, there are some easy considerations. The first is that GCC with the distros represents the GCC technology that was available and current about a year before the initial release for that version was available.
We would recommend taking advantage of the Advance Toolchain.
- The Advance Toolchain's latest compiler will help since the RHEL6 Posix runtime (libc, libm, libpthread,...) are not optimized for POWER7 (POWER4/POWER6 only).
- Advance Toolchain 6.0 includes CPU tuned versions of the Posix runtime for POWER6 and POWER7.
- Advance Toolchain 5.0 includes POWER4/POWERPC970/POWER5/POWER5+ as well).
- Advance Toolchain 5.0 and 6.0 also have POWER7-tuned versions of TCMalloc.
- Advance Toolchain has a improved version of libstdc++ string handling.
- Advance Toolchain 6.0 includes GCC-4.7 - which is 3 generations newer then RHEL6's GCC-4.4.
Currently, the distros build everything with -O2 by default. We find that recompiling with -O3 -mcpu=power7 can help performance significantly.
Second, many applications available today may not have ever been ported to a non-x86 platform. In those cases, there can be some x86-specific code, or techniques, or even assembler code which may need to be modified.
The IBM SDK is perfect for products like this. It provides source code analyzers, migration tool advisors, and other very handy tools to make that porting project easier and faster.
7. Using Java?
When looking at IBM's Power systems, you will want to use the IBM Java product - which is available for easy download on IBM's web sites.
If you're using the Oracle Java on x86 systems, we recommend first to move to the IBM Java on x86 to confirm there are no issues or challenges, and then it's an easy shift to IBM Power systems.
8. Think you need to tune the kernel?
Many people think the Linux kernel "needs" to be tuned, tailored, adjusted, or even re-built.
- Background tracing enabled? The Linux kernel for Power does not use any "always on" tracing or light-weight tracing. There is nothing to turn off.
- AIX kernel tuning parameters? The AIX operating system has implemented a number of RAS-oriented AIX tuning parameters, but this is not something applicable to Linux. Linux generally runs just fine.
- Can I (should I) disable "Storage keys"? Linux does not use storage keys on Power systems. FFDC for Linux is kdump/firmware assisted dump and logs in NVRAM so this is out-of-line and would have no impact on system run-time performance.