IBM®
Skip to main content
    Country/region [select]      Terms of use
 
 
    
     Home      Products      Services & solutions      Support & downloads      My account     
 
developerworks > My developerWorks >  Dashboard > Linux for Power Architecture > ... > Performance > Performance Tuning > Information > Page Comparison
developerWorks
Log In   View a printable version of the current page.
Overview Connect Spaces Forums Wikis
Performance Tuning
Browse space Browse Space    
Version 19 by billburos
on Apr 14, 2009 22:36.

<< Changes from 18 to 19
compared with
Current by billburos
on Apr 20, 2009 16:30.
(show comment)
 
Key
These lines were removed. This word was removed.
These lines were added. This word was added.

View page history


There are 1 changes. View first change.

 A good place to begin consolidating information specific to Linux for performance tuning on the Power platform. There's plenty of general information available on the web and in performance tuning books, so here we can begin adding pieces which aren't typically or easily found.
  
  
 \\
 {tip:title=For discussions or questions...}
 To start a discussion or get a question answered, consider posting on the [Linux for Power Architecture forum|http://www.ibm.com/developerworks/forums/forum.jspa?forumID=375].
 {tip}
 \\
  
 *Table of Contents*
 {toc}
  
  
 \\
 Other Linux on Power performance pages include:
  
  * [Performance products]
 * [Performance monitoring]
 * [Performance problem determination]
 * [Performance insights]
  
  
 \\
 \\
  
 h1. Performance Technologies for Linux
  
 Several areas which can be addressed include
 - The [SystemTap|http://sourceware.org/systemtap/] community project and a Linux Kernel Event Trace (LKET) tool based on SystemTap
 - The [libhugetlbfs|http://sourceforge.net/projects/libhugetlbfs] sourceforge community project which supports the ability to load portions of an executable in 16MB large pages on Power systems. The project also allows malloc's to be backed with 16MB large pages on Power systems.
  
 \\
 ----
 h2. Tuning gcc
  
 Check out [Tuning options to consider with gcc|http://www.ibm.com/developerworks/wikis/display/hpccentral/Tuning+options+to+consider+with+gcc]
  
 If you're tuning for gcc, check out the Advance Toolchain..
  
  * [How to use Advance Toolchain for Linux on Power| http://www.ibm.com/developerworks/wikis/display/hpccentral/How+to+use+Advance+Toolchain+for+Linux+on+POWER]
  
 The Advance Toolchain lets you take advantage of the Power6 Decimal Floating Point available in Power6..
  
  * [How to leverage decimal floating point-point unit on Power6 for Linux| http://www.ibm.com/developerworks/wikis/display/hpccentral/How+to+Leverage+Decimal+Floating-Point+unit+on+POWER6+for+Linux]
  
  
 \\
 ----
 h2. Tuning Java
  
 Check out [Tuning considerations for Java | http://www.ibm.com/developerworks/wikis/display/LinuxP/Tuning+considerations+for+Java]
  
  
 \\
 ---
 h2. Transparent large pages with libhugetlbfs
  
 A wiki page has been added which describes a working example of [tuning stream with libhugetlbfs| http://www-941.ibm.com/collaboration/wiki/display/LinuxP/Tuning+stream+with+libhugetlbfs]. The example highlights several common gotcha's seen when working to get basic performance improvements and better performance consistency of a simple memory intensive workload. The example uses IBM compilers, OpenMP binding of threads to processors, and leveraging the new libhugetlbfs to load either the .bss segment, or malloc's, to 16MB large pages.
 * See the [libhugetlbfs FAQs|http://www-941.ibm.com/collaboration/wiki/display/LinuxP/libhugetlbfs+FAQs]
 * See [libhuge short and simple|http://www-941.ibm.com/collaboration/wiki/display/LinuxP/libhuge+short+and+simple] for the short, sweet, and simple steps to using libhugetlbfs
  
 \\
 ----
 h2. New!! Tuning Lx86 math libraries
  
 http://www.ibm.com/developerworks/wikis/display/LinuxP/Lx86+math+library+performance
  
  
 \\
 ----
 h2. Taking advantage of oprofile
  
 * See http://www.ibm.com/developerworks/wikis/display/LinuxP/Taking+advantage+of+oprofile
  
  
 \\
 ----
 h2. Memory usage - comparing AIX and Linux
  
 * New page being created
  
  
  
 \\
 ----
 h2. Helpful Books on Linux Tuning
  
 - Linux Debugging and Performance Tuning - Tips and Techniques - Steve Best
 - System Performance Tuning - Gian-Paolo D. Musumeci and Mike Loukides
 - Optimizing Linux Performance - A Hands-On guide to Linux Performance Tools - Phillip G. Ezolt
  
 Each of these provides good insights into the tools and techniques available for Linux. Linux has matured nicely with all of the basics available for performance problem determination.
  
  
 \\
 ----
 h2. IBM Compilers
  
 * IBM compilers will generally show better performance than gcc compilers.
 ** New IBM compiler versions for Linux on Power stay in sync with most AIX updates
 ** In our experience, SPECcpu2000 and SPEComp2001 application benchmarks runs show little to no differences between SLES 9 and Redhat 4
 *** Components within each of the benchmarks can show specific differences which can be balanced out in the geometric mean used as the reporting number
 *** These programs are not sensitive to kernels and are generally considered smaller domain problems sets which are not representative of large complex enterprise applications
 * IBM compilers have hardware specific enhancements
 ** If compiled for hardware target and maximum optimization, good performance gains are achieved
 ** IBM compilers specifically support GPUL and Power 5 hardware
 ** IBM compilers support program directed feedback (PDF)
 * New versions
 ** VisualAge Version 8 and Fortran Version 10.1 have many incremental fixes and improvements, many of which show improved performance
 ** The IBM compilers are recommended for all benchmarking and high performance work
 ** These new versions include updates for the OpenMP processing directives which can also result in improved performance
 ** XL C/C+\+ Advanced Edition for Linux
 *** See [Trial web site|http://www-306.ibm.com/software/awdtools/xlcpp/features/linux/xlcpp-linux.html] for access
 ** XL Fortran Advanced Edition for Linux
 *** See [Trial web site|http://www-306.ibm.com/software/awdtools/fortran/xlfortran/features/linux/xlf-linux.html] for access
 * Plans are underway to enhance the performance of gcc compilers which continue to improve
 ** Updates to gcc compilers will be available in distro updates
  
 h2. Related products
  
 * ESSL for Linux on POWER available and often used in specialized HPC workloads where engineering and scientific performance improvements are needed.
  
 * FDPR/Pro - Feedback Directed Program Restructuring (FDPR-Pro) is also available for performance improvements of executables. More information is available at this [web site|http://www.haifa.il.ibm.com/projects/systems/cot/fdpr/fdpr_linux.html]
  
  
 \\
 ----
 h1. Basic tuning tips for Linux
  
  
  
  
 \\
 ----
 h2. Other considerations
  
 Linux sports a dynamic kernel that allows you to modify and change many of the parameters without rebooting the system. Most of these parameters are located n the /proc file systems. There are two ways to change it, by using cat and view, and by using the command sysctl. We describe these methods in more detail here.
  
 You can use cat to view, and use echo to change the variable.
  
 As an example, to disable something like IP forwarding:
  
 {noformat}
 # echo "0" > /proc/sys/net/ipv4/ip_forward
 {noformat}\\ Or, the command sysctl provides a simple interface to all the tunable parameters in the /proc file systems.
  
 {noformat}
 # sysctl -w net.ipv4.ip_forward="0"
 {noformat}\\ Because most of the default kernel parameters for system performance are geared toward workstation workload rather than file server or large computation workload, there are tuning parameters that you can use to tune Linux for better performance. Following are some of the sample tuning parameters that can be applied into Linux. For a complete listing, use sysctl or the YaST2 utility.
  
 h2. File system tuning
  
  
 {note:title=Remainder of the page}
 The remainder of the page needs to be reviewed and updated for currency with the latest support available on SLES 9 SP3 and RHEL 4 U3. It will be easier to move the content below to "child" pages from this main page.
 {note}
  
 In Linux, the bdflush file in the /proc directory governs when the bdflush should be activated to clear cache dirty pages.
  
 {noformat}
 # echo "100 5000 640 2560 150 30000 5000 1884 2" > /proc/sys/vm/bdflush
 {noformat}\\ For heavy I/O in a file server and Web server environment, you can also disable the atime. atime basically stores the information of when is the last time the file has been accessed. You can update your /etc/fstab to reflect this.
  
 {noformat}
 # cat /etc/fstab
 /dev/sda3 / reiserfs defaults 1 1
 /dev/system/home /home reiserfs defaults 1 2
 /dev/system/usr /usr reiserfs defaults 1 2
 /dev/system/var /var reiserfs defaults 1 2
 /dev/system/web /web reiserfs noatime,defaults 1 2
 /dev/sda2 swap swap pri=42 0 0
 devpts /dev/pts devpts mode=0620,gid=5 0 0
 proc /proc proc defaults 0 0
 /dev/cdrom /media/cdrom auto ro,noauto,user,exec 0 0
 {noformat}\\ Rather than changing the whole file system to reflect this change, use the command chattr to tag or mark individual files that do not need to store the access time.
  
 {noformat}
 # chattr -R +A /usr/local/apache/logs/httpd.logs
 {noformat}\\
 h2. Network tuning
  
 To reduce the amount of work done at the TCP stack to check on every packet, you can basically disable by echo "0" to the file:
  
 {noformat}
 # echo "0" > /proc/sys/net/ipv4/tcp_sack
  
 # echo "0" > /proc/sys/net/ipv4/tcp_timestamps
 {noformat}\\ Or you can use the command "sysctl \-w <kernel_parameter="choice">".
  
 Often we see clients that do not properly close the TCP connections and so the server keeps the connections open, and may wind up with a large number of open connections. The Linux TCP stack will probe the TCP connections after a given amount of time of inactivity (by default, it is two hours). You can change this wait time as follows:
  
 {noformat}
 # echo "1800" > /proc/sys/net/ipv4/tcp_keepalive_time
 {noformat}\\ If needed, you can also change the tcp_keepalive_intvl and tcp_keepalive_probes to determine the length of time to wait before the next probe occurs.
  
  
  
 h1. Tuning Tips
  
  
 {note:title=Remainder of the page}
 The remainder of the page needs to be reviewed and updated for currency with the latest support available on SLES 9 SP3 and RHEL 4 U3. It will be easier to move the content below to "child" pages from this main page.
 {note}
  
  
 h2. I/O Subsystem
  
 Selectable I/O Schedulers (2.6 feature only)
 * workload & FS dependent
 * CPU bound workload - use noop I/O Scheduler
 * non-CPU bound workload - CFQ and deadline
 * JFS works better on CPU bound workload
  
 SLES 9 default is CFQ
 * good choice for most workloads
  
 Filesystem Performance
 * on large systems ext3 is not the best choice
 * jfs and xfs are better on large systems from a performance standpoint
 * xfs has edge on very large files
 * jfs is better on smaller files
  
 Sequential Read Tuning
 * Increase max_readahead size using htparm tool
 * Read ahead is a function of pagecache
  
 I/O Scheduler Tuning
 * Increase nr_requests to 1024 (improves on most I/O workloads)
  
 NFS Tuning
 * bump up NFS daemons in large NFS server
 * larger MTU (9000 bytes on gigabit Ethernet)
 * Use NFS over TCP and not UDP on Linux
  
  

 
    About IBM      Privacy      Contact