IBM®
Skip to main content
    Country/region [select]      Terms of use
 
 
    
     Home      Products      Services & solutions      Support & downloads      My account     
 
developerworks > My developerWorks >  Dashboard > Linux for Power Architecture > ... > Performance Tuning > Re-building a SLES 11 kernel for Power
developerWorks
Log In   View a printable version of the current page.
Overview Connect Spaces Forums Wikis
Re-building a SLES 11 kernel for Power
Browse space Browse Space    
Added by wburos, last edited by billburos on Nov 18, 2009  (view change)
Labels: 
(None)

For discussions or questions...

To start a discussion or get a question answered, consider posting on the Linux for Power Architecture forum.

Additional Linux on Power performance information is available on the Performance page

Contents

This page continues an earlier thread for re-building the SLES 10 kernel. As with SLES 10, here we'll show it's easy to re-build the SLES 11 distro kernel if you're careful with the details.


First things first.

As always.. a warning.

Re-building and re-booting a distro kernel is conceptually, and in practice, a fairly easy thing to do. But it is also very easy to mess up your system with a single simple typo, with the practical consequences that you may need to reinstall your system. Being unable to re-boot a shared system where someone else has data that wasn't backed up is a "most unpleasant" feeling.

Be Wary

First. We strongly urge you to practice these steps on a victim system, where you have no problems what-so-ever of completely reinstalling the system. A "victim system" means there is nothing of importance on the system. No important data. No critical users or applications. No key work-in-progress.

Second. These steps are provided simply as a step-by-step guide of how several of us re-built the kernel in the labs on freshly installed victim systems. We most-certainly are not the official source for the right steps, nor do we claim this will work for everyone, and you should not expect any level of support from us on this.

Third. This should be intuitive, but amazingly-enough often isn't clearly understood. So we'll say it here explicitly. If you re-build your own kernel and you use that kernel, you have invalidated any hope for "official support" from your service provider while using that kernel.


SLES 11. Find the kernel-source "src.rpm"

On SLES 11, there are two DVD iso images.

  • SLES-11-DVD-ppc64-GM-DVD1.iso
  • SLES-11-DVD-ppc64-GM-DVD2.iso

If you mount the second DVD, it has the "source" files. Here we assume you've downloaded the DVD iso images to a local directory, let's say: /usr/local/iso

# cd /usr/local/iso
# ls
SLES-11-DVD-ppc64-GM-DVD1.iso  SLES-11-DVD-ppc64-GM-DVD2.iso
# mkdir /dvd
# mount -t iso9660 -o loop SLES-11-DVD-ppc64-GM-DVD2.iso /dvd
# find /dvd -name kernel-source*
/dvd/suse/src/kernel-source-2.6.27.19-5.1.src.rpm

You'll need the *.src.rpm file.

This rpm cleverly doesn't show up as installed if you do a rpm -qa | grep kernel-source. To tell whether you have installed the package, look under /usr/src/packages/SPECS. If there's nothing there, then you have not installed the correct rpm.


Install the kernel-source "src.rpm"

You want the "src.rpm", not just the ".rpm" file. 

cd /dvd/suse/src
rpm -i kernel-source-2.6.27.19-5.1.src.rpm

Copy the "spec" file from SOURCES over to the SPECS sub-directory.

cd /usr/src/packages/SOURCES
cp kernel-ppc64.spec ../SPECS

Edit the ppc64 "spec" file to add a unique identifier

cd /usr/src/packages/SPECS
vi kernel-ppc64.spec

Change the Release identifier from "5" to something like "5.001"

Release:    5

to 

Release:    5.001

Save and file.

Note the current kernel level...

# uname -r
2.6.27-19-5-ppc64


rpmbuild -ba kernel-ppc64.spec

rpmbuild is the clever step that packages all of the patches that constitute the "whole" of the kernel.

# rpmbuild -ba kernel-ppc64.spec
error: Failed build dependencies:
	sparse is needed by kernel-ppc64-2.6.27.19-5.001.ppc64
	fdupes is needed by kernel-ppc64-2.6.27.19-5.001.ppc64
	kernel-dummy is needed by kernel-ppc64-2.6.27.19-5.001.ppc64

Now you need the SLES 11 SDK iso image.

mkdir /sdk
mount -t iso9660 -o loop /usr/local/iso/SLE-11-SDK-DVD-ppc64-GM-Media1.iso /sdk
cd /sdk/suse/ppc64/
rpm -i sparse-0.4.1.git1-1.21.ppc64.rpm
rpm -i fdupes-1.40-42.22.ppc64.rpm

Try again (noticing that there is no such thing as a kernel-dummy rpm file).

# rpmbuild -ba kernel-ppc64.spec
error: Failed build dependencies:
	kernel-dummy is needed by kernel-ppc64-2.6.27.19-5.001.ppc64

Eh?

From the README.SUSE file in /usr/src/packages/SOURCES 

  *  kernel-dummy

     This package is relevant inside the SUSE build system only. We use
     it to synchronize release numbers among the kernel packages. When
     building packages locally, the kernel-dummy package can safely be
     ignored.

Ok. Comment out that line.

%if ! 0%{?opensuse_bs}
# BuildRequires:  kernel-dummy
%endif

We're ready to go. But before we start, we want to modify the "spec" file to compile in parallel.

vi kernel-ppc64.spec

  • After the "build_flavor" line, add a line to define "jobs" to be the number of available CPUs.
%define build_flavor "ppc64"
%define jobs %(cat /proc/cpuinfo | grep processor | wc -l)

Try again.

rpmbuild -ba kernel-ppc64.spec
      <lots of compilation messages>
      <eventually...>
Wrote: /usr/src/packages/SRPMS/kernel-ppc64-2.6.27.19-5.001.nosrc.rpm
Wrote: /usr/src/packages/RPMS/ppc64/kernel-ppc64-2.6.27.19-5.001.ppc64.rpm
Wrote: /usr/src/packages/RPMS/ppc64/kernel-ppc64-base-2.6.27.19-5.001.ppc64.rpm
Wrote: /usr/src/packages/RPMS/ppc64/kernel-ppc64-extra-2.6.27.19-5.001.ppc64.rpm


Building in parallel

Quick tangent.

To confirm you're building in parallel, in a separate window fire up "top" and press the "1" key. Here's an example of an 8 core (16 CPUs with SMT on) Power 6 system running SLES 11.

top - 09:51:50 up 17:30,  2 users,  load average: 20.87, 8.87, 3.80
Tasks: 277 total,  19 running, 256 sleeping,   0 stopped,   2 zombie
Cpu0  : 41.5%us,  3.4%sy,  0.0%ni, 12.4%id,  6.1%wa,  0.0%hi,  0.0%si, 36.5%st
Cpu1  : 52.8%us,  2.1%sy,  0.0%ni,  8.4%id,  0.0%wa,  0.0%hi,  0.3%si, 36.4%st
Cpu2  : 55.7%us,  1.3%sy,  0.0%ni,  7.4%id,  0.0%wa,  0.0%hi,  0.0%si, 35.6%st
Cpu3  : 40.4%us,  2.4%sy,  0.0%ni, 21.0%id,  0.0%wa,  0.0%hi,  0.0%si, 36.2%st
Cpu4  : 45.3%us,  3.7%sy,  0.0%ni, 13.9%id,  0.0%wa,  0.0%hi,  0.3%si, 36.8%st
Cpu5  : 49.2%us,  1.6%sy,  0.0%ni, 12.4%id,  0.0%wa,  0.0%hi,  0.0%si, 36.8%st
Cpu6  : 48.7%us,  2.1%sy,  0.0%ni,  4.7%id,  0.5%wa,  0.0%hi,  0.0%si, 43.9%st
Cpu7  : 46.8%us,  1.8%sy,  0.0%ni,  6.3%id,  0.0%wa,  0.0%hi,  0.0%si, 45.0%st
Cpu8  : 46.1%us,  3.2%sy,  0.0%ni, 10.0%id,  0.0%wa,  0.0%hi,  0.0%si, 40.8%st
Cpu9  : 48.2%us,  2.4%sy,  0.0%ni,  8.9%id,  0.0%wa,  0.0%hi,  0.0%si, 40.5%st
Cpu10 : 50.4%us,  1.8%sy,  0.0%ni,  6.8%id,  2.6%wa,  0.3%hi,  0.0%si, 38.1%st
Cpu11 : 45.3%us,  2.4%sy,  0.0%ni, 10.8%id,  2.4%wa,  0.0%hi,  0.0%si, 39.2%st
Cpu12 : 51.7%us,  2.6%sy,  0.0%ni,  9.5%id,  0.0%wa,  0.0%hi,  0.0%si, 36.1%st
Cpu13 : 41.2%us,  3.4%sy,  0.0%ni, 18.7%id,  0.0%wa,  0.0%hi,  0.0%si, 36.7%st
Cpu14 : 55.5%us,  1.8%sy,  0.0%ni, 10.3%id,  0.0%wa,  0.3%hi,  0.0%si, 32.1%st
Cpu15 : 39.4%us,  2.9%sy,  0.0%ni, 24.3%id,  0.0%wa,  0.0%hi,  0.0%si, 33.4%st
Mem:  130335424k total, 19881792k used, 110453632k free,  1300288k buffers
Swap:  4096384k total,        0k used,  4096384k free, 17655936k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
15460 root      20   0  192m 180m 9664 R   64  0.1   0:07.15 cc1                
20488 root      20   0 67520  61m 9664 R   30  0.0   0:01.13 cc1                
21028 root      20   0 50304  39m 9664 R   13  0.0   0:00.48 cc1                
21114 root      20   0 50176  45m 9664 R   10  0.0   0:00.39 cc1                
21236 root      20   0 50112  44m 9664 R    7  0.0   0:00.27 cc1  
21251 root      20   0 49280  33m 9664 R    6  0.0   0:00.24 cc1                
21332 root      20   0 49216  36m 8384 R    5  0.0   0:00.19 cc1                
21386 root      20   0 49216  30m 9664 R    5  0.0   0:00.19 cc1                
21313 root      20   0 36160  28m 9472 R    4  0.0   0:00.17 cc1                
21267 root      20   0 49216  29m 9600 R    4  0.0   0:00.16 cc1                
21374 root      20   0 32832  27m 9664 R    3  0.0   0:00.12 cc1                
21401 root      20   0 32768  25m 7360 R    2  0.0   0:00.09 cc1                
21346 root      20   0 32768  24m 7360 R    2  0.0   0:00.08 cc1                
21425 root      20   0 32832  20m 7360 R    2  0.0   0:00.06 cc1                
    3 root      RT  -5     0    0    0 S    0  0.0   0:00.07 migration/0        
   90 root      20   0     0    0    0 S    0  0.0   0:06.30 pdflush            
15481 root      20   0 16512  11m 4096 S    0  0.0   0:00.04 as                 
18123 root      20   0  7232 4288 3072 S    0  0.0   0:00.01 make               
18300 root      20   0  7232 4352 3072 S    0  0.0   0:00.02 make               
19403 root      20   0  7232 4416 3072 D    0  0.0   0:00.01 make               
19487 root      20   0  7232 4224 3072 S    0  0.0   0:00.01 make               
19611 root      20   0  7232 4224 3072 S    0  0.0   0:00.01 make               
    1 root      20   0  3648 1088  832 S    0  0.0   0:01.83 init


Install the newly built kernel.

So then we can install the new kernel rpm.

# cd /usr/src/packages/RPMS/ppc64
# rpm -i kernel-ppc64-2.6.27.19-5.001.ppc64.rpm 
Setting up /lib/modules/2.6.27.19-5.001-ppc64

We recommend that you reboot to this kernel and confirm that it boots correctly.

We recommend that you do NOT change the default boot kernel, leaving the test kernel as an alternative choice.


Building a recent mainline kernel

For completeness, here's the quick steps for building a mainline kernel.

Go to kernel.org, download a stable or recent kernel. For example, at the time this is being
written, the current mainline kernel is 2.6.32.rc7 and the current stable kernel is 2.6.31.6.

In this example, we download the current mainline kernel

http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-2.6.32-rc7.tar.bz2

Quick steps are available here: http://kernelnewbies.org/KernelBuild

# cd /root
# tar -jxf linux-2.6.32-rc7.tar.bz2
# cd linux-2.6.32-rc7
# make defconfig

Assuming you've built the SLES 11 kernel above, an interesting exercise is to look at the
differences in the two .config files - the primary one used by SLES 11 and the .config file created for the mainline kernel.

# find /usr/src/ -name .config
/usr/src/linux-2.6.27.19-5-obj/ppc64/kdump/.config
/usr/src/linux-2.6.27.19-5-obj/ppc64/debug/.config
/usr/src/linux-2.6.27.19-5-obj/ppc64/ppc64/.config
/usr/src/linux-2.6.27.19-5-obj/ppc64/default/.config

The mainline kernel version

# find /root/linux-2.6.32-rc7 -name .config
/root/linux-2.6.32-rc7/.config

Compare the two

# sdiff /usr/src/linux-2.6.27.19-5-obj/ppc64/ppc64/.config /root/linux-2.6.32-rc7/.config | less

For the build..

# make all
# su -c "make modules_install install"

If running under ABAT, use

# boot-to --kernel /boot/vmlinux-2.6.32.rc7 --initrd --/boot/initrd-2.6.32.rc7

Note that with the 2.6.32.rc7 kernel, for the kernel performance events config flags

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
CONFIG_EVENT_PROFILE=y
# CONFIG_PERF_COUNTERS is not set
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set


 
    About IBM      Privacy      Contact