IBM®
Skip to main content
    Country/region [select]      Terms of use
 
 
    
     Home      Products      Services & solutions      Support & downloads      My account     
 
developerworks > My developerWorks >  Dashboard > Linux for Power Architecture > ... > Performance Tuning > Re-building a RHEL 5 kernel for Power
developerWorks
Log In   View a printable version of the current page.
Overview Connect Spaces Forums Blogs Wikis
Re-building a RHEL 5 kernel for Power
Added by wburos, last edited by billburos on Jul 06, 2009  (view change)
Labels: 
(None)

For discussions or questions...

To start a discussion or get a question answered, consider posting on the Linux for Power Architecture forum.

Additional Linux on Power performance information is available on the Performance page

Contents


Many thanks to Peter Wong for providing this article.


Introduction

RHEL 5.3

Working through a build break with these steps when building RHEL 5.3 - reported on the forum

We are often asked to build a RHEL 5 kernel (e.g., RHEL 5.2) on Power systems for evaluating a new kernel patch or investigating a performance issue. It is actually fairly easy to build the distro kernel if you are careful with the details.

Here we will present the simple steps for re-building the ppc64 RHEL 5.2 kernel as delivered by the distro. In addition, we will provide instructions to build a kernel with 4KB pages and also with a kernel patch. These steps should also be applicable to re-building the ppc64 RHEL 5.1 kernel, of course you'll need the correct RHEL 5.1 kernel src.rpm file.

  • This page is targeted at technical individuals who are running RHEL 5 on a Power system (ppc64 in the Linux community). These days this generally means the Power5 and Power6 chip architecture. Re-building kernel.org kernels is done a lot by programmers in the community and is a fairly straight-forward process, but the process of re-building the actual shipping distro kernel remains a mystery to many. So the intent here is we want to be fairly specific - what pieces do you need, what steps do you follow, and how to you tell it worked. We're not convinced these steps literally produce the "exact same kernel" as provided by the distro, but for our purposes, this is adequate.

But first, we have a very serious warning. Re-building and re-booting a distro kernel is, in concept and in practice, a fairly easy thing to do. But it is also very easy to mess your system up with a single simple typo, with the practical consequences that you may need to re-install your system. Being unable to re-boot your shared system where someone else has data that wasn't backed up is a "most unpleasant" feeling.

Be Wary

Danger, Will Robinson

First. We strongly urge you to practice these steps on a victim system, where you have no problems what-so-ever of completely re-installing the system. A "victim system" means there is nothing of importance on the system. No important data. No critical users or applications.

Second. These steps are provided simply as a step-by-step guide of how several of us re-build the kernels in the labs on freshly installed victim systems. We most-certainly are not the official source for the right steps, nor do we claim this will work for everyone, so you should not expect any level of support from us on this. See the "First" point above.

Third. This should be intuitive, but amazingly-enough often isn't clearly understood. Therefore, we'll say it here explicitly. If you re-build your own kernel and you use that kernel, you have invalidated any hope for "official support" from your service provider while using that kernel. See the "First" point above.


Find the RHEL 5.2 (or 5.3) kernel source

For re-building RHEL 5.2, you'll want kernel-2.6.18-92.el5.src.rpm. It is on the source CD that comes with your CD set. You will not find it on your standard CDs.

For re-building RHEL 5.3, you'll want kernel-2.6.18-128.el5.src.rpm file.

As an example, we copied the source DVD for the RHEL 5.x release to our system, mounted it, and installed the rpm.

For RHEL 5.2

copy the Source DVD iso to local system - it's a big file...    2,843,054,080 bytes..

# mkdir /dvd
# mount -o loop RHEL5.2-Server-20080430.0-SRPMS-DVD.iso /dvd
# cd /dvd
# find . -name kernel*.src.rpm
./SRPMS/kernel-2.6.18-92.el5.src.rpm
# cd SRPMS
# rpm -i kernel-2.6.18-92.el5.src.rpm

For RHEL 5.3

# mkdir /dvd
# mount -o loop RHEL5.3-Server-20090106.0-SRPMS-DVD.iso /dvd
# cd /dvd
# find . -name kernel*
./SRPMS/kernel-2.6.18-128.el5.src.rpm
# cd SRPMS
# rpm -i kernel-2.6.18-128.el5.src.rpm
# cd /usr/src/redhat/SPECS/
# ls
kernel-2.6.spec

To determine whether you have installed the source RPM, look under /usr/src/redhat/SPECS. If there is nothing there, then you probably have not installed the correct RPM yet.


Install the kernel-2.6.18-92.el5.src.rpm

The instructions are as follows:

rpm -i kernel-2.6.18-92.el5.src.rpm
    (If you see lots of these warnings:

        warning: user brewbuilder does not exist - using root
        warning: group brewbuilder does not exist - using root

     you can ignore them.)

cd /usr/src/redhat/SOURCES
ls *.patch | wc -w 
1809 
    (this directory contains the base kernel source code and 1809 patch files)

cd /usr/src/redhat/SPECS
ls 
kernel-2.6.spec
    (this directory contains the kernel-2.6.spec file which specifies the 
     steps to build the kernel)


Build the kernel

You might have to install unifdef-1.171-5.fc6.ppc.rpm which is needed for the build.

The build instructions are as follows:

cd /usr/src/redhat/SPECS/

vi kernel-2.6.spec

     (Search for...)
#
# Polite request for people who spin their own kernel rpms:
# please modify the "buildid" define in a way that identifies
# that the kernel isn't the stock distribution kernel, for example,
# by setting the define to ".local" or ".bz123456"
#
#% define buildid

     (Change the commented out line)
#% define buildid

     (to something informative - in this case - the normal 64K kernel)
%define buildid .64K

<save and file>

cd /usr/src/redhat
rpmbuild -ba --target ppc64 SPECS/kernel-2.6.spec

     (--target ppc64 is used to indicate that you want to build a ppc64 kernel)

     (the rpmbuild process will take a while to complete...)

Note: During the compilation, gpg (GNU Privacy Guard) may want some system activity to generate security related keys, which causes the build process to pause.

  • If this happens, you can open up another session and run "dd if=/dev/mapper/VolGroup00-LogVol00 of=/dev/null bs=1M count=500" or anything that generate system activities. The paused build process should resume after the request from gpg is satisfied.

Here's an example of the error:

+ gpg --homedir . --batch --gen-key /usr/src/redhat/SOURCES/genkey
gpg: WARNING: unsafe permissions on homedir `.'
++++++++++..+++++++++++++++.++++++++++.+++++.+++++++++++++++.++++++++++++++++++++.
+++++.++++++++++++++++++++.+++++.++++++++++..+++++..+++++++++++++++!++++++++++++++++++++.
++++++++++.++++++++++..+++++++++++++++++++++++++++++++++++++++++++++++++++++++.++++++++++.
+++++++++++++++.+++++!.++++++++++++++++++++.+++++++++++++++.+++++++++++++++++++++++++.
+++++++++++++++..+++++.+++++.+++++.++++++++++.+++++...+++++.++++++++++.+++++!++++++++++++++++++++.
.++++++++++.+++++++++++++++.+++++++++++++++++++++++++

Not enough random bytes available.  Please do some other work to give
the OS a chance to collect more entropy! (Need 274 more bytes)

Admit'ably, this is a little strange. What we have found that works is:

<in another window>

# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda3             20315788   6217952  13049196  33% /
/dev/sda2               101082     26198     69665  28% /boot

# dd if=/dev/sda3 of=/dev/null &

<repeating the dd until the build process resumes..  there's got to be a smarter approach... >


rpmbuild

rpmbuild is the clever program that applies all of the patches to generate the overall kernel source, builds the kernels, modules and packages all the resulting files in an RPM for installation.

The rpmbuild step will result in these final messages:

Wrote: /usr/src/redhat/SRPMS/kernel-2.6.18-92.el5.64K.src.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-devel-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-headers-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debug-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debug-debuginfo-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debug-devel-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-kdump-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-kdump-devel-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debuginfo-common-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debuginfo-2.6.18-92.el5.64K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-kdump-debuginfo-2.6.18-92.el5.64K.ppc64.rpm


Install the newly built kernel

The instructions are as follows:

cd /usr/src/packages/RPMS/ppc64

rpm -i kernel-2.6.18-92.el5.64K.ppc64.rpm

These steps install all the pieces in the right place. The /etc/yaboot.conf file is updated as follows:

boot=/dev/sda1
init-message=Welcome to Red Hat Enterprise Linux Server!\nHit <TAB> for boot options

partition=2
timeout=80
install=/usr/lib/yaboot/yaboot
delay=5
enablecdboot
enableofboot
enablenetboot
nonvram
fstype=raw
default=2.6.18-92.el5.6

image=/vmlinuz-2.6.18-92.el5.64K
        label=2.6.18-92.el5.6
        read-only
        initrd=/initrd-2.6.18-92.el5.64K.img
        root=/dev/VolGroup00/LogVol00
        append="console=hvc0 rhgb quiet"

image=/vmlinuz-2.6.18-92.el5
        label=linux
        read-only
        initrd=/initrd-2.6.18-92.el5.img
        root=/dev/VolGroup00/LogVol00
        append="console=hvc0 rhgb quiet"

The variable "default" is set to "2.6.18-92.el5.6" for the new kernel. We strongly recommend that you keep the boot entry for the RHEL 5.2 kernel around, in case the newly built kernel is having some boot issues.


Confirm the boot

Indeed you must confirm that it boots correctly.

You can verify the base page size by the following command:

getconf PAGESIZE
65536 

Page size of 65536 bytes clearly indicates that we are running on 64KB pages.


Build a kernel with the 4KB support

After you have completed the above steps successfully, we have a good base for building other kernels. Let's build a 4KB kernel.

Let's follow the instructions below.

cd /usr/src/redhat/SPECS
vi kernel-2.6.spec

  Change %define buildid .64K
  To     %define buildid .4K
      <save and file>

cd /usr/src/redhat/SOURCES

vi kernel-2.6.18-ppc64.config

  Change CONFIG_PPC_64K_PAGES=y
  To     CONFIG_PPC_64K_PAGES=n
      <save and file>

Do the same to kernel-2.6.18-ppc64-debug.config and kernel-2.6.18-ppc64-kdump.config

cd /usr/src/redhat
rpmbuild -ba --target ppc64 --without kabichk SPECS/kernel-ppc64.spec

"--without kabichk" is used to disable ABI compatibility check. Without specifying it, the kernel cannot be built. And we assume that ABI compatibility is not crucial for what you are doing.

This should succeed. We will see the following at the end of the build.

Wrote: /usr/src/redhat/SRPMS/kernel-2.6.18-92.el5.4K.src.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-devel-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-headers-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debug-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debug-debuginfo-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debug-devel-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-kdump-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-kdump-devel-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debuginfo-common-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-debuginfo-2.6.18-92.el5.4K.ppc64.rpm
Wrote: /usr/src/redhat/RPMS/ppc64/kernel-kdump-debuginfo-2.6.18-92.el5.4K.ppc64.rpm


Install the 4KB kernel

rpm -e kernel-2.6.18-92.el5.64K
(The 64K kernel might have conflicts with the 4K kernel.)

cd /usr/src/redhat/RPMS/ppc64
rpm -i ppc64/kernel-2.6.18-92.el5.4K.ppc64.rpm

The /etc/yaboot.conf will be modified accordingly.

boot=/dev/sda1
init-message=Welcome to Red Hat Enterprise Linux Server!\nHit <TAB> for boot options

partition=2
timeout=80
install=/usr/lib/yaboot/yaboot
delay=5
enablecdboot
enableofboot
enablenetboot
nonvram
fstype=raw
default=2.6.18-92.el5.4

image=/vmlinuz-2.6.18-92.el5.4K
        label=2.6.18-92.el5.4
        read-only
        initrd=/initrd-2.6.18-92.el5.4K.img
        root=/dev/VolGroup00/LogVol00
        append="console=hvc0 rhgb quiet"

image=/vmlinuz-2.6.18-92.el5
        label=linux
        read-only
        initrd=/initrd-2.6.18-92.el5.img
        root=/dev/VolGroup00/LogVol00
        append="console=hvc0 rhgb quiet"

 
And reboot.

You can verify the base page size by the following command:

getconf PAGESIZE
4096 

The page size clearly indicates that we are on 4KB pages.


Apply a kernel patch

Let's follow the instructions below.

cd /usr/src/redhat/SPECS
vi kernel-2.6.spec

Modify buildid as specified above.

Search for Patch99990 which is almost at the end of all patches.

Assume that the patch right before Patch99990 is Patch12345.

Add "Patch12346: abc.patch" where abc.patch is your patch.

Search for %patch12345 -p1.

Add %patch12346 -p1.

<save and file>

Add abc.patch to /usr/src/redhat/SOURCES.

Run rpmbuild as specified above.


Using the re-built RPM on other RHEL 5.2 systems

It is indeed desirable to install the built RPM on other RHEL 5.2 systems, but you need to be very careful.

Note that RHEL 5.2 systems are different in many ways, e.g., the setup of the volume manager.

Once the RPM is installed, follow the instructions below.

cd /boot

mkinitrd initrd-2.6.18-92.el5.64K.img 2.6.18-92.el5.64K
(Generate a fresh initrd image for booting purposes.)

And then reboot.


In summary

This quick report takes you through the practical steps of rebuilding a RHEL 5.2 kernel for Power systems. This exercise is intended simply as a way to see how the RHEL 5.2 kernel is built, installed, and booted to.


 
    About IBM Privacy Contact