Topic
  • 20 replies
  • Latest Post - ‏2009-07-24T18:19:18Z by SystemAdmin
SystemAdmin
SystemAdmin
706 Posts

Pinned topic Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

‏2009-07-02T22:51:49Z |
Trying to follow the directions at

http://www.ibm.com/developerworks/wikis/display/LinuxP/Re-building+a+RHEL+5+kernel+for+Power

to create a 4K pagesize kernel on RHEL 5.3, I'm finding the directions refer to a kernel-ppc64.spec, which is missing on my system; further, if I just use kernel-2.6.spec, I get:
...
In file included from arch/powerpc/mm/mem.c:35:
include/asm/pgalloc.h: In function 'pgd_free':
include/asm/pgalloc.h:48: error: implicit declaration of function 'subpage_prot_free'
make[1]: *** http://arch/powerpc/mm/mem.o Error 1

I'm clearly missing something. Suggestions?
Updated on 2009-07-24T18:19:18Z at 2009-07-24T18:19:18Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-02T23:52:39Z  
    hmm.. that's strange.. but I haven't tried this recently on rhel 5.3..

    the missing file should've been installed with the kernel src rpm..

    I'll try it here and see where it got moved..
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-03T01:40:38Z  
    hmm.. that's strange.. but I haven't tried this recently on rhel 5.3..

    the missing file should've been installed with the kernel src rpm..

    I'll try it here and see where it got moved..
    Well, I followed the instructions..
    Downloaded the src DVD iso.. mounted that..
    installed the kernel src rpm
    and the kernel-2.6.spec file is there.

    Download RHEL5.3-Server-20090106.0-SRPMS-DVD.iso
    It's a very big file. 2,951,993,344 bytes

    1. mount -o loop RHEL5.3-Server-20090106.0-SRPMS-DVD.iso /mnt
    2. cd /mnt
    3. find . -name kernel*
    ./SRPMS/kernel-2.6.118-128.el5.src.rpm
    1. cd SRPMS
    2. rpm -i kernel-2.6.18-128.el5.src.rpm
    3. cd /usr/src/redhat/SPECS/
    4. ls
    kernel-2.6.spec

    I'll update the wiki page with the RHEL 5.3 quick summary...

    Let me know if something doesn't work with these steps...
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-03T14:49:08Z  
    Well, I followed the instructions..
    Downloaded the src DVD iso.. mounted that..
    installed the kernel src rpm
    and the kernel-2.6.spec file is there.

    Download RHEL5.3-Server-20090106.0-SRPMS-DVD.iso
    It's a very big file. 2,951,993,344 bytes

    1. mount -o loop RHEL5.3-Server-20090106.0-SRPMS-DVD.iso /mnt
    2. cd /mnt
    3. find . -name kernel*
    ./SRPMS/kernel-2.6.118-128.el5.src.rpm
    1. cd SRPMS
    2. rpm -i kernel-2.6.18-128.el5.src.rpm
    3. cd /usr/src/redhat/SPECS/
    4. ls
    kernel-2.6.spec

    I'll update the wiki page with the RHEL 5.3 quick summary...

    Let me know if something doesn't work with these steps...
    kernel-2.6.spec is, in fact, there. That's what's used for the first example, where we rebuild the kernel without any modifications.

    When we switch to building a 4K kernel, the file used is kernel-ppc64.spec; that file does not appear to be there. Browsing rpmfind.com suggests that it may only be included as part of SLES 10.

    Using kernel-2.6.spec for the rpmbuild results in the compile failure I quoted in my original post.

    -roy
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-03T15:53:41Z  
    kernel-2.6.spec is, in fact, there. That's what's used for the first example, where we rebuild the kernel without any modifications.

    When we switch to building a 4K kernel, the file used is kernel-ppc64.spec; that file does not appear to be there. Browsing rpmfind.com suggests that it may only be included as part of SLES 10.

    Using kernel-2.6.spec for the rpmbuild results in the compile failure I quoted in my original post.

    -roy
    ahh... I think I see what you mean.. in the 4K steps section, a SPECS/kernel-ppc64.spec file is specified as the last step.

    
    rpmbuild -ba --target ppc64 --without kabichk SPECS/kernel-ppc64.spec
    


    I wonder if that's created in the preceeding step where we try to be sure just the basic kernel can be built and booted.
    Interesting.. I'll have to snag a system sometime this weekend and try.
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-03T16:11:53Z  
    ahh... I think I see what you mean.. in the 4K steps section, a SPECS/kernel-ppc64.spec file is specified as the last step.

    <pre class="jive-pre"> rpmbuild -ba --target ppc64 --without kabichk SPECS/kernel-ppc64.spec </pre>

    I wonder if that's created in the preceeding step where we try to be sure just the basic kernel can be built and booted.
    Interesting.. I'll have to snag a system sometime this weekend and try.
    ah. even a basic build won't work...

    will check...
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-06T22:46:29Z  
    ah. even a basic build won't work...

    will check...
    Various discussions continue and haven't panned out. Opening a Bugzilla to track.

    We think the "kernel-ppc64.spec" is a typo as we mixed and matched SPECS and CONFIG files. Will test a RHEL 5.2 build tomorrow to confirm.

    I updated the wiki page to point out there's a build break as currently documented.
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-07T15:11:19Z  
    Various discussions continue and haven't panned out. Opening a Bugzilla to track.

    We think the "kernel-ppc64.spec" is a typo as we mixed and matched SPECS and CONFIG files. Will test a RHEL 5.2 build tomorrow to confirm.

    I updated the wiki page to point out there's a build break as currently documented.
    Strange. We are finding that the steps work fine on a number of other Power systems we're playing with, and have not yet determined where/what causes the build break.

    We setup a fresh RHEL 5.3 on a 4-core Power 5 system, installed the kernel-2.6.18-128.el5.src.rpm, and ran rpmbuild without any changes to see if the kernel would build as-is.

    rpmbuild -ba --target ppc64 SPECS/kernel-2.6.spec

    and it built fine.

    We also put a fresh RHEL 5.3 on a 4-core Power 6 system, installed the src.rpm, and re-built the kernel. No problems.

    The system where we saw the same build break was an 8-core system, and we had installed a bunch of other kernel* files... so we need to try that next. It's hard to imagine that more cores would cause a build break, so the thinking is the extra files installed is causing a conflict.

    For kernel files, we just have these installed on the systems where the build is working..

    1. rpm -qa | grep kernel
    kernel-devel-2.6.18-128.el5
    kernel-headers-2.6.18-128.el5
    kernel-2.6.18-128.el5

    For the system which is seeing the build break...

    1. rpm -qa |grep kernel
    kernel-2.6.18-128.el5
    kernel-headers-2.6.18-128.el5
    kernel-debuginfo-2.6.18-128.el5
    kernel-debuginfo-common-2.6.18-128.el5
    kernel-debug-2.6.18-128.el5
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-07T22:50:44Z  
    Strange. We are finding that the steps work fine on a number of other Power systems we're playing with, and have not yet determined where/what causes the build break.

    We setup a fresh RHEL 5.3 on a 4-core Power 5 system, installed the kernel-2.6.18-128.el5.src.rpm, and ran rpmbuild without any changes to see if the kernel would build as-is.

    rpmbuild -ba --target ppc64 SPECS/kernel-2.6.spec

    and it built fine.

    We also put a fresh RHEL 5.3 on a 4-core Power 6 system, installed the src.rpm, and re-built the kernel. No problems.

    The system where we saw the same build break was an 8-core system, and we had installed a bunch of other kernel* files... so we need to try that next. It's hard to imagine that more cores would cause a build break, so the thinking is the extra files installed is causing a conflict.

    For kernel files, we just have these installed on the systems where the build is working..

    1. rpm -qa | grep kernel
    kernel-devel-2.6.18-128.el5
    kernel-headers-2.6.18-128.el5
    kernel-2.6.18-128.el5

    For the system which is seeing the build break...

    1. rpm -qa |grep kernel
    kernel-2.6.18-128.el5
    kernel-headers-2.6.18-128.el5
    kernel-debuginfo-2.6.18-128.el5
    kernel-debuginfo-common-2.6.18-128.el5
    kernel-debug-2.6.18-128.el5
    I suspect the different packages may be a red herring. On the system here, here's all I've got:

    % rpm -q --queryformat "%{NAME}-%{VERSION}.%{ARCH}\n" -a |grep -i kernel
    kernel-headers-2.6.18.ppc64
    kernel-2.6.18.ppc64

    Compiling the stock kernel still works fine; compiling the 4K kernel doesn't.

    -roy
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-08T12:51:11Z  
    I suspect the different packages may be a red herring. On the system here, here's all I've got:

    % rpm -q --queryformat "%{NAME}-%{VERSION}.%{ARCH}\n" -a |grep -i kernel
    kernel-headers-2.6.18.ppc64
    kernel-2.6.18.ppc64

    Compiling the stock kernel still works fine; compiling the 4K kernel doesn't.

    -roy
    Ok. Will ask the developers to look at what got broken with the 4KB path.

    This isn't normally built or supported, so it doesn't get checked in a normal release cycle.

    Is there a particular reason you're trying the 4KB pages?
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-08T18:11:07Z  
    Ok. Will ask the developers to look at what got broken with the 4KB path.

    This isn't normally built or supported, so it doesn't get checked in a normal release cycle.

    Is there a particular reason you're trying the 4KB pages?
    Our match engine, which holds information like user ratings and predictions for what they'll like, has a huge cache; this cache uses very large memory-mapped files that are very carefully structured around page boundaries for block allocation, and it's designed, right now, for 4K page sizes. Re-engineering it is being worked on, but ... will take some time. Meanwhile, we sort of thought the 4K rebuild was going to be, you know, easy. Since there was a helpful wiki article on exactly how to do it :)

    -roy
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-08T19:50:32Z  
    Our match engine, which holds information like user ratings and predictions for what they'll like, has a huge cache; this cache uses very large memory-mapped files that are very carefully structured around page boundaries for block allocation, and it's designed, right now, for 4K page sizes. Re-engineering it is being worked on, but ... will take some time. Meanwhile, we sort of thought the 4K rebuild was going to be, you know, easy. Since there was a helpful wiki article on exactly how to do it :)

    -roy
    chuckle.. good to know.. and a good reason to get it fixed..

    continuing to stir the pot with the programmers.. hopefully it's something easy to fix..
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-09T12:01:23Z  
    chuckle.. good to know.. and a good reason to get it fixed..

    continuing to stir the pot with the programmers.. hopefully it's something easy to fix..
    It's been suggested that we disable PPC_SUBPAGE_PROT in the config files.

    Changing that to "n" gets the build past the build break... the builds are running now..
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-09T12:22:54Z  
    It's been suggested that we disable PPC_SUBPAGE_PROT in the config files.

    Changing that to "n" gets the build past the build break... the builds are running now..
    oops.. never mind.. that didn't work..
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-14T18:39:17Z  
    oops.. never mind.. that didn't work..
    Any luck, Bill?

    -roy
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-14T20:59:43Z  
    Any luck, Bill?

    -roy
    Been poking on the developers.. but the guy that made the key change is out...

    The next thought is to take out a 5.3 patch which introduced the subpage protection for the Lx86 product.. which we think wasn't checked carefully with the 4KB path.

    I haven't tried to do that yet.. I'll dig out the patch number we're talking about...
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-21T17:09:12Z  
    Been poking on the developers.. but the guy that made the key change is out...

    The next thought is to take out a 5.3 patch which introduced the subpage protection for the Lx86 product.. which we think wasn't checked carefully with the 4KB path.

    I haven't tried to do that yet.. I'll dig out the patch number we're talking about...
    Any progress in taking out that patch?
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-21T20:52:35Z  
    Any progress in taking out that patch?
    BTW, my user is now requesting we downgrade to RHEL4. That's a workaround, and won't be making anyone any happier, of course. I've got probably about a week before we have to pull the trigger. What are the odds that we can make headway on this in the next week?

    -roy
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-22T14:01:16Z  
    BTW, my user is now requesting we downgrade to RHEL4. That's a workaround, and won't be making anyone any happier, of course. I've got probably about a week before we have to pull the trigger. What are the odds that we can make headway on this in the next week?

    -roy
    I deleted this line from the SPECS/kernel-2.6-spec
    %patch23256 -p1

    and changed the config file to turn off 64K pages.

    The kernel re-built with no errors.

    I have not had a chance to load and reboot..
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-22T16:01:21Z  
    I deleted this line from the SPECS/kernel-2.6-spec
    %patch23256 -p1

    and changed the config file to turn off 64K pages.

    The kernel re-built with no errors.

    I have not had a chance to load and reboot..
    Ah cool. It re-booted.
    Red Hat Enterprise Linux Server release 5.3 (Tikanga)
    Kernel 2.6.18-128.el5.4K on an ppc64

    p6mld32r.ltc.austin.ibm.com login:
    root@p6mld32r ~# getconf PAGESIZE
    4096

    root@p6mld32r ~# /usr/bin/time -v sleep 0 2>&1 | grep Page
    Page size (bytes): 4096
    I will update the wiki page. We've opened a defect to fix the code.. but..
  • SystemAdmin
    SystemAdmin
    706 Posts

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-24T18:19:18Z  
    Ah cool. It re-booted.
    Red Hat Enterprise Linux Server release 5.3 (Tikanga)
    Kernel 2.6.18-128.el5.4K on an ppc64

    p6mld32r.ltc.austin.ibm.com login:
    root@p6mld32r ~# getconf PAGESIZE
    4096

    root@p6mld32r ~# /usr/bin/time -v sleep 0 2>&1 | grep Page
    Page size (bytes): 4096
    I will update the wiki page. We've opened a defect to fix the code.. but..
    I cleaned up and confirmed the steps on the wiki page for RHEL 5.2 and RHEL 5.3.

    Thanks for catching this and reporting it. Let us know if there's anything else needed.