Topic
20 replies Latest Post - ‏2009-07-24T18:19:18Z by SystemAdmin
SystemAdmin
SystemAdmin
706 Posts
ACCEPTED ANSWER

Pinned topic Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

‏2009-07-02T22:51:49Z |
Trying to follow the directions at

http://www.ibm.com/developerworks/wikis/display/LinuxP/Re-building+a+RHEL+5+kernel+for+Power

to create a 4K pagesize kernel on RHEL 5.3, I'm finding the directions refer to a kernel-ppc64.spec, which is missing on my system; further, if I just use kernel-2.6.spec, I get:
...
In file included from arch/powerpc/mm/mem.c:35:
include/asm/pgalloc.h: In function 'pgd_free':
include/asm/pgalloc.h:48: error: implicit declaration of function 'subpage_prot_free'
make[1]: *** http://arch/powerpc/mm/mem.o Error 1

I'm clearly missing something. Suggestions?
Updated on 2009-07-24T18:19:18Z at 2009-07-24T18:19:18Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    706 Posts
    ACCEPTED ANSWER

    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

    ‏2009-07-02T23:52:39Z  in response to SystemAdmin
    hmm.. that's strange.. but I haven't tried this recently on rhel 5.3..

    the missing file should've been installed with the kernel src rpm..

    I'll try it here and see where it got moved..
    • SystemAdmin
      SystemAdmin
      706 Posts
      ACCEPTED ANSWER

      Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

      ‏2009-07-03T01:40:38Z  in response to SystemAdmin
      Well, I followed the instructions..
      Downloaded the src DVD iso.. mounted that..
      installed the kernel src rpm
      and the kernel-2.6.spec file is there.

      Download RHEL5.3-Server-20090106.0-SRPMS-DVD.iso
      It's a very big file. 2,951,993,344 bytes

      1. mount -o loop RHEL5.3-Server-20090106.0-SRPMS-DVD.iso /mnt
      2. cd /mnt
      3. find . -name kernel*
      ./SRPMS/kernel-2.6.118-128.el5.src.rpm
      1. cd SRPMS
      2. rpm -i kernel-2.6.18-128.el5.src.rpm
      3. cd /usr/src/redhat/SPECS/
      4. ls
      kernel-2.6.spec

      I'll update the wiki page with the RHEL 5.3 quick summary...

      Let me know if something doesn't work with these steps...
      • SystemAdmin
        SystemAdmin
        706 Posts
        ACCEPTED ANSWER

        Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

        ‏2009-07-03T14:49:08Z  in response to SystemAdmin
        kernel-2.6.spec is, in fact, there. That's what's used for the first example, where we rebuild the kernel without any modifications.

        When we switch to building a 4K kernel, the file used is kernel-ppc64.spec; that file does not appear to be there. Browsing rpmfind.com suggests that it may only be included as part of SLES 10.

        Using kernel-2.6.spec for the rpmbuild results in the compile failure I quoted in my original post.

        -roy
        • SystemAdmin
          SystemAdmin
          706 Posts
          ACCEPTED ANSWER

          Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

          ‏2009-07-03T15:53:41Z  in response to SystemAdmin
          ahh... I think I see what you mean.. in the 4K steps section, a SPECS/kernel-ppc64.spec file is specified as the last step.

          
          rpmbuild -ba --target ppc64 --without kabichk SPECS/kernel-ppc64.spec
          


          I wonder if that's created in the preceeding step where we try to be sure just the basic kernel can be built and booted.
          Interesting.. I'll have to snag a system sometime this weekend and try.
          • SystemAdmin
            SystemAdmin
            706 Posts
            ACCEPTED ANSWER

            Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

            ‏2009-07-03T16:11:53Z  in response to SystemAdmin
            ah. even a basic build won't work...

            will check...
            • SystemAdmin
              SystemAdmin
              706 Posts
              ACCEPTED ANSWER

              Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

              ‏2009-07-06T22:46:29Z  in response to SystemAdmin
              Various discussions continue and haven't panned out. Opening a Bugzilla to track.

              We think the "kernel-ppc64.spec" is a typo as we mixed and matched SPECS and CONFIG files. Will test a RHEL 5.2 build tomorrow to confirm.

              I updated the wiki page to point out there's a build break as currently documented.
              • SystemAdmin
                SystemAdmin
                706 Posts
                ACCEPTED ANSWER

                Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                ‏2009-07-07T15:11:19Z  in response to SystemAdmin
                Strange. We are finding that the steps work fine on a number of other Power systems we're playing with, and have not yet determined where/what causes the build break.

                We setup a fresh RHEL 5.3 on a 4-core Power 5 system, installed the kernel-2.6.18-128.el5.src.rpm, and ran rpmbuild without any changes to see if the kernel would build as-is.

                rpmbuild -ba --target ppc64 SPECS/kernel-2.6.spec

                and it built fine.

                We also put a fresh RHEL 5.3 on a 4-core Power 6 system, installed the src.rpm, and re-built the kernel. No problems.

                The system where we saw the same build break was an 8-core system, and we had installed a bunch of other kernel* files... so we need to try that next. It's hard to imagine that more cores would cause a build break, so the thinking is the extra files installed is causing a conflict.

                For kernel files, we just have these installed on the systems where the build is working..

                1. rpm -qa | grep kernel
                kernel-devel-2.6.18-128.el5
                kernel-headers-2.6.18-128.el5
                kernel-2.6.18-128.el5

                For the system which is seeing the build break...

                1. rpm -qa |grep kernel
                kernel-2.6.18-128.el5
                kernel-headers-2.6.18-128.el5
                kernel-debuginfo-2.6.18-128.el5
                kernel-debuginfo-common-2.6.18-128.el5
                kernel-debug-2.6.18-128.el5
                • SystemAdmin
                  SystemAdmin
                  706 Posts
                  ACCEPTED ANSWER

                  Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                  ‏2009-07-07T22:50:44Z  in response to SystemAdmin
                  I suspect the different packages may be a red herring. On the system here, here's all I've got:

                  % rpm -q --queryformat "%{NAME}-%{VERSION}.%{ARCH}\n" -a |grep -i kernel
                  kernel-headers-2.6.18.ppc64
                  kernel-2.6.18.ppc64

                  Compiling the stock kernel still works fine; compiling the 4K kernel doesn't.

                  -roy
                  • SystemAdmin
                    SystemAdmin
                    706 Posts
                    ACCEPTED ANSWER

                    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                    ‏2009-07-08T12:51:11Z  in response to SystemAdmin
                    Ok. Will ask the developers to look at what got broken with the 4KB path.

                    This isn't normally built or supported, so it doesn't get checked in a normal release cycle.

                    Is there a particular reason you're trying the 4KB pages?
                    • SystemAdmin
                      SystemAdmin
                      706 Posts
                      ACCEPTED ANSWER

                      Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                      ‏2009-07-08T18:11:07Z  in response to SystemAdmin
                      Our match engine, which holds information like user ratings and predictions for what they'll like, has a huge cache; this cache uses very large memory-mapped files that are very carefully structured around page boundaries for block allocation, and it's designed, right now, for 4K page sizes. Re-engineering it is being worked on, but ... will take some time. Meanwhile, we sort of thought the 4K rebuild was going to be, you know, easy. Since there was a helpful wiki article on exactly how to do it :)

                      -roy
                      • SystemAdmin
                        SystemAdmin
                        706 Posts
                        ACCEPTED ANSWER

                        Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                        ‏2009-07-08T19:50:32Z  in response to SystemAdmin
                        chuckle.. good to know.. and a good reason to get it fixed..

                        continuing to stir the pot with the programmers.. hopefully it's something easy to fix..
                        • SystemAdmin
                          SystemAdmin
                          706 Posts
                          ACCEPTED ANSWER

                          Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                          ‏2009-07-09T12:01:23Z  in response to SystemAdmin
                          It's been suggested that we disable PPC_SUBPAGE_PROT in the config files.

                          Changing that to "n" gets the build past the build break... the builds are running now..
                          • SystemAdmin
                            SystemAdmin
                            706 Posts
                            ACCEPTED ANSWER

                            Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                            ‏2009-07-09T12:22:54Z  in response to SystemAdmin
                            oops.. never mind.. that didn't work..
                            • SystemAdmin
                              SystemAdmin
                              706 Posts
                              ACCEPTED ANSWER

                              Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                              ‏2009-07-14T18:39:17Z  in response to SystemAdmin
                              Any luck, Bill?

                              -roy
                              • SystemAdmin
                                SystemAdmin
                                706 Posts
                                ACCEPTED ANSWER

                                Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                                ‏2009-07-14T20:59:43Z  in response to SystemAdmin
                                Been poking on the developers.. but the guy that made the key change is out...

                                The next thought is to take out a 5.3 patch which introduced the subpage protection for the Lx86 product.. which we think wasn't checked carefully with the 4KB path.

                                I haven't tried to do that yet.. I'll dig out the patch number we're talking about...
                                • SystemAdmin
                                  SystemAdmin
                                  706 Posts
                                  ACCEPTED ANSWER

                                  Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                                  ‏2009-07-21T17:09:12Z  in response to SystemAdmin
                                  Any progress in taking out that patch?
                                  • SystemAdmin
                                    SystemAdmin
                                    706 Posts
                                    ACCEPTED ANSWER

                                    Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                                    ‏2009-07-21T20:52:35Z  in response to SystemAdmin
                                    BTW, my user is now requesting we downgrade to RHEL4. That's a workaround, and won't be making anyone any happier, of course. I've got probably about a week before we have to pull the trigger. What are the odds that we can make headway on this in the next week?

                                    -roy
                                    • SystemAdmin
                                      SystemAdmin
                                      706 Posts
                                      ACCEPTED ANSWER

                                      Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                                      ‏2009-07-22T14:01:16Z  in response to SystemAdmin
                                      I deleted this line from the SPECS/kernel-2.6-spec
                                      %patch23256 -p1

                                      and changed the config file to turn off 64K pages.

                                      The kernel re-built with no errors.

                                      I have not had a chance to load and reboot..
                                      • SystemAdmin
                                        SystemAdmin
                                        706 Posts
                                        ACCEPTED ANSWER

                                        Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                                        ‏2009-07-22T16:01:21Z  in response to SystemAdmin
                                        Ah cool. It re-booted.
                                        Red Hat Enterprise Linux Server release 5.3 (Tikanga)
                                        Kernel 2.6.18-128.el5.4K on an ppc64

                                        p6mld32r.ltc.austin.ibm.com login:
                                        root@p6mld32r ~# getconf PAGESIZE
                                        4096

                                        root@p6mld32r ~# /usr/bin/time -v sleep 0 2>&1 | grep Page
                                        Page size (bytes): 4096
                                        I will update the wiki page. We've opened a defect to fix the code.. but..
                                        • SystemAdmin
                                          SystemAdmin
                                          706 Posts
                                          ACCEPTED ANSWER

                                          Re: Trouble Compiling New Kernel for Redhat 5.3 on ppc64Trouble

                                          ‏2009-07-24T18:19:18Z  in response to SystemAdmin
                                          I cleaned up and confirmed the steps on the wiki page for RHEL 5.2 and RHEL 5.3.

                                          Thanks for catching this and reporting it. Let us know if there's anything else needed.