Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
21 replies Latest Post - ‏2014-06-11T08:18:35Z by Niraj Kumar Mishra
Niraj Kumar Mishra
Niraj Kumar Mishra
16 Posts
ACCEPTED ANSWER

Pinned topic RHEL Patching Issue.

‏2014-05-15T13:11:01Z |

Dear All,

 

Yesterday We build two relays on RHEL 5 X86_64 bit machine. Today We are applying all relevant patches on these relay machines, but I patches are showing failed status with (Exit code=1). I have checked log and EDRDeployment log. Same exit code=1 I found in log as well. Can You please advise us that why all patches returning the exit code=1 in logs.

Log file message:

 

Line 767:    Command succeeded (Exit Code=1) wait /bin/bash "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/InstallPackages.sh" -f "2012013602" -m "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_PackageMapping_2012013602" -r "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumResolveOutput_2012013602" -g "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumConfig_2012013602" -e "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools" -l "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/../../EDRDeployData/" "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_1.rpm" "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_2.rpm" "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_3.rpm" "/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/__Download/2012013602_4.rpm"         (group:271312,action:271313)

 

EDRDeployment log:

Install Failure: yum -C -y --noplugins localinstall -c '/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumConfig_2012013602' - Error:

 

Thanks

Niraj

  • Niraj Kumar Mishra
    Niraj Kumar Mishra
    16 Posts
    ACCEPTED ANSWER

    Re: RHEL Patching Issue.

    ‏2014-05-19T07:08:49Z  in response to Niraj Kumar Mishra

    Hi All,

     

    Please help me regarding the same.

     

    Thanks

    Niraj

    • Niraj Kumar Mishra
      Niraj Kumar Mishra
      16 Posts
      ACCEPTED ANSWER

      Re: RHEL Patching Issue.

      ‏2014-05-26T11:49:12Z  in response to Niraj Kumar Mishra

      Hi Ben,

       

      Can you please help me, I would be really thankful to you.

       

      Thanks

      Niraj

      • cglimson
        cglimson
        19 Posts
        ACCEPTED ANSWER

        Re: RHEL Patching Issue.

        ‏2014-05-28T09:18:37Z  in response to Niraj Kumar Mishra

        Hi Niraj,

        The Patch RHEL team is looking into this. Apologies for not acknowledging earlier.

        Regards,

        Malaine

         

        • Niraj Kumar Mishra
          Niraj Kumar Mishra
          16 Posts
          ACCEPTED ANSWER

          Re: RHEL Patching Issue.

          ‏2014-05-28T14:39:40Z  in response to cglimson

          Hi Malaine,

           

          I am really thankful to you for your kind response. I am looking forward for your help.

           

          Thanks a lot !!!

          Warm Regards

          Niraj

          • Preeya
            Preeya
            8 Posts
            ACCEPTED ANSWER

            Re: RHEL Patching Issue.

            ‏2014-06-04T08:14:23Z  in response to Niraj Kumar Mishra

            Hi Niraj,

               On the relay endpoint, could you check the location " /var/opt/BESRelay/wwwrootbes/bfmirr/downloads/sha1" to verify that the downloads are available on the relay. Also, would it be possible to show a few more log entries (before the failure is actually reported) for EDR_DeploymentResults.txt , and the client logs?

            Regards,

            Priya
             

             

            • Niraj Kumar Mishra
              Niraj Kumar Mishra
              16 Posts
              ACCEPTED ANSWER

              Re: RHEL Patching Issue.

              ‏2014-06-04T09:13:11Z  in response to Preeya

              Hi Priya,

              Thanks for your nice reply. I am giving you the requested logs.Please have a look and advice.

               

              Thanks

              Niraj

              Attachments

              • Raja9109
                Raja9109
                5 Posts
                ACCEPTED ANSWER

                Re: RHEL Patching Issue.

                ‏2014-06-04T10:54:20Z  in response to Niraj Kumar Mishra

                Try this fix - https://access.redhat.com/site/solutions/31543

              • Preeya
                Preeya
                8 Posts
                ACCEPTED ANSWER

                Re: RHEL Patching Issue.

                ‏2014-06-05T02:48:44Z  in response to Niraj Kumar Mishra

                Hi Niraj,

                  The EDR log shows these statements -

                ====================================================================================================
                Public key for 2012013602_1.rpm is not installed
                [Wed May 14 11:20:33 EDT 2014] 2012013602 ____ Failed to install the following packages:
                [Wed May 14 11:20:33 EDT 2014] 2012013602 ____  libvorbis-1.1.2-3.el5_7.6.x86_64.rpm libvorbis-devel-1.1.2-3.el5_7.6.x86_64.rpm libvorbis-1.1.2-3.el5_7.6.i386.rpm libvorbis-devel-1.1.2-3.el5_7.6.i386.rpm
                [Wed May 14 11:21:12 EDT 2014] 2012014602 Install Failure: yum -C -y --noplugins localinstall -c '/var/opt/BESClient/__BESData/Patches for RHEL5 Native Tools/EDR_YumConfig_2012014602' - Error:

                ====================================================================================================

                The 1st line : "Public key for 2012013602_1.rpm is not installed" could be a reason fixlet action fails.

                The IBM product documentation / IEM dev works explains how to resolve this in the FAQ section -

                1) http://pic.dhe.ibm.com/infocenter/tivihelp/v26r1/topic/com.ibm.tem.doc_9.0/Patch_Man/Patch_Man_RH/c_frequently_asked_questions.html

                2) https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli%20Endpoint%20Manager/page/Patch%20Management%20-%20RedHat%20Enterprise%20Linux%20-%20FAQ

                ~ Priya

                • Niraj Kumar Mishra
                  Niraj Kumar Mishra
                  16 Posts
                  ACCEPTED ANSWER

                  Re: RHEL Patching Issue.

                  ‏2014-06-05T08:38:51Z  in response to Preeya

                  Hi Priya,

                  Excellent....This could be be the reason. I really appreciate your kind help.I will try out this and let you know. I am 100% sure that issue will be resolved.

                   

                  Thanks

                  Niraj

                  • Niraj Kumar Mishra
                    Niraj Kumar Mishra
                    16 Posts
                    ACCEPTED ANSWER

                    Re: RHEL Patching Issue.

                    ‏2014-06-05T10:15:52Z  in response to Niraj Kumar Mishra

                    Hi Priya,

                     

                    I am extremely sorry to ask about another RHEL Patching issue in the same thread. But if you could help me, I would be really thankful to you.

                    Problem description: during the patching of RHEL server, I am getting below message in client log file.

                    Execute prefetch plug-in taking too long to execute. I went through the below link to rectify the issue.

                    http://www-01.ibm.com/support/docview.wss?uid=swg21663775

                    I increased the _BESClient_ActionManager_PrefetchPlugInTimeoutSecondssettin to 300seconds .

                    But still patching are failed on RHEL.

                    Thanks in advance.

                     

                    Thanks

                    Niraj

                    Attachments

                    • Preeya
                      Preeya
                      8 Posts
                      ACCEPTED ANSWER

                      Re: RHEL Patching Issue.

                      ‏2014-06-06T07:50:58Z  in response to Niraj Kumar Mishra

                      Based on the current logs, could you point me the failed fixlets? Could you resend the logs with the failurers? Also did you restart the client after you did the Timeout setting ? There's a line in the log file "execute prefetch plug-in' didn't complete within 60 seconds.  Black listing plug-ins matching the sha1 hash of 'name of 'bash' until agent is restarted"

                       You would need to restart your BESClient before you start any new fixlet execution.

                      ~Priya


                       

                      • Niraj Kumar Mishra
                        Niraj Kumar Mishra
                        16 Posts
                        ACCEPTED ANSWER

                        Re: RHEL Patching Issue.

                        ‏2014-06-06T09:50:28Z  in response to Preeya

                        Hi Priya,

                        Yes, after doing client timeout setting we restarted the besclient. I will share the recent logs from the failing machine that I had executed yesterday. For logs I need to ask linux sysadmin to share.

                        Mean time I would like to share the action info of one the failing fixlet:

                         

                        // Note: The target packages can be found at https://rhn.redhat.com/rhn/errata/details/Packages.do?eid=23518. The dependencies for the target packages will not be listed on that page.
                          // Note: SHA1 and SHA256 are not the same values.
                          begin prefetch block
                          //Action Properties
                          parameter "sitefolder" = "{client folder of site whose (name of it as lowercase starts with "patches for rhel5 native tools")}"
                          parameter "cwd" = "{client folder of the current site}"
                          parameter "fixletid" = "{origin fixlet id of action as string}"
                          parameter "yumVersionOld" = "{if (exists package "yum" whose (rpm version record of it >= rpm package version record "3.2.22-20") of rpm) then "" else "-l"}"
                          //File Locations
                          parameter "EDR_RepodataManifest" = "{parameter "sitefolder"}/EDR_RepodataManifest"
                          parameter "EDR_YumConfig" = "{parameter "cwd"}/EDR_YumConfig_{parameter "fixletid"}"
                          parameter "EDR_YumRepos" = "{parameter "cwd"}/EDR_YumRepos_{parameter "fixletid"}"
                          parameter "EDR_YumFilelists" = "{parameter "cwd"}/EDR_YumFilelists_{parameter "fixletid"}"
                          parameter "EDR_RepodataSpec" = "{parameter "cwd"}/EDR_RepodataSpec_{parameter "fixletid"}"
                          parameter "EDR_PackageSpec" = "{parameter "cwd"}/EDR_PackageSpec_{parameter "fixletid"}"
                          parameter "EDR_PackageMapping" = "{parameter "cwd"}/EDR_PackageMapping_{parameter "fixletid"}"
                          //Log Locations
                          parameter "EDR_DeployDataDir" = "{parameter "cwd"}/../../EDRDeployData/"
                          parameter "EDR_YumResolveOutput" = "{parameter "cwd"}/EDR_YumResolveOutput_{parameter "fixletid"}"
                          parameter "t0" = "{if (exists package "file" whose (rpm version record of it < rpm package version record "4.17-28" AND architecture of it = "x86_64") of rpm AND NOT exists package "file" whose (rpm version record of it >= rpm package version record "4.17-28" AND architecture of it = "x86_64") of rpm) then "file-4.17-28.x86_64" else ""}"
                          parameter "packages" = "{parameter "t0"}"
                          if {not exists setting "_BESClient_RHEL_AllowYumDownloads" whose (value of it = "1") of client}
                        Failed execute prefetch plug-in "/bin/bash" "{parameter "sitefolder"}/SelectRepoFiles.sh" -m "{parameter "EDR_RepodataManifest"}" -s "{parameter "EDR_RepodataSpec"}" -r "{parameter "EDR_YumRepos"}"
                          add prefetch item {concatenation ";" of lines of file (parameter "EDR_RepodataSpec")}
                          collect prefetch items
                          execute prefetch plug-in "/bin/bash" "{parameter "sitefolder"}/CreateYumConfig.sh" -d "{parameter "cwd"}" -f "{download folder}" -p "{parameter "sitefolder"}" -r "{parameter "EDR_YumRepos"}" -c "{parameter "EDR_YumConfig"}" {parameter "yumVersionOld"}
                          execute prefetch plug-in "/bin/bash" "{parameter "sitefolder"}/ResolveDependencies.sh" -f "{parameter "EDR_YumFilelists"}" -y "{parameter "EDR_YumResolveOutput"}" -o "{parameter "EDR_PackageSpec"}" -p "{parameter "fixletid"}" -m "{parameter "EDR_PackageMapping"}" {parameter "yumVersionOld"} -c "{parameter "EDR_YumConfig"}" "{parameter "packages"}"
                          if { (number of lines of file (parameter "EDR_YumFilelists") | 0 ) > 0 }
                          execute prefetch plug-in "/bin/bash" "{parameter "sitefolder"}/SelectRepoFiles.sh" -d "filelists" -m "{parameter "EDR_RepodataManifest"}" -s "{parameter "EDR_RepodataSpec"}" -r "{parameter "EDR_YumRepos"}"
                          add prefetch item {concatenation ";" of lines of file (parameter "EDR_RepodataSpec")}
                          collect prefetch items
                          execute prefetch plug-in "/bin/bash" "{parameter "sitefolder"}/CreateYumConfig.sh" -d "{parameter "cwd"}" -f "{download folder}" -p "{parameter "sitefolder"}" -r "{parameter "EDR_YumRepos"}" -c "{parameter "EDR_YumConfig"}" -x
                          execute prefetch plug-in "/bin/bash" "{parameter "sitefolder"}/ResolveDependencies.sh" -f "{parameter "EDR_YumFilelists"}" -y "{parameter "EDR_YumResolveOutput"}" -o "{parameter "EDR_PackageSpec"}" -p "{parameter "fixletid"}" -m "{parameter "EDR_PackageMapping"}" {parameter "yumVersionOld"} -c "{parameter "EDR_YumConfig"}" "{parameter "packages"}"
                          endif
                          add prefetch item {if (exists (file (parameter "EDR_PackageSpec")) whose (number of lines of it > 0)) then concatenation " ; " of lines of file (parameter "EDR_PackageSpec") else ""}
                          endif
                          end prefetch block
                          parameter "EDR_PackageList" = "{if (exists (file (parameter "EDR_PackageMapping")) whose (number of lines of it > 0)) then concatenation " " of (("%22" & download path (preceding text of first " " of it) & "%22") of lines of file (parameter "EDR_PackageMapping")) else (parameter "packages")}"
                          wait /bin/bash "{folder (parameter "sitefolder")}/InstallPackages.sh" -f "{parameter "fixletid"}" -m "{parameter "EDR_PackageMapping"}" -r "{parameter "EDR_YumResolveOutput"}" -g "{parameter "EDR_YumConfig"}" -e "{parameter "cwd"}" -l "{parameter "EDR_DeployDataDir"}" {parameter "EDR_PackageList"}
                          //Cleanup
                          delete "{parameter "EDR_YumConfig"}"
                          delete "{parameter "EDR_YumRepos"}"
                          delete "{parameter "EDR_RepodataSpec"}"
                          delete "{parameter "EDR_PackageSpec"}"
                          delete "{parameter "EDR_PackageMapping"}"
                          delete "{parameter "EDR_YumResolveOutput"}"
                          delete "{parameter "EDR_YumFilelists"}"

                        Once again thank you so much for your great support

                         

                        Thanks

                        Niraj

                        • Niraj Kumar Mishra
                          Niraj Kumar Mishra
                          16 Posts
                          ACCEPTED ANSWER

                          Re: RHEL Patching Issue.

                          ‏2014-06-06T14:42:41Z  in response to Niraj Kumar Mishra

                          Hi Priya,

                          I am attaching logs for the two failing machines. If you could please look into these logs and advise me, I would be really thankful to you.

                           

                          Thanks

                          Niraj

                          Attachments

                          • Niraj Kumar Mishra
                            Niraj Kumar Mishra
                            16 Posts
                            ACCEPTED ANSWER

                            Re: RHEL Patching Issue.

                            ‏2014-06-09T09:43:47Z  in response to Niraj Kumar Mishra

                            Hi Priya,

                             

                            Can you please help me.

                             

                            Thanks

                            Niraj

                            • Preeya
                              Preeya
                              8 Posts
                              ACCEPTED ANSWER

                              Re: RHEL Patching Issue.

                              ‏2014-06-10T08:33:43Z  in response to Niraj Kumar Mishra

                              Hi Niraj,

                                The setting to be applied for the endpoint is "_BESClient_ActionManager_PrefetchPlugInTimeoutSeconds" , I see that both your clients have the field name set as "_BESClient_ActionManager_PrefetchPlugInTimeoutSecondssetting".

                              Change the name and try running the fixlets again.

                               

                              Thanks, Priya

                              • Niraj Kumar Mishra
                                Niraj Kumar Mishra
                                16 Posts
                                ACCEPTED ANSWER

                                Re: RHEL Patching Issue.

                                ‏2014-06-10T14:12:59Z  in response to Preeya

                                Hi Priya,

                                I am sorry but as per below IBM technote setting name mentioned like "_BESClient_ActionManager_PrefetchPlugInTimeoutSecondssetting".

                                http://www-01.ibm.com/support/docview.wss?uid=swg21663775

                                I am going to change the setting name as per your instruction and try to apply the baseline again and will let you know the result.

                                Thanks

                                Niraj

                                 

                                 

                                • Niraj Kumar Mishra
                                  Niraj Kumar Mishra
                                  16 Posts
                                  ACCEPTED ANSWER

                                  Re: RHEL Patching Issue.

                                  ‏2014-06-10T15:37:07Z  in response to Niraj Kumar Mishra

                                  Hi Priya,

                                   

                                  Still some of the patches are failing.. I will share the recent log again

                                   

                                  Thanks a lot.

                                   

                                  Thanks

                                  Niraj

                                  • Niraj Kumar Mishra
                                    Niraj Kumar Mishra
                                    16 Posts
                                    ACCEPTED ANSWER

                                    Re: RHEL Patching Issue.

                                    ‏2014-06-10T16:41:45Z  in response to Niraj Kumar Mishra

                                    Hi Priya,

                                     

                                    As per your suggestion, I did the changes and try to apply the baseline on failing machines. This time some of the patches got applied but still most of them are failing.

                                    I am giving you the log info for one of the failing fixlet.

                                     

                                    Line 582:    Relevant - RHBA-2013:0066 - Vim Bug Fix Update - Red Hat Enterprise 5.0 (x86_64) (fixlet:307796)

                                                    Line 1894:    ItemizedDownloadsAvailable: true (action id 307796)

                                                    Line 1895:    ActionLogMessage: (group:307323,action:307796) Non-Distributed - DownloadsAvailable

                                                    Line 1896:    ActionLogMessage: (group:307323,action:307796) Submitting download request

                                                    Line 1897:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-enhanced-7.0.109-7.2.el5.x86_64.rpm'

                                                    Line 1898:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-minimal-7.0.109-7.2.el5.x86_64.rpm'

                                                    Line 1899:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-common-7.0.109-7.2.el5.x86_64.rpm'

                                                    Line 1908:    ActionLogMessage: (group:307323,action:307796) Execute prefetch plug-in taking too long to execute.

                                                    Line 1911:    ActionLogMessage: (group:307323,action:307796) ending sub action

                                                    Line 2297:    Not Relevant - RHBA-2013:0066 - Vim Bug Fix Update - Red Hat Enterprise 5.0 (x86_64) (fixlet:307796)

                                     

                                    Also I am attaching the log file as well. Can you please look into this and advise me.

                                     

                                    Thanks and Regards

                                    Niraj

                                    Attachments

                                    • Preeya
                                      Preeya
                                      8 Posts
                                      ACCEPTED ANSWER

                                      Re: RHEL Patching Issue.

                                      ‏2014-06-11T03:12:10Z  in response to Niraj Kumar Mishra

                                      Hi Niraj,

                                         Going through the logs, I didn't see any failed fixlets. Fixlets in your log file are either fixed or non relevant. The EDR log doesn't show any failed fixlets either. Which are the failed fixlets that you see?

                                        Also for the baseline that you run - are these fixlets still relevant? Check the attachment : Non-Relevant fixlets.

                                       

                                      Thanks,

                                      Priya

                                      Attachments

                                      • Niraj Kumar Mishra
                                        Niraj Kumar Mishra
                                        16 Posts
                                        ACCEPTED ANSWER

                                        Re: RHEL Patching Issue.

                                        ‏2014-06-11T08:18:35Z  in response to Preeya

                                        Hi Priya,

                                        Thanks for your quick reply. Hope you are doing good. Above text messages provided by me from log...

                                        I have given an example above for one of the failed fixlet in the baseline action info. In client log...for failed fixlet it is showing....

                                        Line 582:    Relevant - RHBA-2013:0066 - Vim Bug Fix Update - Red Hat Enterprise 5.0 (x86_64) (fixlet:307796)

                                                        Line 1894:    ItemizedDownloadsAvailable: true (action id 307796)

                                                        Line 1895:    ActionLogMessage: (group:307323,action:307796) Non-Distributed - DownloadsAvailable

                                                        Line 1896:    ActionLogMessage: (group:307323,action:307796) Submitting download request

                                                        Line 1897:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-enhanced-7.0.109-7.2.el5.x86_64.rpm'

                                                        Line 1898:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-minimal-7.0.109-7.2.el5.x86_64.rpm'

                                                        Line 1899:    ActionLogMessage: (group:307323,action:307796) Download url: 'RedHatProtocol://vim-common-7.0.109-7.2.el5.x86_64.rpm'

                                                        Line 1908:    ActionLogMessage: (group:307323,action:307796) Execute prefetch plug-in taking too long to execute.

                                                        Line 1911:    ActionLogMessage: (group:307323,action:307796) ending sub action

                                                        Line 2297:    Not Relevant - RHBA-2013:0066 - Vim Bug Fix Update - Red Hat Enterprise 5.0 (x86_64) (fixlet:307796)

                                        I am attaching one snapshot for error emssage.

                                         

                                        Thanks

                                        Niraj

                • cglimson
                  cglimson
                  19 Posts
                  ACCEPTED ANSWER

                  Re: RHEL Patching Issue.

                  ‏2014-06-05T09:49:31Z  in response to Preeya

                  Thanks, Priya!