Installing and Updating sos report on RHEL7, 8 and 9 for IBM Power Servers: Importance and Procedures

Steps

• On the Disconnected System:

  • The public General Availability (GA) version of the `sos report` can be used in production and is available for download via Red Hat Product Errata. In Red Hat Customer Portal search for `sos`, select the latest Product Errata:
    https://access.redhat.com/errata-search/?q=sos&p=1&sort=portal_publication_date+desc&rows=10
    
  • Ensure you are logged in with your Red Hat subscription account and then navigate to 'Update Packages'. Make sure to select the appropriate `sos report` version that corresponds with the Red Hat Enterprise Linux (RHEL) version you are using (such as RHEL7, 8, or 9), and download the `sos` RPM. If the download button is not visible, verify that your subscription permits you to download packages. In the example below, we are downloading the `sos` RPM for RHEL8. The 'noarch' version is compatible with IBM Power Servers, as well as other architectures such as x86, IBM z Systems, and more.
    
  • Transfer the RPM file to the system either via an FTP server or from a USB/DVD drive. For additional details on how to set up yum/dnf repository for locally-mounted DVD, refer to section 3.1.3, “Customizing the Linux operating system” on page 78 in the IBM PowerVC Version 2.0 Redbook.
  • Install the package by using the local RPM install command:
    RHEL8 and 9

    sudo dnf install /path/to/rpm

    REHL7

    sudo yum localinstall /path/to/rpm

    (Replace /path/to/rpm with the actual path to the RPM file).
  • The RPM command can be utilized to install and update the `sos report` package on any version of RHEL.
    To install:

    rpm -ivh /path/to/sos-<version>.noarch.rpm

    To install and update:

    rpm -Uvh /path/to/sos-<version>.noarch.rpm

    (Replace /path/to/ with the actual path to the `sos` RPM file).

Part 2: Installing SOS Report on an Air-Gapped System

1. Preparing the Package Repository:

   • On a system with internet access, create a local repository containing the `sos` package and its dependencies. This can be done by mirroring the required repositories or manually downloading the necessary RPMs.

2. Transferring the Repository:

   • Copy the entire local repository to a FTP server or USB drive or other removable media.

3. Installing from the Local or cloned Repository :

   • Connect the USB/DVD drive to the air-gapped system.
   • Mount the USB/DVD drive and add the repository to the YUM configuration. This typically involves creating a .repo file in /etc/yum.repos.d/ pointing to the repository on the USB/DVD drive or FTP server.
   • Install the SOS report by using YUM:
     RHEL8 and 9

     sudo dnf install sos

     RHEL7

     sudo yum install sos

• Before uninstalling, it's a good practice to verify that the `sos report` tool is indeed installed on your system. Execute the following command:

  sudo dnf list installed sos

  (This command will list the SOS package along with its version if it is installed.)
  or use RPM command on any version of RHEL (7,8,9):

  [root@sos ~]# rpm -qa sos
  sos-4.6.0-2.el8.noarch
  

• To remove the `sos report` package on any version of RHEL, use the following command:

  for i in $(rpm -qa | grep sos); do rpm -e --nodeps ${i}; done

  Note: We will use the RPM command to uninstall only the sos report tool, leaving its dependencies intact. Using `dnf remove` on RHEL 8 and 9 would uninstall the `sos report` along with all its dependencies. Keep in mind that removing dependencies could impact other packages installed on your RHEL server, potentially leading to system instability or functionality issues.
   

• To use `podman` and pull the support-tools container image, first log in to the registry at registry.redhat.io

  [root@sos ~]# podman login registry.redhat.io
  Username: borislav.ivanov.stoymirski
  Password:
  Login Succeeded!

• To successfully install and execute the support-tools container image on IBM Power Servers running RHEL 7.9, and to efficiently collect an sos report archive from the host within the container image, the following command should be run:

  podman container runlabel RUN registry.redhat.io/rhel8/support-tools
  

  (This command will install and RUN the SOS package on RHEL 7.9)

  Trying to pull registry.redhat.io/rhel8/support-tools:latest...
  Getting image source signatures
  Checking if image destination supports signatures
  Copying blob a7dde6f953b3 done
  Copying blob 068f2e6b47b2 done
  Copying config e8aa68d02c done
  Writing manifest to image destination
  Storing signatures
  
  

  Note:
  If a message, "Error: creating container storage: the container name "support-tools" is already in use by 828bd4de6a68cc4dbc1cc41c43cd2cba9920f84d29373496fa4f1aa357faf400. You have to remove that container to be able to reuse that name: that name is already in use

  
  Error: `/proc/self/exe run -it --name support-tools --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=support-tools -e IMAGE=registry.redhat.io/rhel8/support-tools:latest -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host registry.redhat.io/rhel8/support-tools:latest` failed: exit status 125" , is displayed on the terminal after running the 'podman' command, type `sudo podman rm support-tools` to remove the `support-tools` container and rerun the 'podman' command to avoid issues with `sos report`.

  [root@Sos ~]# sudo podman rm support-tools
  support-tools
  

  To verify that you are connected and operating within a support-tools container, you can view environment variable by running:

  [root@sos /]# env | grep -i image
  IMAGE=registry.redhat.io/rhel8/support-tools:latest
  

  
  Next, generate a new `sosreport` archive by running the `sos report` command:

  [root@sos /]# sos report
  
  sosreport (version 4.6.0)
  
  This command will collect diagnostic and configuration information from
  this Red Hat Enterprise Linux system and installed applications.
  
  An archive containing the collected information will be generated in
  /host/var/tmp/sos.s_2own11 and may be provided to a Red Hat support
  representative.
  
  Any information provided to Red Hat will be treated in accordance with
  the published support policies at:
  
          Distribution Website : https://www.redhat.com/
          Commercial Support   : https://access.redhat.com/
  
  Creating compressed archive...
  
  Your sosreport has been generated and saved in:
          /host/var/tmp/sosreport-sos-podmansos-2023-11-25-kyarksu.tar.xz
  
   Size   21.77MiB
   Owner  root
   sha256 7e93464ca5246991cbe3cf098d8ef3ba3d018c133152c31bf81a6e32308b1e4b
  
  Please send this file to your support representative.
  

  Press ENTER to continue, or CTRL+C to quit.
  When instructed to “enter the case ID that you are generating this report for []”, type: <IBM or Red Hat case number >
  A file called "sosreport-nodename-currentdate.tar.xz" is created in “/host/var/tmp/sosreport-nodename-currentdate.tar.xz“

  • Exit the `support-tools` container by using `Ctrl+C` in the Terminal or detach from the container without stopping it with  `Ctrl+P` followed by ` Ctrl+Q`.
  • To see all containers, including those that have exited or stopped, use and copy the <Container ID>:

    [root@sos ~]# podman ps -a
    CONTAINER ID  IMAGE                                          COMMAND        CREATED         STATUS                    PORTS       NAMES
    145fb3e77c4f  registry.redhat.io/rhel8/support-tools:latest  /usr/bin/bash  10 minutes ago  Exited (0) 9 minutes ago              support-tools
    

  • Use podman cp to Copy the File: The podman cp command allows you to copy files between a container and the local filesystem. To copy a file from a container to the host, use:

    podman cp <container_id>:/path/in/container /path/on/host

    For instance, if our <container_id> is 145fb3e77c4f, the following command can be used to copy the `sosreport` archive to the target host:

    [root@sos ~]# podman cp 145fb3e77c4f:/host/var/tmp/sosreport-sos-podmansos-2023-11-25-kyarksu.tar.xz /root/sosreport-sos-podmansos-2023-11-25-kyarksu.tar.xz

  • To return to a container that is still running, you can use the podman attach command if you want to attach to a container's STDIN, STDOUT, and STDERR, or podman exec to start a new process inside a running container. Here's how you can do it:

    1. Using podman attach:

       • If you want to reattach to the container's main process (for instance, if it's a shell), use:

         podman attach <container_id_or_name>

         (This command will reconnect you to the terminal session you detached from if the container is still running.)

    2. Using podman exec:

       • If you want to execute a new command or open a new shell inside a running container, use:

         podman exec -it <container_id_or_name> /bin/bash
         
       • For instance, if our <container_id> is 145fb3e77c4f, the following command can be used to execute a new command or open a new shell:

         [root@sos ~]# podman ps -a
         CONTAINER ID  IMAGE                                          COMMAND        CREATED         STATUS                    PORTS       NAMES
         145fb3e77c4f  registry.redhat.io/rhel8/support-tools:latest  /usr/bin/bash  10 minutes ago  Exited (0) 9 minutes ago              support-tools
         
         [root@sos ~]# podman exec -it 145fb3e77c4f /bin/bash

       • Here, /bin/bash is an example. It opens a bash shell inside the container. You can replace /bin/bash with any command you wish to run inside the container.
       • The -it flag is used for interactive processes, like a shell.

  Part 4: Transfer the log data to existing IBM case via Ecurep URL:

  • Transfer the `sos report` archive to the target system.

    [root@sos /]# ls /host/var/tmp/sosreport-sos-podmansos-2023-11-25-kyarksu.tar.xz
    /host/var/tmp/sosreport-wgpuos-podmansos-2023-11-25-kyarksu.tar.xz

  • If required, transfer the file to a system with internet access. When using FTP as a protocol, make sure to use the binary transfer method.
  • Prepare the diagnostic data:
    https://www.ibm.com/support/pages/node/739597
    
    Send the diagnostic data to IBM:
    https://www.ibm.com/support/pages/enhanced-customer-data-repository-ecurep-send-data
    
    Secure upload the file to existing IBM case via Ecurep URL:
    https://www.secure.ecurep.ibm.com/app/upload_sf