IBM developerWorks: Wikis - Linux for Power Architecture

One important question to be answered before you are installing the operating system is how you plan to protect your server(s) against disk failures. This means hardware failures as well as faillures caused by an operating system or missusage of the system. You can, of course, try to ignore them and rely on you luck - but think again; how possible might be a disk failure and what impact does it have to my daily business and how much money do I have to spend to avoid them?

In modern operating environment, the need of protecting the systems against hardware failures is critical. And easier to implement than to trying to recover from a failure.

Nowadays the prices of additional hardware (disks and adapters) is reasonable even for smaller systems to implement mechanism to avoid and recover from disk failures. Furthermore modern operating systems offer the possibility to create software-based RAID (Redundant Arry of Independent Disks) with minimal overhead associated to that task. Hardware RAID adapters are available which offers improved performance by offloading the task of creating and managing and disk array from the operating systems. And beyond that, some disk subsystems implement RAID capabilites which moves the RAID logic and administration completely away from the server side.

All mentioned possibilities are available for Linux on POWER systems. Implementing a software RAID will be covered later when we are talking about the installation of Linux itself. Using RAID capable disk subsystems is out of scope of this document. Please refer to the individual setup instructions for your disk subsystem to enable hardware RAID (if available) there.

In this chapter we will focus on implementing hardware RAID using RAID capable SCSI adapter within the IBM eServer p5 and OpenPower systems.

Requirements to implement hardware RAID

To be able to use hardware RAID on IBM eServer p5 and OpenPower systems you must have HW RAID capable adapters. Refer to table x.x for the actual feature codes of the possible SCSI hardware RAID adapter.

Feature Code OpenPower	Feature Code p5 Server	Part Number	Description
1975	5703	52P8630	PCI-X Dual Channel Ultra320 SCSI RAID Adapter
1975	5709	52P8631	Dual Channel SCSI RAID Enablement Card

All listed adapters offer the following features:

Boot support
RAID0, RAID10, RAID5 support
Two independet SCSI channels
...

More details about the IBM Dual Channel SCSI RAID Adapter can be found at:
http://publib.boulder.ibm.com/infocenter/pseries/index.jsp?topic=/com.ibm.pseries.doc/hardware.htm

-> Select "Installable Options"
-> SCSI PCI Adapters

Each IBM eServer p5 and OpenPower system offers different possible configurations for the internal disk drives. For more details about the valid and supported configurations please refer to:

IBM eServer p5 and OpenPower Redpapers at htpp://www.redbooks.ibm.com
IBM Offering Information at http://www.ibm.com/common/ssi/OIX.wss

Note: At the time of writing, internal HW RAID is not supported on the IBM eServer p5 510 and OpenPower 710.

Enabling HW RAID boot support

To be able to use HW RAID boot support, you must create the RAID array before you install the operating system. This can be done using either the IBM Standalone Diagnostic CD-ROM or booting the Linux rescue mode.

Note: Booting a system with the IBM Standalone Diagnostic CD-ROM take quite a long time, compared to using the Linux rescue mode. Nevertheless, there are more configuration and rescue possibilities within the IBM Standalone Diagnostics. Furthermore if you plan to use the IBM Virtual I/O Server appliance you must use the IBM Standalone Diagnostics to setup HW RAID.

IBM Standalone Diagnostics

The IBM Standalone Diagnostics CD-ROM provides hardware diagnostics and service-related utilities for POWER, PowerPC, eServer i5 system with common pSeries I/O, and RS/6000-based systems. The standalone diagnostics CD-ROM would be used in the following situations when it makes sense to test the hardware independent of the operating system:

When there is no operating system installed on a system or partition
When the operating system does not have support for the service related function you wish to perform
When there may be a problem with the boot device
When the service documentation specifically recommends running standalone diagnostics

The actual IBM Standalone Diagnostic CD-ROM can be downloaded at the following web page:
http://techsupport.services.ibm.com/server/mdownload/diags

Diagnostics, which are available for AIX® and Linux® systems and logical partitions, can help you perform hardware analysis. If a problem is found, you will receive a service request number (SRN) or a service reference code (SRC) that can help pinpoint the problem and determine a corrective action.

Additionally, there are various service aids in the diagnostics that can help you with service tasks on the system or logical partition.

You can run the IBM Standalone Diagnostic the following ways:

Running the eServer stand-alone diagnostics from CD-ROM
Running the eServer stand-alone diagnostics from a Network Installation Management server

Running the stand-alone diagnostics from CD-ROM on a server with an HMC attached

Use this procedure to run the stand-alone diagnostics on a system with an HMC attached. If you have logical partitions, note the following considerations:

When running diagnostics in a logically partitioned system, you must run diagnostics in the logical partition containing the resource or resources that you want to test.
The device from which you are loading stand-alone diagnostics must be made available to the logical partition on which you want to run diagnostics. This might require moving the device (for example, the optical drive or network adapter connected to the Network Installation Management (NIM) server that has a stand-alone diagnostic image) from the logical partition that currently contains the device to the logical partition on which you want to run diagnostics.

Perform the following steps from the HMC to run stand-alone diagnostics:

Note: If you need help with any of these steps, contact your system operator.

Remove all tapes, diskettes, and optical media, and put the diagnostic CD-ROM into the optical media drive in the managed system (not the optical media drive in the HMC).
Shut down the operating system from the HMC by performing the following steps:
1. In the navigation area, select Server and partition > Server Management.
2. In the contents pane, expand the server that contains the partition you want to test.
3. Right-click the partition and select Open Terminal Window.
4. In the VTerm window, login as root user and enter any requested passwords.
5. Shut down the operating system using one of the following commands:
  - If AIX is running, type the shutdown -F command.
  - If Linux is running, type the shutdown -h now command
6. Close the VTerm window.
From the HMC, right-click the partition and select Activate. Ensure the Open a terminal window or console session box is selected and click OK.
Continue with the Selecting options for stand-alone diagnostics procedure.

Running the stand-alone diagnostics from CD-ROM on a system without an HMC attached

To run eServer stand-alone diagnostics, perform the following steps:

Choose from the following options:
1. If the system is powered on, perform the following steps:
  1. Let the system administrator and system users know that the system unit will be shut down.
  2. Stop all programs including the operating system. See Start or stop the system for details.
2. If the system is powered off, you will need to start it in order to put the diagnostic CD-ROM in the optical media drive.
Put the diagnostic CD-ROM in the optical media drive.
Re-start the server.
Continue with the Selecting options for stand-alone diagnostics procedure.

Using IBM eServer stand-alone diagnostics service aids to setup HW RAID

After you've loaded the IBM eServer stand-alone diagnostics, you can use the included PCI-X SCSI Disk Array service aid to configure and setup hardware RAID.

Note: The loading of the IBM eServer stand-alone diagnostic takes some time!

First select the language.

Now select the correct terminal emulation. VT100 should be appropriate.

At this point scroll down the list, using the arrow-down button, until you reach the entry PCI-X SCSI Disk Array and press Enter.

Before you can create a SCSI disk array you must format the disks you want to use in the array. The disks must have a format of 522 byte per sector.
-> Erklärung warum, soweit möglich.
To do so, select "Create an Array Candidate pdisk and Format to 522 Byte Sectors" from the menu.

Now you must select the apropriate SCSI controller.

Note: If you have multiple controllers in the system the location code shown at the selection helps you identifiying the correct one.
-> Beispiel angeben

Now you can select the disks you want to prepare for use in a SCSI disk array.

Note: Continuing with this option will format the disks. All data on the disks will be lost. Some disks require that their microcode be updated to the latest level before being formatted to 522 byte sectors. Also, some disks do not support being formatted to 522 byte sectors. The PCI-X SCSI Disk Array Manager will inform the user of these situations when they exist.

Finally a message will display asking if you want to continue. To proceed with the format, select OK or press Enter to continue. To return to the previous menu without formatting the disks, select Cancel.

After the formatting is complete, the disks will be Array Candidate pdisks and will be ready for use in PCI-X SCSI Disk Arrays. This operation will also zero all the data on the disks. The PCI-X SCSI RAID Controller keeps track of the disks that have their data zeroed. These Zeroed Array Candidate pdisks can be used to create a disk array that will be immediately protected against disk failures, and they are the only disks that can be added to an existing disk array. An Array Candidate pdisk will lose its Zeroed state after it has been used in an array or is unconfigured. It will also lose its Zeroed state after the system has been rebooted. To return an Array Candidate pdisk to the Zeroed state, follow the steps previously described in this section for preparing disks for use in disk arrays.

To create a SCSI disk array select "Create a SCSI Disk Array" from the list.

Select the apropriate SCSI disk controller on which you want to create the array.

Select the RAID level for the array.

Select the stripe size in kilobytes for the array.

Select the disks that you want to use in the array according to the requirements on this screen.

A Dialog Screen summarizes your selections. Press Enter to create the array.

You can now use the disk array like any other SCSI disk drive:

You can partition the drive.
You can add the drive to a volume group.
You can create Logical Volumes and File Systems.

Use standard Linux procedures to perform these tasks, and treat the array in the same way that you would treat any physical SCSI disk.

Linux rescue mode

The Linux rescue mode, as it name implies, is normally used for system recovery (e.g. if a system fails to boot). But it can also be used to create bootable SCSI disk arrays.

You can enter the Linux rescue mode the following ways:

Using the installation CD-ROMs
Using a network installation source

Starting the Linux rescue mode from CD-ROM on a server with an HMC attached

Use this procedure to run the stand-alone diagnostics on a system with an HMC attached. If you have logical partitions, note the following considerations:

When running diagnostics in a logically partitioned system, you must run diagnostics in the logical partition containing the resource or resources that you want to test.
The device from which you are loading stand-alone diagnostics must be made available to the logical partition on which you want to run diagnostics. This might require moving the device (for example, the optical drive or network adapter connected to the Network Installation Management (NIM) server that has a stand-alone diagnostic image) from the logical partition that currently contains the device to the logical partition on which you want to run diagnostics.

Perform the following steps from the HMC to start the Linux rescue mode:

Remove all tapes, diskettes, and optical media, and put the first Linux CD-ROM into the optical media drive in the managed system (not the optical media drive in the HMC).
Shut down the operating system from the HMC by performing the following steps:
1. In the navigation area, select Server and partition > Server Management.
2. In the contents pane, expand the server that contains the partition you want to test.
3. Right-click the partition and select Open Terminal Window.
4. In the VTerm window, login as root user and enter any requested passwords.
5. Shut down the operating system using one of the following commands:
  - If AIX is running, type the shutdown -F command.
  - If Linux is running, type the shutdown -h now command
6. Close the VTerm window.
From the HMC, right-click the partition and select Activate. Ensure the Open a terminal window or console session box is selected and click OK.
At the installation screen, abort the installation. Confirm your choice if necessary.
Select "Install operating system"
Select "Boot operating system"
1. Select "Start Rescue System"
At the login prompt, enter "root" as user.

Running the stand-alone diagnostics from CD-ROM on a system without an HMC attached

To run eServer stand-alone diagnostics, perform the following steps:

Choose from the following options:
1. If the system is powered on, perform the following steps:
  1. Let the system administrator and system users know that the system unit will be shut down.
  2. Stop all programs including the operating system. See Start or stop the system for details.
2. If the system is powered off, you will need to start it in order to put the diagnostic CD-ROM in the optical media drive.
Put the diagnostic CD-ROM in the optical media drive.
Re-start the server.
At the installation screen, abort the installation. Confirm your choice if necessary.
Select "Install operating system"
Select "Boot operating system"
1. Select "Start Rescue System"
At the login prompt, enter "root" as user.

Using Linux rescue mode to setup HW RAID

After you've logged on, you can use the IBM SCSI RAID utility "iprconfig" to configure and setup a SCSI disk array. Simply type "iprconfig" at the comand line and press enter.

# iprconfig

You should see the entry menu for the IBM Power RAID Configuration Utility. To start creating a disk array, selct "Work with disk arrays".

                    IBM Power RAID Configuration Utility

Select one of the following:

   1. Display hardware status
   2. Work with disk arrays
   3. Work with disk unit recovery
   4. Work with SCSI bus configuration
   5. Work with driver configuration
   6. Work with disk configuration
   7. Download microcode
   8. Analyze log


Selection: 2

e=Exit

Before you can create a SCSI disk array you must format the disks you want to use in the array. The disks must have a format of 522 byte per sector. -> Erklärung warum, soweit möglich.
To do so, select "Format device for RAID function" from the menu.

                             Work with Disk Arrays

Select one of the following:

   1. Display disk array status
   2. Create a disk array
   3. Delete a disk array
   4. Add a device to a disk array
   5. Format device for RAID function
   6. Format device for JBOD function (512)
   7. Create a hot spare
   8. Delete a hot spare

Selection: 5

e=Exit   q=Cancel

Select the drives you want to format for RAID functionality

Confirm your selection

After the formatting is complete, the disks will be Array Candidate pdisks and will be ready for use in PCI-X SCSI Disk Arrays. This operation will also zero all the data on the disks. The PCI-X SCSI RAID Controller keeps track of the disks that have their data zeroed. These Zeroed Array Candidate pdisks can be used to create a disk array that will be immediately protected against disk failures, and they are the only disks that can be added to an existing disk array. An Array Candidate pdisk will lose its Zeroed state after it has been used in an array or is unconfigured. It will also lose its Zeroed state after the system has been rebooted. To return an Array Candidate pdisk to the Zeroed state, follow the steps previously described in this section for preparing disks for use in disk arrays.

To create a SCSI disk array select "Create a disk array" from the list.

                             Work with Disk Arrays

Select one of the following:

   1. Display disk array status
   2. Create a disk array
   3. Delete a disk array
   4. Add a device to a disk array
   5. Format device for RAID function
   6. Format device for JBOD function (512)
   7. Create a hot spare
   8. Delete a hot spare

Selection: 2

e=Exit   q=Cancel

???Select the apropriate SCSI disk controller on which you want to create the array.
Select the disks that you want to use in the array according to the requirements on this screen.

Select the RAID level for the array.

Select the stripe size in kilobytes for the array.

A Dialog Screen summarizes your selections. Press Enter to create the array.

You can now use the disk array like any other SCSI disk drive:

You can partition the drive.
You can add the drive to a volume group.
You can create Logical Volumes and File Systems.

Use standard Linux procedures to perform these tasks, and treat the array in the same way that you would treat any physical SCSI disk.