Adding and removing SAN storage

Add external SAN storage to Integrated Analytics System for backup and restore capabilities.

Note: This functionality is not available on customer-supplied rack M4002-001 model. For more information on this model, see Integrated Analytics System M4002-001 model overview.

Before you begin

Prior to adding or removing external storage to the appliance, the system state must be stopped. The script will check to ensure that the appliance is in the proper state prior to executing. Run apstop command to stop the appliance.

The default design for patch panel SAN interface is that node0101 and node0102 are fc_host 4 and 6. You can verify it by running the following command:
[apuser@node0101 ~]$ storage_setup --show_san_hardware_info
See /var/log/appliance/platform/xcat/setup_storage.log to watch its progress.
Done
Displaying SAN information for all customer ports...
Displaying SAN information for node node0101...
Patch Panel port: FCB1 FC_Host: 4 PCI SLOT: P1-C3 Port 1 WWN: 0x100000109b3ba23a State: Online
Patch Panel port: FCA1 FC_Host: 6 PCI SLOT: P1-C2 Port 1 WWN: 0x100000109b3b7cdd State: Online
Displaying SAN information for node node0102...
Patch Panel port: FCB2 FC_Host: 4 PCI SLOT: P1-C3 Port 1 WWN: 0x100000109b3b67f6 State: Online
Patch Panel port: FCA2 FC_Host: 6 PCI SLOT: P1-C2 Port 1 WWN: 0x100000109b3ba39f State: Online
Displaying SAN information for node node0103...
Patch Panel port: NA FC_Host: 5 PCI SLOT: P1-C2 Port 0 WWN: 0x100000109b3b6bbb State: Online
Patch Panel port: NA FC_Host: 6 PCI SLOT: P1-C3 Port 1 WWN: 0x100000109b3b6bbc State: Online
Displaying SAN information for node node0104...
Patch Panel port: NA FC_Host: 5 PCI SLOT: P1-C2 Port 0 WWN: 0x100000109b3b6c33 State: Online
Patch Panel port: NA FC_Host: 6 PCI SLOT: P1-C3 Port 1 WWN: 0x100000109b3b6c34 State: Online
Displaying SAN information for node node0105...
Patch Panel port: NA FC_Host: 5 PCI SLOT: P1-C2 Port 0 WWN: 0x100000109b3b6c3c State: Online
Patch Panel port: NA FC_Host: 6 PCI SLOT: P1-C3 Port 1 WWN: 0x100000109b3b6c3d State: Online
Displaying SAN information for node node0106...
Patch Panel port: NA FC_Host: 5 PCI SLOT: P1-C2 Port 0 WWN: 0x100000109b2168ff State: Online
Patch Panel port: NA FC_Host: 6 PCI SLOT: P1-C3 Port 1 WWN: 0x100000109b216900 State: Online
Displaying SAN information for node node0107...
Patch Panel port: NA FC_Host: 5 PCI SLOT: P1-C2 Port 0 WWN: 0x100000109b3b81df State: Online
Patch Panel port: NA FC_Host: 6 PCI SLOT: P1-C3 Port 1 WWN: 0x100000109b3b81e0 State: Online
Done

About this task

Adding SAN storage

To add external SAN storage, you need to log in to node0101 from the command prompt.

The SAN storage must be powered on and connected to the Fibre Channel connectors on the rear panel of the appliance.

When logged in, type the command:
storage_setup -e san
See /var/log/appliance/platform/xcat/setup_storage.log to watch its progress.
Making sure the script is called from node0101...
Done
Checking if GPFS already setup...
Done
Checking link state...
Done
Checking block device state...
Done
Checking multipath devices and populating GPFS stanza file...
Done
Beginning GPFS network disk configuration...
This step will format all external luns, a DESTRUCTIVE operation.
Proceed? [n|y]y
Converting SAN lun(s) into GPFS NSD...
Done
Creating GPFS Filesystem...
Done
Mounting GPFS filesystem on all nodes...
Done
Setup complete, issue apstop;apstart to refresh storage to the container...
Done
Setting folder+permission structure for external storage...
Done

Removing SAN storage

Before you remove SAN storage, the system state must be disabled (stopped).

To remove external SAN storage, you need to log in to node0101 from the command prompt.

When logged in, type the command:

storage_setup -d san
See /var/log/appliance/platform/xcat/setup_storage.log to watch its progress.
Validating extra gpfs mount was added...
Done
Validating system is stopped before storage shared with container can be removed...
Done
Undoing GPFS specific step...
This step will erase GPFS config on storage, a DESTRUCTIVE operation.
Proceed? [n|y]y
Done
Unmounting GPFS filesystem on all nodes...
Done
Deleting GPFS Filesystem...
Done
Reverting GPFS NSD...
Done
Removal complete. If done managing storage, start the system software by running apstart....

Verifying SAN storage connectivity

After setting up SAN storage, use the following commands to verify the status of the links and WWNs. If you are using FCA1, FCA2, FCA3, FCA4 connections, run them on node0101 and node0102. If not, connect directly to nodes using the patch panel, and run the commands on every node. Run the commands as apuser:

  • To check the port status: 
    $ cat /sys/class/fc_host/host*/port_state
    Sample output: 
    $ cat /sys/class/fc_host/host*/port_state
Online
Online
Online
Online
Online
Online
  • To get the WWNs: 
    $ cat /sys/class/fc_host/host*/port_name
    Sample output: 
    $ cat /sys/class/fc_host/host*/port_name
0x100000109b1dxxxx
0x100000109b1dxxxx
0x100000109b1cxxxx
0x100000109b1cxxxx
0x100000109b1dxxxx
0x100000109b1dxxxx
    In the example output, WWN is the string of numbers after 0x.