Attaching direct storage on IBM Z
Direct storage attachment can lead to an I/O performance improvement on IBM Z® Systems. This can be achieved by setting up a physical connection to the disks that contain file system data for each node in the participating clusters. The I/O workload is routed automatically by IBM Storage Scale to the directly attached storage using the SAN over a Fibre Channel (FC) network, which also reduces load on the cluster IP network.
IBM Storage Scale on IBM Z supports several types of shared backend storage like Direct Access Storage Devices (DASD) or SCSI devices. For more information about how to set up those devices for shared access, see Getting started with IBM Spectrum® Scale for Linux® on IBM Z.
For ECKD-DASDs, by default, if all paths to the disk are unavailable, then the corresponding device in Linux waits for one of the paths to recover. I/O requests are blocked while the device is waiting. The failfast attribute of ECKD-DASDs can be set to immediately return the I/O requests as failed, while no path to the device is available. IBM Storage Scale then performs a failover of the I/O traffic to the network connection. For more information on how to set the failfast attribute, see Enabling and disabling immediate failure of I/O requests and chzdev - Configure IBM Z devices.
When you use SCSI devices, a timeout can be set before the error recovery. The default timeout is 30 seconds. For more information about using SCSI devices, Working with SCSI devices. Similar to the ECKD-DASDs, IBM Storage Scale also handles the path failover and failback between SAN path and network connection.
lcl for local, and the column "Device/NSD ID" displays the
device name and the "NSD node" column must remain empty as shown in the following example:
mmdiag --iohist
=== mmdiag: iohist ===
I/O history:
I/O start time RW Buftype disk:sectorNum nSec time Type Device/NSDID NSD node
--------------- -- ----------- ----------------- ----- ------- ---- ---- --
09:02:45.109249 W data 3:19736896 512 1.110 lcl dasdd1
09:02:45.113789 W data 4:42704704 512 1.097 lcl dasde1
09:02:45.118548 W data 1:16916288 512 1.136 lcl dasdb1
09:02:45.123191 W data 5:21751616 512 1.219 lcl dasda1
09:02:45.127730 W data 2:16916288 512 1.262 lcl dasdc1
09:02:45.132335 W data 3:29004608 512 1.170 lcl dasdd1
09:02:45.136838 W data 4:3216192 512 1.044 lcl dasde1
09:02:45.141494 W data 1:17722176 512 1.132 lcl dasdb1
09:02:45.146166 W data 5:28198720 512 1.048 lcl dasda1
09:02:45.150742 W data 2:10872128 512 1.033 lcl dasdc1
09:02:45.155404 W data 3:17319232 512 1.120 lcl dasdd1
09:02:45.159955 W data 4:16110400 512 1.039 lcl dasde1
cli
instead of lcl as shown in the following example: mmdiag --iohist
=== mmdiag: iohist ===
I/O history:
I/O start time RW Buf type disk:sectorNum nSec time ms Type Device/NSD ID NSD node
--------------- -- ----------- ----------------- ----- ------- ---- ------------------ ----------
17:58:34.489275 W logData 4:21198704 8 0.450 cli 9113AC14:60D9C779 192.168.22.220
17:58:34.719027 W logData 4:21198704 8 0.338 cli 9113AC14:60D9C779 192.168.22.220
17:58:34.720150 W logData 4:21198704 8 0.308 cli 9113AC14:60D9C779 192.168.22.220
17:58:34.721608 W logData 4:21198704 8 0.251 cli 9113AC14:60D9C779 192.168.22.220
17:58:36.039115 W logData 4:21198704 8 0.589 cli 9113AC14:60D9C779 192.168.22.220
17:58:36.042718 W iallocSeg 4:22204032 16 0.674 cli 9113AC14:60D9C779 192.168.22.220
17:58:36.042694 W iallocSeg 3:22204464 16 0.698 cli 9113AC14:60D9C778 192.168.22.220
17:58:36.043405 W logDesc 4:21198664 8 0.279 cli 9113AC14:60D9C779 192.168.22.220
18:00:58.540496 R metadata 3:21198656 512 1.218 cli 9113AC14:60D9C778 192.168.22.220
18:00:59.096484 R inode 2:21701464 8 0.478 cli 9113AC14:60D9C777 192.168.22.220
18:00:59.486800 W logData 4:21198704 8 0.493 cli 9113AC14:60D9C779 192.168.22.220
18:00:59.487349 W inode 2:21701464 8 0.421 cli 9113AC14:60D9C777 192.168.22.220