Storage server and disk configuration for OCFS2
This OCFS2 study used DASD type disk storage, which uses the IBM® ECKD™ format and FICON® channels.
Using Fibre Channel Protocol (FCP) connected SCSI disks would have been also appropriate for this system. The DASD devices were attached with four FICON Express4 channels (4 Gb/sec link speed) using switched connections to four host adapters on the IBM System Storage™ DS8000® storage unit.
When selecting the disks for shared storage on the IBM System Storage DS8000, they were spread out as much as possible among ranks (areas) of the disk array drawers, to prevent disk access from being concentrated in any one area. The IBM System Storage DS8000 consists of two server complexes, each having two power processors and separate caches each with half of the total cache size. The disk storage was selected from the ranks pre-configured on this IBM System Storage DS8000, to use the caches from both processor complexes.
More information about performance considerations for databases for Linux® on IBM System z® is available at the following Web site:
See the documentation for your storage control unit to determine the disk layout that gives the best performance. For the IBM System Storage DS8000 series, refer to IBM System Storage DS8000 Performance Monitoring and Tuning, an IBM Redbooks® publication, available at:
https://www.redbooks.ibm.com/abstracts/sg247146.html
If I/O latency is important to the undertaking, and striped storage is to be used, implement striping at the level of the physical storage server, if it is available. The striping, therefore, must be configured for the disk devices that are going to be used for shared storage before they are introduced to Linux as devices. The IBM System Storage DS8000 storage server has the facility to stripe disk storage, creating logical disks that are physically spread across ranks of physical disk arrays in the unit. Striping was introduced with IBM System Storage DS8000 microcode level r3 code 63.0106.0, and can be upgraded to the appropriate level.
On an enterprise system, being able to expand volumes dynamically is an important convenience supplied by the Logical Volume Manager (LVM), so administrators must give extra thought to prepare for expanding a system without it. The problem appears to be that the LVM must be developed with the same cluster awareness of which nodes are alive in a cluster as O2CB has for managing the file system. There is not an existing version of clustered LVM that works with OCFS2. OCFS2 Version 1.4 was announced as having moved towards being able to resize volumes dynamically, but at the time of this writing, it is not fully supported.