Shared disk management

Edit online

When planning to manage shared disks, a decision must be made between using OCFS2 and the Oracle Automatic Storage Manager.

OCFS2 Cluster file system versus ASM

To share data in a cluster, it is essential to have a file system that is able to coordinate accesses from independent nodes to shared files, while ensuring data consistency. There are two cluster file systems supported for Oracle RAC for Linux™ on IBM® System z®:
  • OCFS2
  • Oracle Automatic Storage Manager (ASM), which is distributed with the Oracle RDBMS package.

ASM is a special solution for Oracle databases and cannot be used as a standard file system.

ASM is an application that manages every aspect of storage of data on a set of disks, from the lowest level where it determines where the bits are physically stored, providing a virtual file system that looks and acts like a hierarchical file system. The disks that are under management by ASM are not accessible to Linux as other mounted file systems, but these disks are available as virtualized storage that can be managed through the user interface program.

Note: It is not recommended or required to stack the volume managers ASM and LVM. Because ASM stripes the data over the disks automatically, the usage of the Linux logical volume manager (LVM) is not required for striping. On the current distributions, multi-pathing with FCP disks is handled by the multipath daemon.

From the user perspective, a file system controlled by ASM is similar to a normal file system, although it is accessed using the command asmcmd in the ASM HOME.

An alternative to ASM is to use OCFS2, one of the Linux file systems for shared storage. To the user, a file controlled by OCFS2 appears as a normal file, but OCFS2 supports shared access to files without corruption of the data. For more information about OCFS2 and how it can be used on IBM System z, refer to this document on DeveloperWorks:

https://www.ibm.com/developerworks/linux/linux390/perf/tuning_filesystems.html

OCFS2 Version 1.4 comes as part of the SLES 10 SP2 distribution. OCFS2 Version 1.2 comes as part of the SLES 10 SP1 distribution. OCFS2 exists on the SLES 11 distribution in the HA Extension.

The most serious shortcoming for OCFS2 on large databases is that OCFS2 does not do data striping like ASM does. The administrator would have to arrange to do striping of data on the disks using the control program on the IBM System Storage® DS8000® disk array (storage pool striping), before configuring the storage for the Oracle RAC system.

The setup of OCFS2 is described in this white paper:

Oracle Cluster File System Version 1.2, shared file system for Linux on IBM System z

Oracle Automatic Storage Manager

The disks that are managed by the Oracle Automatic Storage Manager (ASM) in Oracle RAC 10.2.0.2 and 10.2.0.3 can be configured as raw devices. With ASM in 10.2.0.3 and 10.2.0.4, block devices are also supported as storage that is easier to set up and produces the same results. The Linux support for raw devices is deprecated and might be removed, which is discussed in The Oracle Cluster Registry and the Voting Disk.

ASM uses direct I/O and is able to provide striping of the disk space. ASM manages the disks that are assigned to it down to the logic that operates on the tracks on the disks. To the system administrator ASM provides a virtualized file system. The file system can be accessed by the user named oracle using the command asmcmd from the Oracle $HOME_ASM environment. Storage is logically organized around diskgroups created by the user. Diskgroups can easily be created and have disks added or removed from them. ASM uses direct I/O.

In planning for ASM, select the disks to use for data files that are spread across the available disk arrays according to the model. To understand the logical structures in the IBM System Storage DS8000 storage server, consult IBM Redbooks® publication DS8000 Performance Monitoring and Tuning, which is linked in References.