GPFS architecture

Use this information to understand the architecture of GPFS.

Interaction between nodes at the file system level is limited to the locks and control flows required to maintain data and metadata integrity in the parallel environment.

A discussion of GPFS architecture includes:

Special management functions
In general, GPFS performs the same functions on all nodes. It handles application requests on the node where the application exists. This provides maximum affinity of the data to the application.
Use of disk storage and file structure within a GPFS file system
A file system (or stripe group) consists of a set of disks that store file data, file metadata, and supporting entities such as quota files and recovery logs.
GPFS and memory
GPFS uses three areas of memory: memory allocated from the kernel heap, memory allocated within the daemon segment, and shared segments accessed from both the daemon and the kernel.
GPFS and network communication
Within the GPFS cluster, you can specify different networks for GPFS daemon communication and for GPFS command usage.
Application and user interaction with GPFS
There are four ways to interact with a GPFS file system.
NSD disk discovery
When the GPFS daemon starts on a node, it discovers the disks defined as NSDs by reading a disk descriptor that is written on each disk owned by GPFS. This enables the NSDs to be found regardless of the current operating system device name assigned to the disk.
Failure recovery processing
In general, it is not necessary to understand the internals of GPFS failure recovery processing since it is done automatically. However, some familiarity with the concepts might be useful when failures are observed.
Cluster configuration data files
GPFS commands store configuration and file system information in one or more files collectively known as GPFS cluster configuration data files. These files are not intended to be modified manually.
GPFS backup data
The GPFS mmbackup command creates several files during command execution. Some of the files are temporary and deleted at the end of the backup operation. There are other files that remain in the root directory of the fileset or the file system and they must not be deleted.
Clustered configuration repository
The Clustered Configuration Repository (CCR) is used by GPFS and many other IBM Spectrum Scale components like GUI, the CES services and the monitoring service to store files and values across the cluster, and to return requested files and values to the caller.

Parent topic: Product overview