Summary of changes

This topic summarizes changes to the IBM Spectrum Scale licensed program and the IBM Spectrum Scale library. Within each topic, these markers (Start of change End of change ) surrounding text or illustrations indicate technical changes or additions that are made to the previous edition of the information.

Start of change
Summary of changes
for IBM Spectrum Scale 5.1.2
as updated, August 2022
This release of the IBM Spectrum Scale licensed program and the IBM Spectrum Scale library includes the following improvements. All improvements are available after an upgrade, unless otherwise specified.
Important: IBM Spectrum Scale 5.1.x supports Python 3.6 or later. It is recommended that Python 3.6 is installed through the OS package manager (For example, yum install python3). If you install Python 3.6 by other means, unexpected results might occur, such as failure to install gpfs.base for prerequisite checks, and workarounds might be required.
AFM and AFM DR-related changes
  • Added the afmResyncVer2 parameter to improve the replication performance. For more information, see AFM resync version 2.
  • Added AFM NFSv4 support. For more information, see AFM Network File System version 4 support.
  • Added a parameter to query uncached files on an AFM fileset. For more information, see the mmafmctl command.
  • Added a parameter to query dirty files on an AFM fileset. For more information, see the mmafmctl command.
  • Added parameters to query replication statistics and estimated time by using the --read-stats and --write-stats. For more information, see the mmafmctl command.
  • Documented errors in the AFM error logs with a description. For more information, see Active File Management error logs.
  • Both an AFM-DR secondary fileset and an SW fileset on a home target will not have any extra files. Because when the afmResyncVer2 parameter is enabled, the extra files are deleted. For more information, see Resync on SW filesets.
  • Start of changeWith the updated afmRPO parameter you can disable afmRPO. For more information, see mmchfileset command.End of change
  • Start of changeIntroduced support of Ubuntu 22.04 as AFM Gateway. For more information, see AFM and AFM DR limitations.End of change
AFM to Cloud Object Storage
Big data and analytics changes
For information on changes in IBM Spectrum Scale Big Data and Analytics support and HDFS protocol, see Big Data and Analytics - summary of changes.

IBM Spectrum Scale Container Storage Interface driver changes
For information on changes in the IBM Spectrum Scale Container Storage Interface driver, see IBM Spectrum Scale Container Storage Interface driver - Summary of changes.

Start of changeFile audit logging changesEnd of change
Start of changeFiles and directories within the file audit logging fileset can inherit Security-Enhanced Linux (SELinux) security contexts from their parent directories. For more information, see Monitoring file audit logging with rsyslog and SELinux.End of change
Data protection changes
Added mmxcp command that performs parallel copies of files from the specified source to the specified target in a single IBM Spectrum Scale cluster. For more information, see the mmxcp command and the Performing parallel copy with mmxcp command.
IBM Spectrum Scale Erasure Code Edition changes
For information on changes in the IBM Spectrum Scale Erasure Code Edition, see IBM Spectrum Scale Erasure Code Edition - Summary of changes.

File system core improvements
Display the fully qualified domain name (FQDN) of the specified nodes
Introduced the --fqdn parameter in the mmgetstate command to display the daemon fully qualified domain name (FQDN) of the specified nodes. The FQDN is the full name that is known to IBM Spectrum Scale at the time the node is added to the cluster. For more information, see mmgetstate command.
NUMA-aware RDMA adapter selection and NUMA-aware RDMA thread affinity
You can use the verbsNumaAffinity parameter of the mmchconfig command to enable NSD servers to select a NUMA-aware RDMA adapter for RDMA-based communication with NSD clients. This option also sets the RDMA thread's affinity to CPUs on the NUMA node to which the associated RDMA device is attached. For more information, see mmchconfig command.
Changing file system name is restricted if file audit logging or clustered watch folder is enabled for the file system
You cannot use the mmchfs command to change the file system name if file audit logging or clustered watch folder is enabled for the file system. For more information, see mmchfs command.
Support for GPUDirect Storage

Introduced the support for NVIDIA's GPUDirect Storage (GDS). This feature allows achieving low latencies and high throughput when reading data directly from an NSD server to a GPU buffer. It requires an InfiniBand fabric with GDS capable hardware and the CUDA environment that is installed on the GDS clients.

Both IBM Spectrum Scale client and NSD server must be running on IBM Spectrum Scale 5.1.2 to enable the support for GDS. For more information, see GPUDirect Storage support for IBM Spectrum Scale.

IBM Spectrum Scale 5.1.2 supports GDS Read operations. Writes (cuFileWrite) are supported in compatibility mode.

CCR is required to run the mmchconfig release=LATEST command
Cluster configuration by using primary and secondary server is discontinued. If the Cluster Configuration Repository (CCR) is not enabled, you cannot run the mmchconfig release=LATEST command. Use the mmchcluster --ccr-enable command to enable CCR in the cluster. For more information about the mmchconfig release=LATEST command, see mmchconfig command.
Support 4K disk sectors for IBM Spectrum Scale on Windows
Introduced 4K disk sector support for IBM Spectrum Scale on the Windows platform. Removed the restriction from the IBM Spectrum Scale FAQ. For more information about the updated FAQ question, see IBM Spectrum Scale support for 4K disk sectors.
Repair functionality of mmfsck command in online mode is deprecated
The repair functionality of mmfsck command in online mode is no longer available. The report-only operation still works in the online mode. For more information, see mmfsck command.
--metadata-block-size parameter of mmcrfs command is being deprecated
The --metadata-block-size parameter of the mmcrfs command can be used to specify a metadata block size that is different from the data block size. This option is being deprecated because it is no longer required to use for performance improvements for file systems with file system format 5.0.0 or later. This parameter will be removed in a future release. For more information, see mmcrfs command.
Display size of the LROC devices in the mmlsnsd command output
The mmlsnsd -L command output now shows the Size (MB) column for LROC devices. For more details, see mmlsnsd command.
File creation performance improvement
The performance of file creation has been improved by reducing the context switch between kernel and user space, by implementing the atomic_open inode operation.
mmap performance improvement
The performance of mmap when pre-fetching file data blocks has been improved.
DIO vectored read performance improvement
The performance of vectored reads in direct-IO mode has been improved with more efficient memory pool buffer locking.
New diskDown callback to inform about the reason for a disk failure
The diskDown callback is triggered on the file system manager when the status of a disk in a file system changes to down. The diskDown callback is similar to the diskFailure callback but provides the reason for triggering the diskDown callback. For more information, see mmaddcallback command.
Support for IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.0.1 (IF01)
IBM Spectrum Scale 5.1.2 supports IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.0.1 (IF01) or later. The older versions of GKLM are referred to as IBM Security Lifecycle Manager or SKLM in the documentation. The configuration information is the same for both GKLM and SKLM. For more information, see mmkeyserv command.
Clear QoS configuration automatically when system objects are deleted
You do not need to use the special option QOSSkipValidate=1 to override the QoS configuration validation. The following commands automatically clean up QoS data when those system objects are deleted:
  • mmdelfileset (filesets)
  • mmdelnode (nodes)
  • mmdelnodeclass (node classes)
  • mmauth (remote cluster names)
  • mmdeldisk (pools)
For more information, see mmqos command.
O_TMPFILE support (Kernel version greater than 3.15 required)
IBM Spectrum Scale 5.1.2 now supports the use of the O_TMPFILE open flag. The O_TMPFILE is used in the open(2) call. This allows the creation of an unnamed temporary regular file. Anything written to the resulting file is lost when the last file descriptor is closed unless the file is given a name. To link the temporary file, linkat(2) or gpfs_linkat() calls can be used.
gpfs_linkat API (Linux only)
The gpfs_linkat API is defined in the gpfs.h header file. The gpfs_linkat() function links a file to a directory name, and atomically replaces the destination if the file exists. The gpfs_linkat API has a similar interface to the linkat(2) system call with the following differences:
  • When a new path specifies an existing file, it is atomically replaced; EEXIST is never returned.
  • AT_EMPTY_PATH does not require CAP_DAC_READ_SEARCH.
For more information, see gpfs_linkat() subroutine.
Fine-Grained Directory Locking (FGDL) for link
When concurrent hard links are created in a single directory from multiple nodes, FGDL is enabled to improve performance of link creation.
New parameters to retain the user changes in mmsysmon.conf parameters after an RPM update
Introduced a set of parameters in the mmchconfig command to ensure that the changes in some of the mmsysmon.conf parameters do not disappear after an RPM update. For more information about how to set value for these parameters, see mmchconfig command.
Installation toolkit changes
  • Support for Ubuntu 22.04 on x86 starting IBM Spectrum Scale 5.1.2.5.
  • Start of changeSupport for Red Hat Enterprise Linux 8.5 on x86_64, PPC64LE, and s390x.End of change
  • Support for user-defined profiles of GPFS configuration parameters. For more information, see spectrumscale command and Setting configuration parameters before installation.
  • Support for populating cluster state information from mixed CPU architecture nodes. The information is populated from nodes that have the same CPU architecture as the installer node. Information for nodes of a different CPU architecture is not populated.
  • Support for IBM Spectrum Scale Erasure Code Edition on IBM Z®.
  • Faster upgrades due to optimizations in the upgrade path.
Management API changes
The following endpoints are added:
  • Start of changePOST Filesystems/{filesystemPath}/loopbackDevice/attach/ {fileName}/{nodeName}End of change
  • Start of changeDELETE Filesystems/{filesystemPath}/loopbackDevice/detach/{fileName}/{nodeName}End of change
  • Start of changeGET Filesystems/{filesystemPath}/loopbackDevice/{fileName}/{nodeName} End of change
  • Start of changeGET filesystems/{filesystemName}/directory/{path}End of change
The following endpoints are modified:
  • Start of change PUT config/monitor/pause/{node}End of change
  • Start of changePUT config/monitor/resume/{node}End of change
  • Start of changePOST filesystems/{filesystemName}/directory/{path}End of change
  • Start of changeGET nsds/nsdNameEnd of change
  • Start of changePOST Filesystems/{filesystemPath}/loopbackDevice/attach/ {fileName}/nodes/{name}End of change
  • Start of changeDELETE Filesystems/{filesystemPath}/loopbackDevice/detach/{fileName}/nodes/{name}End of change
  • Start of changeDELETE filesystems/{filesystemName}/directory/{path}End of change
  • Start of changeGET Filesystems/{filesystemPath}/loopbackDevice/{fileName}/nodes/{name}End of change
  • Start of changePUT smb/shares/shareName End of change
  • POST Nodes
  • POST Filesystems/{filesystemName}/filesets
  • GET Filesystems/{filesystemName}/filesets
  • DELETE Filesystems/{filesystemName}/filesets/{filesetName}/link
  • PUT Filesystems/{filesystemName}/filesets/{filesetName}
  • GET encryption/clients
  • GET encryption/keys
  • GET encryption/servers
  • GET encryption/tenants
  • POST encryption/servers
  • POST encryption/tenants
  • POST encryption/keys
  • POST encryption/clients
  • POST encryption/client/register
  • PUT encryption/client/deregister
  • PUT encryption/clients
  • PUT encryption/servers
  • DELETE encryption/tenants
  • DELETE encryption/clients/{clientName}
  • DELETE encryption/servers/{serverName}

For more information, see IBM Spectrum Scale management API endpoints.

Management GUI changes
Total requests written to and read from remote systems can be viewed and sorted in the GUI performance metric table on the Active File Management page.

Start of changeSMB changesEnd of change
Start of change

Removed the 'The CTDB nodes file for CES' name from the IBM Spectrum Scale other configurations list mentioned underneath the sub-section Updating IP addresses or host names in other configurations.

Replaced option '--all' with '-a' that is used in mmces service command statement to failback or restore the SMB protocol configuration. For more information see, Failback or restore steps for the SMB protocol.

End of change
NFS changes
  • Upgraded CES NFS-Ganesha to NFS V3-stable version. This upgrade adds more stability, fixes defects, and improves the NFS performance.
  • Start of change Updated limitations for NFS Ganesha protocol. For more information, see General CES NFS Linux limitations.End of change
  • Start of changeIntroduced configuration parameters:DIR_CHUNK, READDIR_RES_SIZE, and SERVER_SCOPE. For more information, see mmnfs command.End of change
Protocols authentication changes
IBM Spectrum Scale protocol authentication uses a custom keytab file instead of a system keytab file. This helps to remove conflicts with other applications that use a system keytab file, for example, sssd. For more information, see mmuserauth command.

Object protocol changes
On IBM Spectrum Scale 5.1.2.0, installation or upgrade of the object protocol requires the preconfiguration of OpenStack repositories on all protocol nodes. For more information, see OpenStack repository configuration required by the object protocol.
Note: This repository configuration is not required in IBM Spectrum Scale 5.1.2.1 and later releases.
Packaging changes
  • The gpfs.kafka package is removed from the IBM Spectrum Scale installation images for Linux.
Start of changePerformance monitoring changesEnd of change
Start of change

Start of changeFrom IBM Spectrum Scale 5.1.2, a security feature is introduced to limit the connections of performance monitoring sensors to a list of known IBM Spectrum Scale node IPs. For more information, see Configuring the performance monitoring tool.End of change

End of change
Python-related changes
From IBM Spectrum Scale release 5.1.0, all Python code in the IBM Spectrum Scale product is converted to Python 3. The minimum supported Python version is 3.6.

For compatibility reasons on IBM Spectrum Scale 5.1.0.x and later on Linux 7.x (7.7 and later), a few Python 2 files are packaged and they might trigger dependency-related messages. In certain scenarios, Python 2.7 might also be required to be installed. Multiple versions of Python can co-exist on the same system. For more information, see the entry about mmadquery in Guidance for Red Hat Enterprise Linux 8.x on IBM Spectrum Scale nodes.

The Python code in IBM Spectrum Scale 5.0.y or earlier continues to be in Python 2.

Tip:
  • IBM Spectrum Scale 5.1.x.x uses Python 3 code, and it runs best with operating systems that also use Python 3 internally such as Linux 8.x, SLES 15, and Ubuntu 20.04.
  • IBM Spectrum Scale 5.0.x.x uses Python 2 code, and it runs best with operating systems that also use Python 2 internally, such as Linux 7.x.
System health changes
  • Start of change

    Added a new option --temporary to the mmhealth command. For more information, see the mmhealth command.

    End of change
  • Added a new component GDS in the parameter list of the mmhealth node show and mmhealth cluster show commands. For more information, see the mmhealth command.

    For more information, see GPUDirect Storage support for IBM Spectrum Scale

  • Added a new component, HEALTHCHECK, to improve the experience of users with Call Home capability by providing them a direct support from IBM. For more information, see Proactive system health alerts.
  • New API sub-command added to the mmperfmon delete command, which can be used to delete expired API keys. For more information, see mmperfmon command and Configuring performance monitoring API keys.

  • New events added for the following:
    • Call Home events
    • CES network events
    • File Audit Logging events
    • Start of changeFile system eventsEnd of change
    • GPFS events
    • Start of changeGUI eventsEnd of change
    • Local Cache events
    • Network events
    • Watchfolder events

    For more information, see Events

Call home changes
Call home has the following improvements:
Linux on Z changes

The following changes are made to the IBM Spectrum Scale Erasure Code Edition on IBM Z:

  • Support for the IBM Spectrum Scale Erasure Code Edition on IBM Z15 server.

  • Support for four storage nodes that are running in native LPARs.

  • Support for up to three NVMe drives with a U.2 form factor per storage node.

  • Support for HiperSockets network connection.

  • Automatic mapping from the server’s drive slots to PCIe buses. There is no need to manually create an Enclosure Descriptor File (EDF).

For more information, see IBM Spectrum Scale ECE 5.1.2.

Commands, data types, and programming APIs
The following section lists the modifications to the documented commands, structures, and subroutines:
New commands
mmxcp
New structures
There are no new structures.
New subroutines
gpfs_linkat()
New user exits
There are no new user exits.
Changed commands
  • Start of changegpfs.snapEnd of change
  • Start of changemmlsquotaEnd of change
  • Start of changemmchfsEnd of change
  • mmaddcallback
  • mmafmconfig
  • Start of changemmafmcosconfigEnd of change
  • mmafmctl
  • mmaudit
  • mmbackup
  • Start of changemmchclusterEnd of change
  • Start of changemmchconfigEnd of change
  • Start of changemmchfilesetEnd of change
  • mmchfs
  • Start of changemmcrclusterEnd of change
  • Start of changemmcrfsEnd of change
  • mmfsck
  • mmgetstate
  • mmhealth
  • mmkeyserv
  • mmlsnsd
  • mmnetverify
  • Start of changemmnfsEnd of change
  • mmperfmon
  • mmqos
  • mmsetquota
  • mmuserauth
  • Start of changemmxcpEnd of change
  • spectrumscale
Changed structures
There are no changed structures.
Changed subroutines
Start of changegpfs_fcntl()End of change
Deleted commands
mmcesdr
Deleted structures
There are no deleted structures.
Deleted subroutines
There are no deleted subroutines.
Messages
The following are the new, changed, and deleted messages:
New messages
6027-1829, 6027-1830, 6027-2417, 6027-2418, 6027-2419, 6027-3265, 6027-3335, 6027-4303,6027-4304, 6027-4305, 6027-4306,Start of change 6027-984, 6027-3411, 6027-3412, 6027-3413, 6027-3516, 6027-3951, 6027-3952, 6027-3953, 6027-4020, 6027-4021, 6027-4022, 6027-4023, 6027-4024, 6027-4212, 6027-4307, 6027-4308, 6027-4309, 6027-4310, and 6027-4311End of change
Changed messages
Start of change6027-475, 6027-477, 6027-1311, 6027-1312, 6027-1323, 6027-1324, 6027-1325, 6027-2050, 6027-3245, 6027-3263, 6027-3265, and 6027-3328End of change
Deleted messages
6027-1922
Documentation changes
List of documentation changes in product guides and respective IBM Documentation sections

The following is a list of documentation changes including changes in topic titles, changes in placement of topics, and deleted topics:

Table 1. List of changes in documentation
Guide IBM Documentation section List of changes
Problem Determination Guide Monitoring

Applied the following changes:

  • Re-arranged the list of first three chapters:

    • Monitoring system health by using IBM Spectrum Scale GUI
    • Monitoring system health by using the mmhealth command
    • Performance monitoring
  • Changed following section headings:
    • Set up event notifications to Event notifications
    • Monitoring tip events to Monitoring tips by using GUI
    • Monitoring thresholds to Monitoring thresholds by using GUI
Start of changeTroubleshootingEnd of change Start of changeRemoved the following topics:
  • Protocols cluster disaster recovery issues
  • Message queue events
End of change
Message number changes:
  • 6027-4305 changed to 6027-4304
  • 6027-4306 changed to 6027-4305
  • 6027-4307 changed to 6027-4306
Administration Guide Configuring
  • Changed the topic title from Configuration parameters for AFM to Configuration parameters for AFM and AFM-DR.
  • Merged all AFM and AFM-DR configuration parameters tables into a single table.
  • Removed the following topics:
    • Parallel data transfer configuration parameters for AFM
    • Configuration parameters for AFM-based DR
    • Parallel data transfer configuration parameters for AFM-based DR
Administering
  • Removed the following topics:
    • Creating the inband disaster recovery setup
    • Creating the outband disaster recovery setup
    • Performing failover for protocols cluster when primary cluster fails
    • Performing failback to old primary for protocols cluster
    • Performing failback to new primary for protocols cluster
    • Backing up and restoring protocols and CES configuration information
    • Updating protocols and CES configuration information
Command and Programming Reference Guide Command reference Removed mmcesdr command.
Start of changeProgramming referenceEnd of change Start of changeAdded the following API documents:End of changeStart of change
  • PUT health/config/monitor/resume/{server}
  • PUT health/config/monitor/pause/{server}
End of change
Concepts, Planning, and Installation Guide Product overview

Changed the topic title from File audit logging events' descriptions to File audit logging events.

End of change