Summary of changes
This topic summarizes changes to the IBM Spectrum Scale licensed program and the IBM Spectrum Scale library. Within each topic, these markers ( ) surrounding text or illustrations indicate technical changes or additions that are made to the previous edition of the information.
for IBM Spectrum Scale 5.1.2
as updated, February 2024
- Commands, data types, and programming APIs
- Messages
- Stabilized, deprecated, and discontinued features
- Documentation changes
- AFM and AFM DR-related changes
-
- Added the afmResyncVer2 parameter to improve the replication performance. For more information, see AFM resync version 2.
- Added AFM NFSv4 support. For more information, see AFM Network File System version 4 support.
- Added a parameter to query uncached files on an AFM fileset. For more information, see the mmafmctl command.
- Added a parameter to query dirty files on an AFM fileset. For more information, see the mmafmctl command.
- Added parameters to query replication statistics and estimated time by using the --read-stats and --write-stats. For more information, see the mmafmctl command.
- Documented errors in the AFM error logs with a description. For more information, see Active File Management error logs.
- Both an AFM-DR secondary fileset and an SW fileset on a home target will not have any extra files. Because when the afmResyncVer2 parameter is enabled, the extra files are deleted. For more information, see Resync on SW filesets.
- With the updated afmRPO parameter you can disable afmRPO. For more information, see mmchfileset command.
- Introduced support of Ubuntu 22.04 as AFM Gateway. For more information, see AFM and AFM DR limitations.
- AFM to Cloud Object Storage
-
- AFM to Cloud Object Storage read performance improvements by using multiple gateway nodes and mapping for large files. For more information, see AFM to cloud object storage parallel read data transfer.
- An access key and a secret key can contain special characters such as hyphen (-) or period (.). For more information see, mmafmcoskeys command.
- Added the --fast-readdir option and the afmObjectFastReaddir parameter for the read-dir performance improvement. For more information see, mmafmcosconfig command and mmchfileset command.
- Big data and analytics changes
- For information on changes in IBM Spectrum Scale Big Data and Analytics support and HDFS protocol, see Big Data
and Analytics - summary of changes.
- IBM Spectrum Scale Container Storage Interface driver changes
- For information on changes in the IBM Spectrum Scale Container Storage Interface driver, see
IBM Spectrum Scale Container Storage Interface driver - Summary of
changes.
- File audit logging changes
- Files and directories within the file audit logging fileset can inherit Security-Enhanced Linux (SELinux) security contexts from their parent directories. For more information, see Monitoring file audit logging with rsyslog and SELinux .
- Data protection changes
- Added mmxcp command that performs parallel copies of files from the specified source to the specified target in a single IBM Spectrum Scale cluster. For more information, see the mmxcp command and the Performing parallel copy with mmxcp command.
- IBM Spectrum Scale Erasure Code Edition changes
- For information on changes in the
IBM Spectrum
Scale Erasure Code Edition,
see
IBM Spectrum
Scale Erasure Code Edition - Summary of changes.
- File system core improvements
-
- Display the fully qualified domain name (FQDN) of the specified nodes
- Introduced the --fqdn parameter in the mmgetstate command to display the daemon fully qualified domain name (FQDN) of the specified nodes. The FQDN is the full name that is known to IBM Spectrum Scale at the time the node is added to the cluster. For more information, see mmgetstate command.
- NUMA-aware RDMA adapter selection and NUMA-aware RDMA thread affinity
- You can use the verbsNumaAffinity parameter of the mmchconfig command to enable NSD servers to select a NUMA-aware RDMA adapter for RDMA-based communication with NSD clients. This option also sets the RDMA thread's affinity to CPUs on the NUMA node to which the associated RDMA device is attached. For more information, see mmchconfig command.
- Changing file system name is restricted if file audit logging or clustered watch folder is enabled for the file system
- You cannot use the mmchfs command to change the file system name if file audit logging or clustered watch folder is enabled for the file system. For more information, see mmchfs command.
- Support for GPUDirect Storage
-
Introduced the support for NVIDIA's GPUDirect Storage (GDS). This feature allows achieving low latencies and high throughput when reading data directly from an NSD server to a GPU buffer. It requires an InfiniBand fabric with GDS capable hardware and the CUDA environment that is installed on the GDS clients.
Both IBM Spectrum Scale client and NSD server must be running on IBM Spectrum Scale 5.1.2 to enable the support for GDS. For more information, see GPUDirect Storage support for IBM Spectrum Scale.
IBM Spectrum Scale 5.1.2 supports GDS Read operations. Writes (cuFileWrite) are supported in compatibility mode.
- CCR is required to run the mmchconfig release=LATEST command
- Cluster configuration by using primary and secondary server is discontinued. If the Cluster Configuration Repository (CCR) is not enabled, you cannot run the mmchconfig release=LATEST command. Use the mmchcluster --ccr-enable command to enable CCR in the cluster. For more information about the mmchconfig release=LATEST command, see mmchconfig command.
- Support 4K disk sectors for IBM Spectrum Scale on Windows
- Introduced 4K disk sector support for IBM Spectrum Scale on the Windows platform. Removed the restriction from the IBM Spectrum Scale FAQ. For more information about the updated FAQ question, see IBM Spectrum Scale support for 4K disk sectors.
- Repair functionality of mmfsck command in online mode is deprecated
- The repair functionality of mmfsck command in online mode is no longer available. The report-only operation still works in the online mode. For more information, see mmfsck command.
- --metadata-block-size parameter of mmcrfs command is being deprecated
- The --metadata-block-size parameter of the mmcrfs command can be used to specify a metadata block size that is different from the data block size. This option is being deprecated because it is no longer required to use for performance improvements for file systems with file system format 5.0.0 or later. This parameter will be removed in a future release. For more information, see mmcrfs command.
- Display size of the LROC devices in the mmlsnsd command output
- The mmlsnsd -L command output now shows the Size (MB) column for LROC devices. For more details, see mmlsnsd command.
- File creation performance improvement
- The performance of file creation has been improved by reducing the context switch between kernel
and user space, by implementing the
atomic_open
inode operation. - mmap performance improvement
- The performance of mmap when pre-fetching file data blocks has been improved.
- DIO vectored read performance improvement
- The performance of vectored reads in direct-IO mode has been improved with more efficient memory pool buffer locking.
- New diskDown callback to inform about the reason for a disk failure
- The diskDown callback is triggered on the file system manager when the status of a disk in a file system changes to down. The diskDown callback is similar to the diskFailure callback but provides the reason for triggering the diskDown callback. For more information, see mmaddcallback command.
- Support for IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.0.1 (IF01)
- IBM Spectrum Scale 5.1.2 supports IBM Security Guardium Key Lifecycle Manager (GKLM) 4.1.0.1 (IF01) or later. The older versions of GKLM are referred to as IBM Security Lifecycle Manager or SKLM in the documentation. The configuration information is the same for both GKLM and SKLM. For more information, see mmkeyserv command.
- Clear QoS configuration automatically when system objects are deleted
- You do not need to use the special option QOSSkipValidate=1 to override the
QoS configuration validation. The following commands automatically clean up QoS data when those
system objects are deleted:
- mmdelfileset (filesets)
- mmdelnode (nodes)
- mmdelnodeclass (node classes)
- mmauth (remote cluster names)
- mmdeldisk (pools)
- O_TMPFILE support (Kernel version greater than 3.15 required)
- IBM Spectrum Scale 5.1.2 now supports the use of the O_TMPFILE open flag. The O_TMPFILE is used in the open(2) call. This allows the creation of an unnamed temporary regular file. Anything written to the resulting file is lost when the last file descriptor is closed unless the file is given a name. To link the temporary file, linkat(2) or gpfs_linkat() calls can be used.
- gpfs_linkat API (Linux only)
- The gpfs_linkat API is defined in the gpfs.h
header file. The gpfs_linkat() function links a file to a directory name,
and atomically replaces the destination if the file exists. The
gpfs_linkat API has a similar interface to the
linkat(2) system call with the following differences:
- When a new path specifies an existing file, it is atomically replaced; EEXIST is never returned.
- AT_EMPTY_PATH does not require CAP_DAC_READ_SEARCH.
- Fine-Grained Directory Locking (FGDL) for link
- When concurrent hard links are created in a single directory from multiple nodes, FGDL is enabled to improve performance of link creation.
- New parameters to retain the user changes in mmsysmon.conf parameters after an RPM update
- Introduced a set of parameters in the mmchconfig command to ensure that the changes in some of the mmsysmon.conf parameters do not disappear after an RPM update. For more information about how to set value for these parameters, see mmchconfig command.
- Installation toolkit changes
-
- Support for Ubuntu 22.04 on x86 starting IBM Spectrum Scale 5.1.2.5.
- Support for Red Hat Enterprise Linux 8.5 on x86_64, PPC64LE, and s390x.
- Support for user-defined profiles of GPFS configuration parameters. For more information, see spectrumscale command and Setting configuration parameters before installation.
- Support for populating cluster state information from mixed CPU architecture nodes. The information is populated from nodes that have the same CPU architecture as the installer node. Information for nodes of a different CPU architecture is not populated.
- Support for IBM Spectrum Scale Erasure Code Edition on IBM Z®.
- Faster upgrades due to optimizations in the upgrade path.
- Management API changes
- The following endpoints are added:
- POST Filesystems/{filesystemPath}/loopbackDevice/attach/ {fileName}/{nodeName}
- DELETE Filesystems/{filesystemPath}/loopbackDevice/detach/{fileName}/{nodeName}
- GET Filesystems/{filesystemPath}/loopbackDevice/{fileName}/{nodeName}
- GET filesystems/{filesystemName}/directory/{path}
- Management GUI changes
- Total requests written to and read from remote systems can be viewed and sorted in the GUI performance metric table on the Active File Management page.
- SMB changes
-
Removed the 'The CTDB nodes file for CES' name from the IBM Spectrum Scale other configurations list mentioned underneath the sub-section Updating IP addresses or host names in other configurations.
Replaced option '--all' with '-a' that is used in mmces service command statement to failback or restore the SMB protocol configuration. For more information see, Failback or restore steps for the SMB protocol.
- NFS changes
-
- Upgraded CES NFS-Ganesha to NFS V3-stable version. This upgrade adds more stability, fixes defects, and improves the NFS performance.
- Updated limitations for NFS Ganesha protocol. For more information, see General CES NFS Linux limitations.
- Introduced configuration parameters:DIR_CHUNK, READDIR_RES_SIZE, and SERVER_SCOPE. For more information, see mmnfs command.
- Protocols authentication changes
- IBM Spectrum Scale protocol authentication uses a custom keytab file instead of a system keytab file. This helps to remove conflicts with other applications that use a system keytab file, for example, sssd. For more information, see mmuserauth command.
- Object protocol changes
- On IBM
Spectrum Scale 5.1.2.0, installation or upgrade
of the object protocol requires the preconfiguration of OpenStack repositories on all protocol
nodes. For more information, see OpenStack repository configuration required by the object protocol.Note: This repository configuration is not required in IBM Spectrum Scale 5.1.2.1 and later releases.
- Packaging changes
-
- The gpfs.kafka package is removed from the IBM Spectrum Scale installation images for Linux.
- Performance monitoring changes
-
From IBM Spectrum Scale 5.1.2, a security feature is introduced to limit the connections of performance monitoring sensors to a list of known IBM Spectrum Scale node IPs. For more information, see Configuring the performance monitoring tool.
- Python-related changes
- From IBM Spectrum
Scale release 5.1.0, all Python code in
the IBM Spectrum
Scale product is converted to Python 3. The
minimum supported Python version is 3.6.
For compatibility reasons on IBM Spectrum Scale 5.1.0.x and later on Linux 7.x (7.7 and later), a few Python 2 files are packaged and they might trigger dependency-related messages. In certain scenarios, Python 2.7 might also be required to be installed. Multiple versions of Python can co-exist on the same system. For more information, see the entry about mmadquery in Guidance for Red Hat Enterprise Linux 8.x on IBM Spectrum Scale nodes.
The Python code in IBM Spectrum Scale 5.0.y or earlier continues to be in Python 2.
Tip:- IBM Spectrum Scale 5.1.x.x uses Python 3 code, and it runs best with operating systems that also use Python 3 internally such as Linux 8.x, SLES 15, and Ubuntu 20.04.
- IBM Spectrum Scale 5.0.x.x uses Python 2 code, and it runs best with operating systems that also use Python 2 internally, such as Linux 7.x.
- System health changes
-
- Added new IP addresses for esupport.ibm.com to consider during firewall configurations. For more information, see Firewall recommendations for call home and Installing call home.
-
Added a new option
--temporary
to the mmhealth command. For more information, see the mmhealth command. -
Added a new component GDS in the parameter list of the mmhealth node show and mmhealth cluster show commands. For more information, see the mmhealth command.
For more information, see GPUDirect Storage support for IBM Spectrum Scale
- Added a new component,
HEALTHCHECK
, to improve the experience of users with Call Home capability by providing them a direct support from IBM. For more information, see Proactive system health alerts. -
New API sub-command added to the mmperfmon delete command, which can be used to delete expired API keys. For more information, see mmperfmon command and Configuring performance monitoring API keys.
- New events added for the following:
- Call Home events
- CES network events
- File Audit Logging events
- File system events
- GPFS events
- GUI events
- Local Cache events
- Network events
- Watchfolder events
For more information, see Events
- Call home changes
- Call home has the following improvements:
- New commands are added to scheduled data upload list. For more information, see Scheduled data upload.
- Linux on Z changes
-
The following changes are made to the IBM Spectrum Scale Erasure Code Edition on IBM Z:
-
Support for the IBM Spectrum Scale Erasure Code Edition on IBM Z15 server.
-
Support for four storage nodes that are running in native LPARs.
-
Support for up to three NVMe drives with a U.2 form factor per storage node.
-
Support for HiperSockets network connection.
-
Automatic mapping from the server’s drive slots to PCIe buses. There is no need to manually create an Enclosure Descriptor File (EDF).
For more information, see IBM Spectrum Scale ECE 5.1.2.
-
- Commands, data types, and programming APIs
- The following section lists the modifications to the documented commands, structures, and subroutines:
- New commands
- mmxcp
- New structures
- There are no new structures.
- New subroutines
- gpfs_linkat()
- New user exits
- There are no new user exits.
- Changed commands
-
- gpfs.snap
- mmlsquota
- mmchfs
- mmaddcallback
- mmafmconfig
- mmafmcosconfig
- mmafmctl
- mmaudit
- mmbackup
- mmchcluster
- mmchconfig
- mmchfileset
- mmchfs
- mmcrcluster
- mmcrfs
- mmfsck
- mmgetstate
- mmhealth
- mmkeyserv
- mmlsnsd
- mmnetverify
- mmnfs
- mmperfmon
- mmqos
- mmsetquota
- mmsdrrestore
- mmuserauth
- mmxcp
- spectrumscale
- Changed structures
- There are no changed structures.
- Changed subroutines
- gpfs_fcntl()
- Deleted commands
- mmcesdr
- Deleted structures
- There are no deleted structures.
- Deleted subroutines
- There are no deleted subroutines.
- Messages
- The following are the new, changed, and deleted messages:
- New messages
- 6027-1829, 6027-1830, 6027-2417, 6027-2418, 6027-2419, 6027-3265, 6027-3335, 6027-4303,6027-4304, 6027-4305, 6027-4306, 6027-984, 6027-3411, 6027-3412, 6027-3413, 6027-3516, 6027-3951, 6027-3952, 6027-3953, 6027-4020, 6027-4021, 6027-4022, 6027-4023, 6027-4024, 6027-4212, 6027-4307, 6027-4308, 6027-4309, 6027-4310, and 6027-4311
- Changed messages
- 6027-475, 6027-477, 6027-1311, 6027-1312, 6027-1323, 6027-1324, 6027-1325, 6027-2050, 6027-3245, 6027-3263, 6027-3265, and 6027-3328
- Deleted messages
- 6027-1922
- Documentation changes
-
- List of documentation changes in product guides and respective IBM Documentation sections
-
The following is a list of documentation changes including changes in topic titles, changes in placement of topics, and deleted topics:
Table 1. List of changes in documentation Guide IBM Documentation section List of changes Problem Determination Guide Monitoring Applied the following changes:
-
Re-arranged the list of first three chapters:
- Monitoring system health by using IBM Spectrum Scale GUI
- Monitoring system health by using the mmhealth command
- Performance monitoring
- Changed following section headings:
- Set up event notifications to Event notifications
- Monitoring tip events to Monitoring tips by using GUI
- Monitoring thresholds to Monitoring thresholds by using GUI
Troubleshooting Removed the following topics: - Protocols cluster disaster recovery issues
- Message queue events
Message number changes:- 6027-4305 changed to 6027-4304
- 6027-4306 changed to 6027-4305
- 6027-4307 changed to 6027-4306
Administration Guide Configuring - Changed the topic title from Configuration parameters for AFM to Configuration parameters for AFM and AFM-DR.
- Merged all AFM and AFM-DR configuration parameters tables into a single table.
- Removed the following topics:
- Parallel data transfer configuration parameters for AFM
- Configuration parameters for AFM-based DR
- Parallel data transfer configuration parameters for AFM-based DR
Administering - Removed the following topics:
- Creating the inband disaster recovery setup
- Creating the outband disaster recovery setup
- Performing failover for protocols cluster when primary cluster fails
- Performing failback to old primary for protocols cluster
- Performing failback to new primary for protocols cluster
- Backing up and restoring protocols and CES configuration information
- Updating protocols and CES configuration information
Command and Programming Reference Guide Command reference Removed mmcesdr command. Programming reference Added the following API documents: - PUT health/config/monitor/resume/{server}
- PUT health/config/monitor/pause/{server}
Concepts, Planning, and Installation Guide Product overview Changed the topic title from File audit logging events' descriptions to File audit logging events.
-