Readme and Release notes for release 3.4.0.2 (GPFS) IBM General Parallel File System 3.4.0.2 GPFS-3.4.0.2-x86_64-Linux Readme

During internal testing, a rare but potentially serious problem has been discovered in GPFS. Under certain conditions, a read from a cached block in the GPFS pagepool may return incorrect data which is not detected by GPFS. The issue is corrected in GPFS 3.3.0.5 (APAR IZ70396) and GPFS 3.2.1.19 (APAR IZ72671). All prior versions of GPFS are affected.

The issue has been discovered during internal testing, where an MPI-IO application was employed to generate a synthetic workload. IBM is not aware of any occurrences of this issue in customer environments or under any other circumstances. Since the issue is specific to accessing cached data, it does not affect applications using DirectIO (the IO mechanism that bypasses file system cache, used primarily by databases, such as DB2® or Oracle).

This issue is limited to the following conditions:

1. The workload consists of a mixture of writes and reads, to file offsets that do not fall on the GPFS file system block boundaries;
2. The IO pattern is a mixture of sequential and random accesses to the same set of blocks, with the random accesses occurring on offsets not aligned on the file system block boundaries; and
3. The active set of data blocks is small enough to fit entirely in the GPFS pagepool.

The issue is caused by a race between an application IO thread doing a read from a partially filled block (such a block may be created by an earlier write to an odd offset within the block), and a GPFS prefetch thread trying to convert the same block into a fully filled one, by reading in the missing data, in anticipation of a future full-block read. Due to insufficient synchronization between the two threads, the application reader thread may read data that had been partially overwritten with the content found at a different offset within the same block. The issue is transient in nature: the next read from the same location will return correct data. The issue is limited to a single node; other nodes reading from the same file would be unaffected.

Complete the following steps to install the fix package:

1. Unzip and extract the update package (&lt filename >.tar.gz file) with one of the following commands:
   
   gzip -d -c &lt filename >.tar.gz | tar -xvf -
   
   or
   
   tar -xzvf &lt filename >.tar.gz



2. Verify the udpate's RPM images in the directory. Normally, the list of RPM images in this directory would be similar to one of the following:
   
   GPFS update
   gpfs.base.&lt update_version >.&lt arch >.update.rpm
   gpfs.docs.&lt update_version >.noarch.rpm
   gpfs.gpl.&lt update_version >.noarch.rpm
   gpfs.msg.en_US.&lt update_version >.noarch.rpm
   
   GPFS update with GPL licensed kernel module
   gpfs.base.&lt update_version >.&lt arch >.update.rpm
   gpfs.docs.&lt update_version >.noarch.rpm
   gpfs.gpl.&lt update_version >.noarch.gpl.rpm
   gpfs.msg.en_US.&lt update_version >.noarch.rpm
   
   where
   &lt update_version &gt specifies the version number of the update you downloaded, for example, 3.1.0-7 .
   and
   &lt arch &gt specifies the system architecture, for example i386 for 32-bit System x or x86_64 for 64-bit System x.
   
   For specific filenames, check the Readme for the GPFS update by clicking the "View" link for the update on the Download tab.
   
   
3. Follow the installation and migration instructions in your GPFS Concepts, Planning and Installation Guide.

In the below instructions, node-by-node upgrade cannot be used to migrate from GPFS 2.3 to later releases. For example, upgrading from 2.3.x to 3.1.y requires complete cluster shutdown, upgrade install on all nodes and then cluster startup.

Upgrading GPFS may be accomplished by either upgrading one node in the cluster at a time or by upgrading all nodes in the cluster at once. When upgrading GPFS one node at a time, the below steps are performed on each node in the cluster in a sequential manner. When upgrading the entire cluster at once, GPFS must be shutdown on all nodes in the cluster prior to upgrading.

When upgrading nodes one at a time, you may need to plan the order of nodes to upgrade. Verify that stopping each particular machine does not cause quorum to be lost or that an NSD server might be the last server for some disks. Upgrade the quorum and manager nodes first. When upgrading the quorum nodes, upgrade the cluster manager last to avoid unnecessary cluster failover and election of new cluster managers.

1. Prior to upgrading GPFS on a node, all applications that depend on GPFS (e.g. Oracle) must be stopped. Any GPFS file systems that are NFS exported must be unexported prior to unmounting GPFS file systems. If tracing was turned on, then tracing must be turned off before shutting down GPFS as well.
2. Stop GPFS on the node. Verify that the GPFS daemon has terminated and that the kernel extensions have been unloaded (mmfsenv -u ). If the command mmfsenv -u reports that it cannot unload the kernel extensions because they are "busy", then the install can proceed, but the node must be rebooted after the install. By "busy" this means that some process has a "current directory" in some GPFS filesystem directory or has an open file descriptor. The freeware program lsof can identify the process and the process can then be killed. Retry mmfsenv -u and if that succeeds then a reboot of the node can be avoided.
3. Upgrade GPFS using the RPM command as follows:
   
   GPFS update
   rpm -U gpfs.base-&lt update_version >.&lt arch >.update.rpm
   rpm -U gpfs.docs-&lt update_version >.noarch.rpm
   rpm -U gpfs.gpl-&lt update_version >.noarch.rpm
   rpm -U gpfs.msg.en_US-&lt update_version >.noarch.rpm
   
   GPFS update with GPL licensed kernel module
   rpm -U gpfs.base-&lt update_version >.&lt arch >.update.rpm
   rpm -U gpfs.docs-&lt update_version >.noarch.rpm
   rpm -U gpfs.gpl-&lt update_version >.noarch.gpl.rpm
   rpm -U gpfs.msg.en_US-&lt update_version >.noarch.rpm
   
   
4. Check the GPFS FAQ to see if any additional images or patches are required for your Linux installation: General Parallel File System FAQs (GPFS FAQs)
    
5. Recompile any GPFS portability layer modules you may have previously compiled. The recompilation and installation procedure is outlined in the following file:
   /usr/lpp/mmfs/src/README

[October 26, 2010]

[July 30, 2010]

[April 1, 2010]

[March 31, 2010]

[December 17, 2009]

[November 9, 2009]

[November 7, 2008]

[September 11, 2008]

The update images listed below and contained in the tar image with this README are maintenance packages for GPFS. The update images are a mix of normal RPM images that can be directly applied to your system.

The update images require a prior level of GPFS. Thus, the usefulness of this update is limited to installations that already have the GPFS product. Contact your IBM representative if you desire to purchase a fully installable product that does not require a prior level of GPFS.

After all RPMs are installed, you have successfully updated your GPFS product.

Update to Version:

Update from Version:

Update (tar file) contents: