Recovery From ENOMEM Error When Copying Files Larger Than 32K (.indirect segment pblm)

Answer

• Environment
• Background
• The problem and the solution
• Recommended fixes

---

Environment

APAR IY13763 was created to correct the error and is intended for JFS only and no other filesystem types (JFS2, AFS, GPFS, etc). This fix applies to Standard/Large/Compressed JFS types on systems running AIX 4.3.3. At AIX 5.1 JFS and JFS2 filesystems by default are not affected by this problem.

---

Background

Here is a brief overview of the journaled file system geometry:

• JFS uses direct, single indirect, and double indirect disk addressing. The disk inode contains 9 slots for disk addresses. The first 8 are used for the direct addressing, while the ninth is used for indirect addressing.

• Direct addressing

  If the file is les than or equal to 32K, then the file will use direct addressing, which means the inode will point directly to the data blocks.

• Single Indirect addressing

  If the file is too larger for direct addressing, it will use a form of indirect addressing. Single indirect addressing is called when the file's data blocks can be addressed by a single indirect block. In this case, the direct block addresses are unused and the indirect block address contains the disk address of the one and only indirect block. Each 4 byte entry in the indrect block refers to a data block; therefore, single indirect addressing is used for files greater than 32K yet less than 4MB.

• Double Indirect addressing

  If the file is larger than 4 MB, double indirect addressing is used. In this case, the indirect field contains the block number of the double indirect block for the file. The double indirect block contains 512 pointers to the indirect blocks. As stated before, each indirect block contains 1024 block addresses for the data for the file.

  For more in-depth coverage of this subject, the following IBM publications are recommended:

  • AIX Logical Volume Manager, from A to Z: Introduction and Concepts
  • AIX V4 documentation library

---

The problem and the solution

The problem with the ENOMEM error occurs when the system has reached an architectural limit in AIX. The data read from the indirect blocks of the filesystem are cached in an indirect segment of virtual memory. Once the indirect data has been cached, it remains in virtual memory until the filesystem is unmounted. Once this segment is full, there is no more room to cache data and the result is the 'out of memory' error message.

Note that the problem is with the indirect data and since a file that is less than 32K does not use indirect blocks, these smaller files are not affected by the problem.

A temporary work-around for this problem is unmounting and remounting the filesystem, which clears out the .indirect cache. Here are the procedures for correcting the problem: APAR IY13763 (orginally IY11431) increases the number of 256MB indirect segments from one to eight. If this is not already applied to the system, it can either be downloaded from "Quick links to AIX fixes" website or ordered.

First check to see if you may already have this APAR installed.

# instfix -ik IY13763 

If it is there you will receive the informational message:

All filesets for IY13763 were found 

If you do receive this message, proceed with the procedures provided below. If you do not receive this message, first apply the fix and then complete the following steps for the filesystem(s) affected:

1. # umount /fsname 
   

2. # chfs -a options=rw,mind /fsname 
   

   This step will not only update /etc/filesystems, but will place the information in the LVCB (logical volume control block). The "mind" (Multi INDirect) option, will allow that filesystem to use the eight .indirect segments instead of one.

3. # mount /fsname 
   

4. Verify the filesystem is mounted with the correct options:

   # mount 
   

   You should see something similar to the following:

    
   /dev/lvname      /fsname               jfs    Jun 27 14:40 
   rw,mind,log=/dev/loglv05 
   

Other than implementing these procedures, updating the operating system to 5.1 is the only other option.

---

Recommended fixes

APAR             DESCRIPTION                            AIX LEVEL 
IY13763          MULTI-SEGMENT .INDIRECT FOR JFS        4.3.3.