IC66156: CONVERSION FROM IDS 10 TO 11.50.XC6 ADDS MANY SHARED MEMORY SEGMENTS AFTER CHECKING FOR FREE SPACE IN PARTITION HEADER PAGES

APAR status

• Closed as program error.

Error description

• When upgrading from IDS 10 to IDS 11.50.xC6, the conversion
  process begins dynamically allocating new shared memory segments
  immediately after the following check for free space in
  partition header pages as noted in the online.log.  Depending
  upon how many chunks are allocated to the instance being
  converted, this can result in IDS running out of shared memory
  and crashing before the conversion completes.
  
  The number of segments allocated seems to correspond to the
  number of chunks defined in the instance.  The larger the number
  of chunks defined, the more likely the conversion will run out
  of shared memory.  In testing, even a box with large memory
  resources ran out of shared memory when converting an instance
  with 2047 chunks.
  
  By comparison, the same conversion process to IDS 11.50.xC5W4,
  with the same 2047 chunk instance, jumps right from the check
  for free space in partition header pages into conversion control
  block initialization, without adding any new segments.
  
  Example of the 11.50.xC6 online.log output during the
  conversion:
  
            12:05:02  Conversion from version 10.00 Started
            12:05:20  Checking partition header pages for needed
  free space
            12:05:20  Checking for free space in partition header
  pages succeeded
            12:05:20  Segment locked: addr=c000000033400000,
  size=8388608
            12:05:20  Dynamically allocated new virtual shared
  memory segment (size 8192KB)
            12:05:20  Memory sizes:resident:15968 KB,
  virtual:50896 KB, no SHMTOTAL limit
            12:05:20  Segment locked: addr=c000000033c00000,
  size=8388608
            ...
            ...
            ...
            12:05:30  Memory sizes:resident:15968 KB,
  virtual:2434768 KB, no SHMTOTAL limit
            12:05:30  Segment locked: addr=c0000000c5400000,
  size=8388608
            12:05:30  Dynamically allocated new virtual shared
  memory segment (size 8192KB)
            12:05:30  Memory sizes:resident:15968 KB,
  virtual:2442960 KB, no SHMTOTAL limit
            12:05:30  shmat: [EMFILE][24]: out of shared memory
  segments, check system SHMSEG
            12:05:30  out of virtual shared memory
  
            12:05:30  oninit: Fatal error in shared memory
  initialization
  
  
  By comparison, here is the same section of the online.log in
  11.50.FC5W4:
  
            13:58:09  Conversion from version 10.00 Started
            13:58:27  Checking partition header pages for needed
  free space
            13:58:27  Checking for free space in partition header
  pages succeeded
            13:58:27  Conversion control block initialization
            13:58:27  Succeeded
            13:58:27  Initialization of Conversion Session
            ...
            ...
            ...
            13:59:38  Conversion Completed Successfully
  
  The test for this difference in 11.50.FC6 behavior is fairly
  straightforward.  Create an IDS 10 instance with a 2 GB cooked
  file and run a script to create 2047 chunks in that single file,
  such as:
  
  onspaces -c -d dbsp0001 -p /tmpdata/informix/cooked_files/dbsp1
  -o 0 -s 1000
  onspaces -c -d dbsp0002 -p /tmpdata/informix/cooked_files/dbsp1
  -o 1000 -s 1000
  onspaces -c -d dbsp0003 -p /tmpdata/informix/cooked_files/dbsp1
  -o 2000 -s 1000
  onspaces -c -d dbsp0004 -p /tmpdata/informix/cooked_files/dbsp1
  -o 3000 -s 1000
  ...
  ...
  onspaces -c -d dbsp2044 -p /tmpdata/informix/cooked_files/dbsp1
  -o 2043000 -s 1000
  onspaces -c -d dbsp2045 -p /tmpdata/informix/cooked_files/dbsp1
  -o 2044000 -s 1000
  onspaces -c -d dbsp2046 -p /tmpdata/informix/cooked_files/dbsp1
  -o 2045000 -s 1000
  onspaces -c -d dbsp2047 -p /tmpdata/informix/cooked_files/dbsp1
  -o 2046000 -s 1000
  onspaces -c -d dbsp2048 -p /tmpdata/informix/cooked_files/dbsp1
  -o 2047000 -s 1000
  
  Steps to run the test:
  1. Create 2047 chunks in the IDS 10 instance.
  2. Create a backup of the instance for later restore back to IDS
  10
  3. Shutdown the instance.
  
  4. Start the upgrade of the instance using IDS 11.50.FC6
  software.
  5. Instance will immediately begin allocating several shared
  memory segments, and may run out of shared memory and crash.
  
  6. Restore the IDS 10 instance from backup.
  7. Start the upgrade of the instance using IDS 11.50.FC5W4
  8. Note that this upgrade will complete successfully.  There may
  be a couple of shared memory segments added here or there during
  the process, but not at all near the number seen in FC6.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IDS users with lots of chunk migrating to 11.50xC6           *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * When there are many chunks, the engine allocates lots of     *
  * memory during conversion if conversion guard is turned on.   *
  * For 2048 chunks on 4K pagesize platforms, the engine         *
  * allocates 4GB memory.  On 2K pagesize platforms, the engine  *
  * allocates 2GB memory.                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Upgrade to IDS 11.50.xC7 and above.                          *
  ****************************************************************
  

Problem conclusion

• This problem is fixed to allocate less memory during conversion
  in IDS 11.50.xC7.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  IBM IDS ENTRP E

• Fixed component ID

  5724L2304

Applicable component levels

• RB15 PSY

     UP