IBM Support

Performance - File OPEN and CLOSE Operations Causing a High Volume of Disk Activity

Troubleshooting


Problem

This document provides an explanation of a high volume of disk activity during file OPEN and CLOSE operations.

Resolving The Problem

File full open and close operations are expensive. And, the last job on the system to close an update-capable file also incurs the additional expense of disk I/O related to writing changed memory pages to DASD. Any changed pages of the file or related access paths that are in memory at the time of the last close are forced to DASD. Consider the following two scenarios:

1.

In this scenario, a batch job repeatedly opens, updates, and closes a file. It is the only job that uses the file. This job will always be the last closer; thus, every close operation can result in synchronous writes due to writing change file and access path pages to DASD.
2. In this scenario, many jobs repeatedly open, update, and close a file. At any time, several jobs might have the file open at the same time. However, with the correct timing, some job will eventually be the last one to close the file. And, when that happens, that job incurs synchronous writes due to writing changed file and access path pages to DASD. And, depending on the timing and environment, the number of pages could be significant. If for example, the other jobs have caused many updates before this last close operation, and those changes are still in memory, many pages might need to be written to DASD.
Writing a simple never-ending program that runs concurrently with an existing job that repeatedly opens, updates, and closes a file, can improve the performance of the existing job or jobs. No programming changes are required to the existing application.

Here is an example call stack of a job that was the last closer. In this example, RPG is closing the file. The call stack shows that changed physical file pages (#dbfords) and the changed access path pages (#dbforix) are being forced from memory and written to DASD. The last closer causing changed pages to be forced to DASD is not specific to RPG.
 
Call  level  Program Module Offset Procedure
---------- ------- ------ ------ ---------
003 000000B8 wait__21SmSarAsyncTrackerWaitFv
004 00000200 forceChangesSynchronously__8IxRadix4F23IxExtentOfBasePageForcei13IxForceReason
005 00000090 ixsyncforcechanges
006 000008E8 #dbforix
007 00000F7C #dbfords
008 000015B4 #dbxdacr
009 000005D0 #dbdcr
010 000000EC #cfmir
011 0000012C syscall_A_portal
012 QDMCLOSE 00002580
013 000001D0 cblabranch
014 000000C4 aiuser_program_call_portal
015 QRNXIO QRNXOCIO 00000214 _QRNX_CLOSE


Here are two programs DRIVER and WRITE that might be comparable to an existing application in that it repeatedly opens, updates, and closes a file:
  • The DRIVER program calls the WRITE program 10,000 times. The WRITE program opens a file, adds a record, closes the file, then returns.
  • The library/QAPMSYSTEM file is just an empty copy of the QMPGDTA/QAPMSYSTEM file that has been journaled with journal cache *YES.

DRIVER program to call WRITE program 10000 times:                                                                      
             PGM                                                              
             DCL       VAR(&LOOP) TYPE(*DEC) LEN(6 0)                        
             OVRDBF    FILE(QAPMSYSTEM) TOFILE(library/QAPMSYSTEM) +          
                           OVRSCOPE(*JOB)                                      
             PGMLOOP:                                                        
                       CALL library/WRITE                                  
             CHGVAR    VAR(&LOOP) VALUE(&LOOP + 1)                          
             IF        COND(&LOOP *LT 10000) THEN(GOTO CMDLBL(PGMLOOP))      
             ENDPGM    
                                                     

Program WRITE to add records:
FqapmsystemUF A E             disk    block(*no) usropn                    
C                   open      qapmsystem                                    
C                   write     QAPMSYSM                                      
C                   close     qapmsystem                                    
C                   return                                                  
C                   update    qapmsysm
                                     

Here is the new program, UPDATE, that could be used to improve the performance of the application mentioned previously. The program must open the file for update, read a record, update a record, and then stay active to keep the file open. In the following example, the first record is read and written with no changes. The record lock is released at the time that the record is updated. The idea is that the program keeps the file open while sitting on the DSPLY. Additionally, there are other techniques that could be used to create a never-ending program.

UPDATE program keeps file open if DSPLY is active:
FqapmsystemUF A E             disk                                          
C                   read      qapmsystem                                    
C                   update    qapmsysm                                      
C     intnum        dsply                                                    
C                   seton                                        lr          
C                   return                                                  
C                   write     qapmsysm  
           
  • The UPDATE program requires that at least one record exists in the target file. The WRITE can be called to add a single record.


Here are the results of calling the DRIVER program when no other jobs have the output file open:
 
System model #:  570                                                                          
Command: CALL PGM(library/DRIVER)                                                              
  Clock    CPU   sync reads  sync writes  async reads  async writes   EAO Index                
    sec    sec     DB   NDB     DB   NDB     DB   NDB     DB    NDB  Excp   Cnt                
 39.328  3.399     22    72  40496 30752      0     0  40298      1     0     0  
     

Here are the improved results of calling the DRIVER program when the UPDATE program is called in a separate job and is sitting on the DSPLY operation:
 
System model #:  570                                                              
Command: CALL PGM(library/DRIVER)                                                  
  Clock    CPU   sync reads  sync writes  async reads  async writes   EAO Index  
    sec    sec     DB   NDB     DB   NDB     DB   NDB     DB    NDB  Excp   Cnt  
 14.226  2.518      0    39  11243  8149      0     0  11616      0     0     0


Keywords: Database In-use Table, DB in-use, opens, closes, last updater, inuse, dbinuse, db-inuse, dbiut

[{"Line of Business":{"code":"LOB68","label":"Power HW"},"Business Unit":{"code":"BU070","label":"IBM Infrastructure"},"Product":{"code":"SWG60","label":"IBM i"},"ARM Category":[{"code":"a8m0z0000000CKdAAM","label":"Performance-\u003EDatabase Performance"}],"Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Type":"MASTER"}]

Historical Number

518725093

Document Information

More support for:
IBM i

Component:
Performance->Database Performance

Software version:
All Versions

Operating system(s):
IBM i

Document number:
634787

Modified date:
23 October 2024

UID

nas8N1013080

Manage My Notification Subscriptions