How to calculate appropriate size for the TCQ/TCX CISIZE

Body

The Transmission Control Queue (TCQ) is a VSAM file that is used to hold all Processes that have been submitted to Connect:Direct. These processes can be:

• Executing

• Queued for execution

• Held for retries

• Held for future execution

• Held in error

The Transmission Control Index (TCX) is used to point to the processes in the TCQ.

There are two guidelines when determining how you want to resize your TCQ and TCX.

• A single process must fit within 43 TCQ records.

• Ideally, the average process should fit in a single TCQ record.

NOTEA: To accommodate a 1M process in 43 CIs you need to define your TCQ with a RECORDSIZE of 24569 and a CISIZE of 24576. The largest CISIZE for the TCX is 32768 with a RECORDSIZE of 32761. The IDCAMS allocation will round to the next 512 boundary if you don’t code it that way for the CISIZE and these values are on the boundary.

NOTEB: The maximum number of processes that the TCX can index is 65,535, this is the maximum number of processes that you can have in the queue at one time. This number of indexes will fit on one track of DASD. You can allocate your TCQ in RECORDS or CYLINDERS. The RECORDS allocation will be allocate the space based on one TCQ record per CI. So if you only allocate space for 65,535 records you had better hope all of your processes fit in one CI or at least 95% of them fit one CI. To allow for processes that will not fit in one CI you need to add additional records to this count. When allocating with CYLINDERS you just take a guess and see if it will hold everything you do.

If you want to allocate your TCQ better than explained in the notes above then you will want to do the following.

The size of your TCQ and TCX is dependent on several factors in your Connect:Direct environment. The size of the processes, and how many processes you want the queue to hold at any given time. When you consider how many processes you want the queue to hold this will include processes executing, processes waiting for their turn to execute, processes waiting for connection with a remote node, processes on hold, processes held in error, and processes that are going through timer/retry.

First you need to determine what the average size of a process in your installation is. A simple process consists of a COPY step and nothing more. Experience tells us that the average process usually consist of a COPY step, an IF statement, and two (2) RUN TASK steps to execute DMNOTIFY. But a process can have more steps to it than this.

Each component of the process requires a specified number of bytes in the TCQ.

Process Header     1,616 bytes

COPY Step             1,024 bytes

RUN TASK             128 bytes

RUN JOB               176 bytes

GOTO Step              96 bytes

EXIT Step                 96 bytes

IF Statement           208 bytes

If you do not allocate the TCQ CISIZE for the average process size you can under allocate the space. It is important to determine the size of the average process so that you can properly allocate the TCQ CISIZE.  And not over or under allocate the dataset.

It is best if you define the record size to be able to hold one (1) average record. This does mean that you will be over allocated for some of your processes and under allocated for others.

Let’s assume that your average process consist of a COPY step, an IF statement, and two RUN TASK steps. The average size of your process would be:

Process Header             1,616 bytes

COPY Step                    1,024 bytes

1 – IF statement                208 bytes

2 – RUN TASK steps        256 bytes (128 * 2 = 256)

Total    3,104 bytes

You could code your TCQ CISIZE as 3104 but the IDCAMS define of the TCQ will round up the CI size to the next valid CI Size. Valid CISIZE is a multiple of 512 for CISIZEs 8K or under and a multiple of 2k for CISIZEs over 8K.  So 3104 would be rounded up to 3584 for the CISIZE.

The minimum and maximum record size is CISIZE – 7, or 3104 – 7 = 3097. Now you have your TCQ CISIZE and the record size. But if you code the CISIZE on the 512K boundary the minimum and maximum record size is 3584 – 7 = 3577.

Next you need to determine how many records you want this dataset to hold. Remember that this number is not just the processes executing, it also includes processes waiting for their turn to execute, processes waiting on a connection, processes on hold, and process going through timer/retry. You should consider how long and how many of the non executing processes sit in the TCQ before either executing, flushed, deleted, or purged. You need to consider if there are time periods or the day, week, month, or year where the number of processes in the TCQ would be larger than normal and plan for that.

Let’s assume that you have determined that you run about 3,000 processes a day. You would want to double or triple this number to allow for processes that are on hold, waiting on connection, going through timer/retry, or waiting on their turn to execute. In this case let’s double the number of records for the TCQ. So we want to have 6,000 records in our TCQ.

You use this number to determine the TCX CISIZE needed to control the desired number of records in the TCQ (It is recommended that you allocate your TCX as outlined in NOTEA and NOTEB above.). The calculation is:

Maximum number of TCQ records = ((TCX MAX Rec Size – 12) / 2) * 8

If you take the TCX CISIZE coded in the sample JCL of 1024 your maximum record would be 1017. The calculation would be:

((1017–12) / 2)*8=4012 maximum number of TCQ records that this TCX can manage.

For a TCQ that you estimate will need to hold 6,000 records this TCX is not large enough. The next 512 boundary is 1536 for the CISIZE with a record size of 1529 (CISIZE-7). When you do the calculation on this you get:

((1529-12)/2)*8=6064

A TCX CISIZE of 1536 will allow you to manage 6,064 TCQ records. With these values modify your IDCAMS define to build your TCQ and TCX to this size.

Regardless of what your average size process is it can only fit into a maximum of 43 TCQ records. Beginning with Connect:Direct for z/OS V4.7 you can have a process that is up to 1MB in size. If you think that you might have a process that would be that large you will need to define a TCQ CISIZE that would hold a 1MB process in a maximum of 43 TCQ records. Most installations will not have processes that approach this size but you need to be aware that you might have processes that would be large enough that they would not fit into 43 TCQ records of 3,584 bytes. And a 1MB process will not fit in 43 TCQ records of 3,584 bytes.

If you wanted to just be on the safe side you could code the TCQ CISIZE to be 24576. This CISIZE will hold a 1MB process. If you do code a CISIZE of this size you will have a lot of wasted space in your TCQ with an average process of 3,104 bytes, but you will be able to keep most processes in one CI. What you need to do in this case is to determine what your largest process is and how many of them would execute in basically the same time period and adjust the allocation for the TCQ CI to accommodate even your largest processes in the fewest CIs.

With a CISIZE of 3,584 bytes the largest process that you can get into 43 TCQ records is 153,112 bytes. So it is important that you determine if you have anything larger than that.

The IBM Sterling Connect:Direct for z/OS Configuration Guide in Chapter 1 (Connect:Direct for z/OS Installation Guide V5.0 Chapter 2) - Planning the Installation under the section entitled Transmission Control Queue (TCQ) and Index (TCX) (Enlarging the TCQ V4.8 & V5.0) it recommends that if you do not know what an average process size is in your installation to assume that the average process contains five (5) COPY steps. This will cover a wide range of differences in the process sizes in your installation.

Most installations have used the IDCAMS that is distributed in the SDGACNTL (CNTL) member DGACVDEF (CDVDEF). If you have been using this definition with no problem then you could continue to use it because you know that you don’t currently have a process that would not fit in a maximum of 43 TCQ records. Just remember that you might have a process in the future that would require more that 43 TCQ records at the current allocation. You should review the RECORDS parameter to ensure that it is large enough to hold the highest number of processes you expect to be in the queue at any given time.

Beginning with IBM Sterling Connect:Direct for z/OS V5.0 you have the option of keeping a copy of the process script in the TCQ to be reviewed in case you need to troubleshoot the process that was submitted. These records need to be considered when determining the space to allocate for the TCQ.

You can have two types of records in the TCQ, a process record and a PR queue record. The PR queue record is a process record that is no longer retained in the queue either by a purge command or by the successful completion of the process. The PR queue records are where the process script will be retained until there is a need of the space for new processes coming into the TCQ.

You code the initialization parameter PROCESS.RETENTION=YES if you want to retain the process scripts in case you need it for tracking or troubleshooting purposes. It is recommended that you do not set this parameter to YES until you have enlarged your TCQ if you plan to use it.

If saving the PR queue records are important to you, you need to give some consideration to adding some additional spaces to the allocation of the TCQ. Depending on how many processes you want to be able to hold in the queue at one time, if you are going to go for the maximum size CI to be prepared for a 1M process, how many of the PR queue records you want to ensure are available will determine whether you want to allocate your TCQ a little larger or not.

If you have a TCQ that is allocated for 3-5 cylinders you might want to double or triple the allocation. But if you have a TCQ that is already at 200 cylinders before you even take into account the PR queue records you might only want to add 50-100 cylinders to the allocation.

DEFINE CLUSTER                             -
    (NAME(SYS.CONNECT.PROD.TCQ)            -    
     RECORDS(6000)                         -    
     VOLUMES(LVPOOL)                       -    
     NUMBERED                              -    
     REUSE                                 -    
     RECORDSIZE(3577 3577)                 -    
     SHAREOPTIONS(3 3))                    -    
    DATA                                   -    
     (CONTROLINTERVALSIZE(3584)            -    
      NAME(SYS.CONNECT.PROD.TCQ.DATA))
   DEFINE CLUSTER                           -   
    (NAME(SYS.CONNECT.PROD.TCX)            -   
     RECORDS(2)                            -   
     VOLUMES(LVPOOL)                       -   
     NUMBERED                              -   
     REUSE                                 -   
     RECORDSIZE(1529 1529)                 -   
     SHAREOPTIONS(3 3))                    -   
    DATA                                   -   
     (CONTROLINTERVALSIZE(1536)            -   
      NAME(SYS.CONNECT.PROD.TCX.DATA))     /*

You should use the TCQ.THRESHOLD initialization parameter to warn you it the TCQ begins to get to close to being full. You can code NO for no message so that when it becomes full your C:D is stopped until some of the processes already in the queue complete to allow for continuing work to take place. You can code YES which will provide a message at 90% and every time the percentage changes while over 90%. You can also code a value between 0-90 if you prefer a lower threshold. The PR queue records do not count against the threshold.