Packaging and deployment performance

Edit online

You might have to change the way that packaging and deployment are done to improve their performance. Such is the case if large data sets or members are packaged and deployed.

Optional properties are provided to tailor the packaging and deployment process. However, using these properties in a normal packaging situation might decrease performance. So, it is wise to run some benchmark testing first.

You can use two main options to adapt the packaging and deployment process:
  • Use extended-format data sets for the large data sets and specify additional allocation parameters such as BUFNO and MAXVOL.
  • Use the extended UNIX cp command instead of XMIT, OCOPY, and RECEIVE. If you use the cp command, you must use it both for the packaging and the deployment.

Package definitions

The following table presents the optional packaging properties that you can specify in the Properties tab of the Package Definition editor.
Property Description
_ALLOCATION_PARMS_PDS Additional allocation parameters for temporary PDS allocation. For example, you can enter the following parameters: 
BUFNO(55) DATACLASS(VSAMEXT) MAXVOL(32)
_ALLOCATION_PARMS_SEQ Additional allocation parameters for the allocation of temporary sequential files. For example, you can enter the following parameters: 
BUFNO(55) DATACLASS(VSAMEXT) MAXVOL(32)
_TEMPORARY_PREFIX Prefix for temporary file allocations. It is like the temporary prefix that is used in deployment. However, as it is not yet available through the editor, it can be specified as a property. For example, you can enter the following prefix: DOHERTL.TEMPDSN
debug Debug messages. If you set this property to true, additional debug messages about the timing of the various steps of the process will be generated. If you set it to false, no additional debug messages will be generated.
useEXTPREF Use of the extended format. If you set this property to true, DSNTYPE(EXTPREF) will be used for the allocations of temporary PDS and temporary sequential files.

See Use of DSNTYPE(EXTPREF).
useCP Use of the improved z/OS® cp command. If you add this property, you do not use the XMIT, OCOPY, and RECEIVE commands. Instead, you use the cp command to copy load modules into the z/OS file system (zFS) and out again, as it keeps the ALIAS and other important information.
cpBinaries Low-level qualifiers to be transferred as binary files. If you add this property, you must specify a list of data set low-level qualifiers, separated by semi-colons. For example, SBLZPAX contains PAX files that need to be transferred as binary files.
_STDERR_LINES Number of error message lines to display. You can specify a number to limit the number of lines that are produced by the cp command. For example, if you set this property to 50, STDERR is limited to 50 lines.

As cp calls the binder, some binder warning messages might not be relevant.

Deployment definitions

The following table presents the optional deployment properties that you can specify in the Properties tab of the Deployment Definition editor. They will be used in the backup phase and the deployment phase.
Property Description
_ALLOCATION_PARMS_PDS Additional allocation parameters for temporary PDS allocation. For example, you can enter the following parameters: 
BUFNO(55) DATACLASS(VSAMEXT) MAXVOL(32)
_ALLOCATION_PARMS_SEQ Additional allocation parameters for the allocation of temporary sequential files. For example, you can enter the following parameters: 
BUFNO(55) DATACLASS(VSAMEXT) MAXVOL(32)
_ALLOCATION_PARMS_DEP Additional allocation parameters for physical PDS allocations in the BLZDEPZP program.
_TEMPORARY_PREFIX Prefix to be used for temporary file allocations in the BLZBKPZP program.

The temporary prefix that is specified in the z/OS Deployment tab is not passed through to BLZBKPZP. So, if you want to use the temporary prefix in BLZBKPZP, you must specify the _TEMPORARY_PREFIX property. In the deployment, the value that is specified in the editor overrules this property.

For example: DOHERTL.TEMPDSN
debug Debug messages. If you set this property to true, additional debug messages about the timing of the various steps of the process will be generated. If you set it to false, no additional debug messages will be generated.
useEXTPREF Use of the extended format. If you set this property to true, DSNTYPE(EXTPREF) will be used for the allocations of temporary PDS and temporary sequential files.

See Use of DSNTYPE(EXTPREF).
useCP Use of the improved z/OS cp command. If you add this property, you do not use the XMIT, OCOPY, and RECEIVE commands. Instead, you use the cp command to copy load modules into the z/OS file system (zFS) and out again, as it keeps the ALIAS and other important information.
cpBinaries Low-level qualifiers to be transferred as binary files. If you add this property, you must specify a list of data set low-level qualifiers, separated by semi-colons. For example, SBLZPAX contains PAX files that need to be transferred as binary files.
_STDERR_LINES Number of error message lines to display. You can specify a number to limit the number of lines that are produced by the cp command. For example, if you set this property to 50, STDERR is limited to 50 lines.

As cp calls the binder, some binder warning messages might not be relevant.

Use of DSNTYPE(EXTPREF)

If large data sets are to be packaged and deployed, then using DSNTYPE(EXTPREF) can give large performance improvements. However, it changes how the packaging and deployment were originally designed to work. The packaging uses the XMIT command to create the object to be deployed. The advantage is that the DCB attributes of the data set that is deployed do not need to be stored anywhere. The RECEIVE command that is run at deployment time will create the data set, if it does not already exist, with the same DCB attributes as the packaged data set.

However, if DSNTYPE(EXTPREF) is specified on the temporary internal PDS, when a specific list of members or a list of member patterns is packaged, then the temporary internal PDS is allocated as a Variable Spanned file (RECFM=VS). Such an allocation, which is valid for IEBCOPY, gives the most performance improvement. However, at deployment time, when the sequential XMIT file is received, it will create a RECFM=VS file. If the deployment to data set (second column of the table in the z/OS Deployment tab) already exists, there is no problem, as IEBCOPY will copy from the RECFM=VS to the deployment to data set. However, if the deployment to data set does not exist, then the deployment will allocate the data set as it was received, RECFM=VS, which is not correct.

As such, packaging and deployment will store the DCB information of the data set being deployed, in the generated manifest. So, if the deployment to data set does not exist, and if the useEXTPREF=true property is set, then the deployment will use the information in the manifest to allocate the deployment to data sets.

<?xml version="1.0"?>                                                                                                                                    
<manifest type="MANIFEST_LIST">                                                                                                                          
  <container name="DOHERTL.TEST.LOAD" type="PDS" DSNTYPE="LIBRARY" DSORG="PO" RECFM="U" LRECL="0" BLKSIZE="21504" UNIT="BLOCK" PRI="383232" SEC="653889" >
    <resource name="HS011" type="PDSMember" lastModifiedTimestamp="2021/03/03 06:52:27"/>                                                             
    <resource name="HS034" type="PDSMember" lastModifiedTimestamp="2021/03/03 13:34:33"/>                                                                
    <resource name="HS040" type="PDSMember" lastModifiedTimestamp="2021/03/03 04:41:12"/>                                                                
    <resource name="EX358" type="PDSMember" lastModifiedTimestamp="2021/03/03 00:51:51"/>                                                                
    <resource name="CM013" type="PDSMember" lastModifiedTimestamp="2021/03/03 06:28:05"/>                                                                
    <resource name="CN101" type="PDSMember" lastModifiedTimestamp="2021/03/03 01:20:25"/>                                                                
    <resource name="CN110" type="PDSMember" lastModifiedTimestamp="2021/03/03 05:31:46"/>                                                                
    <resource name="BM199" type="PDSMember" lastModifiedTimestamp="2021/03/02 22:28:05"/>                                                                
    <resource name="BR001" type="PDSMember" lastModifiedTimestamp="2021/03/03 06:19:04"/>                                                                
    <resource name="AV201" type="PDSMember" lastModifiedTimestamp="2021/03/03 05:53:30"/>                                                                
    <resource name="KF243" type="PDSMember" lastModifiedTimestamp="2021/03/03 06:15:26"/>                                                                
    <resource name="KO017" type="PDSMember" lastModifiedTimestamp="2021/03/03 06:20:03"/>                                                                
    <resource name="KO024" type="PDSMember" lastModifiedTimestamp="2021/03/03 06:18:27"/>                                                                
    <resource name="KO109" type="PDSMember" lastModifiedTimestamp="2021/03/03 06:06:24"/>                                                                
    <resource name="KO125" type="PDSMember" lastModifiedTimestamp="2021/03/03 04:22:12"/>                                                                
    <resource name="KP055" type="PDSMember" lastModifiedTimestamp="2021/03/03 01:29:23"/>                                                                
    <resource name="KP087" type="PDSMember" lastModifiedTimestamp="2021/03/03 03:59:48"/>                                                                
    <resource name="KP171" type="PDSMember" lastModifiedTimestamp="2021/03/03 13:39:37"/>                                                                
    <resource name="KP187" type="PDSMember" lastModifiedTimestamp="2021/03/03 01:09:57"/>                                                                
    <resource name="KP267" type="PDSMember" lastModifiedTimestamp="2021/03/03 00:08:18"/>                                                                
  </container>                                                                                                                                           
</manifest>
Note:
  • You can copy all the members in a data set by specifying an asterisk (*) in the package definition. If the useEXTPREF property is not specified and smaller packages are being processed, then it is slightly faster than if you specify the useEXTPREF property.
  • If BUFNO is specified, it needs to be monitored. If its assigned value is too high, then s878-10 abends might occur because the required storage was not available.

Using cp versus XMIT, OCOPY, and RECEIVE

Optionally, packaging and deployment can use the new cp support available with z/OS.

As the cp command works only with zFS, parameters such as EXTPREF and MAXVOL do not need to be supported. However, the cp command requires the specification of the useCP and cpBinaries properties and accepts the _STDERR_LINES property.

To copy a load module, the cp command calls the binder. No relinking occurs but the content of the load module changes. These changes do not impact the running of the load module. However, as the binder is called during the copy processing, some binder warning messages might appear. These messages are the messages that you would have seen during the initial link edit of the load module. The following lines are message examples:

IEW2609W 5104 SECTION UTP336G USABILITY ATTRIBUTE OF REUSABLE CONFLICTS WITH REQUESTED USABILITY OF REENTRANT.

IEW2646W 5383 ESD RMODE(24) CONFLICTS WITH USER-SPECIFIED RMODE(ANY) FOR SECTION AP701TW CLASS B_TEXT.

The size of the load module is identical in the original data set and in the deployed data set. So, it is a good indication that the module is the same. An AMBLIST command can be run to further compare contents. See AMBLIST in the z/OS documentation.

The first set of tests compared the timings of XMIT, OCOPY, and RECEIVE with cp for a load library that was allocated 34,000 tracks. The library contained 20 large load modules whose sizes ranged from 00333A08 to 09C77D48.
  • Timings for XMIT, OCOPY, and RECEIVE:
    • Packaging of 20 large load modules: 5 minutes 14 seconds
    • Deployment of 20 large load modules: 11 minutes 23 seconds
  • Timings for cp:
    • Packaging of 20 large load modules: 2 minutes 4 seconds
    • Deployment of 20 large load modules: 5 minutes 4 seconds
The second set of tests compared the timings of XMIT, OCOPY, and RECEIVE with cp for more normal-sized members and data sets. The shipped Engineering Workflow Management (EWM) data sets were used for testing. Because the SBLZPAX library is large (96,000 tracks), two tests were conducted, with and without it.
  • Test 1: Packaging and Deployment tests without the SBLZPAX library. Tests showed that for normal-sized data sets, the cp command was slower than XMIT, OCOPY, and RECEIVE.
    • Packaging with XMIT and OCOPY (1381 objects): 1 minute 4 seconds
    • Deployment with OCOPY and RECEIVE (1381 objects): 2 minutes 6 seconds
    • Packaging with cp (1381 objects): 1 minute 28 seconds
    • Deployment with cp (1381 objects): 3 minutes 52 seconds
  • Test 2: Packaging and Deployment tests without the SBLZPAX library. Tests showed that for a package with large-sized data sets, the cp command was much faster than XMIT and OCOPY.
    • Packaging with XMIT and OCOPY (1524 objects): 17 minutes 28 seconds
    • Deployment with OCOPY and RECEIVE (1524 objects): 21 minutes 57 seconds
    • Packaging with cp (1524 objects): 4 minutes 20 seconds
    • Deployment with cp (1524 objects): 13 minutes 36 seconds