Getting IBM Z data to import

WWR2 : Performance concepts and configuration : Importing data for performance assessments : Getting IBM Z data to import

Typically, you ask a customer's z/OS system administrator to provide storage performance data for you to assess in Storage Modeller. Performance data is in the form of RMF records and is stored in the system SMF data sets.

Note: The maximum size supported for a SORTED .TRS file is 20GB. If your file is bigger, please reduce the amount of intervals and/or LPARs collected.

To extract and prepare the necessary SMF data before sending it to you, the system administrator must perform the following steps:

1. Set up the Resource Measurement Facility (RMF) Monitor to gather data. This needs to be done only once per system.

2. Dump the relevant SMF data. Relevant record types are 74-1, 74-5, and 74-8.

3. Optionally, change the time stamps in the SMF records to a common time zone. This is necessary only if data is included from multiple z/OS systems that run in different time zones.

4. Sort the SMF records into chronological order.

5. Terse (compress) the sorted data set.

6. Transfer the compressed file to a system of your choice, so that you can access it and upload it to Storage Modeller.

The following JCL examples can be used to perform these six steps. These individual steps can be combined into a single job or run individually, whichever is most convenient. Add a JOB card to the JCL if the installation requires it.

Note: The JCL in the examples in each step can be copied to an editor and then updated to fit your solution.

1. Select the JCL inside the screenshot.

2. Right-click and select Copy.

3. Paste into an editor.

4. Update the JCL.

Step 1: Setting up RMF

The RMF is included in all z/OS installations. However, in the rare case that the RMF is disabled, the customer should contact their system administrator to enable RMF including the Monitor I session. The default gathering options for Monitor I sessions are specified in the ERBRMF00 and/or ERBRMF02 parmlib members. It might be necessary to change the default options to include CACHE and ESS(LINK,RANK). This ensures that comprehensive information is gathered for the attached storage controllers.

If at least one volume of an SSID of the z/OS host system is online at the time of data collection, RMF gathers the CACHE and ESS performance data for that SSID (that is, for that LCU/LSS of the storage system). The data is the same for a given storage system LCU/LSS regardless the z/OS LPAR from which it is gathered. Therefore, if multiple z/OS LPARs have online volumes from the same storage system LCU/LSS, it is sufficient to collect the data with the CACHE and ESS(LINK,RANK) options for only one of those LPARs. This reduces the amount of duplicate information in the RMF records and reduces the overall size of the resulting performance data file. However, if it cannot be guaranteed that at least one volume per SSID will be online at any given time, it is better to collect that data from multiple LPARs. The Storage Modeller will detect and discard duplicated data.

When the RMF Monitor I session is running with the appropriate options, the customer should let the systems gather the performance data over time.

For the best accuracy of the performance analysis, it is recommended to have at least two days and no more than seven days of data for all the target storage systems, gathered at an interval of 5 to 15 minutes, over a time period that includes the peak load on the systems. The exact time period to include in the resulting performance file can be specified in Step 2, when the SMF data is dumped.

Step 2: Dumping the SMF Data

Step 2 runs the IFASMFDP program to extract and dump the SMF records from the system data sets.

//*

//* ====================

//* DUMP SMF DATATSET(S)

//* ====================

//SMFDUMP EXEC PGM=IFASMFDP

//IDD1 DD DISP=SHR,DSN=<input_smfdata_system1>

//IDD2 DD DISP=SHR,DSN=<input_smfdata_system2>

// :

//IDDN DD DISP=SHR,DSN=<input_smfdata_systemN>

//SMFDATA DD DISP=(NEW,PASS),DSNTYPE=LARGE,

// SPACE=(CYL,(100,100),RLSE),UNIT=SYSDA,

// DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=0)

//SYSPRINT DD SYSOUT=*

//SYSIN DD *

INDD(IDD1,OPTIONS(DUMP))

INDD(IDD2,OPTIONS(DUMP))

INDD(IDDN,OPTIONS(DUMP))

OUTDD(SMFDATA,TYPE(74(1,5,8)))

DATE(2022060,2022066)

//*

Figure 92 Dump the SMF data

Multiple IDD inputs can be specified, one per z/OS system for which SMF data is to be dumped. It is important that all z/OS systems attached to the target storage systems are included. But all included systems should be in the same parallel SYSPLEX and should use a common IODF configuration.

Ideally, the RMF interval duration should be the same on all systems that are included in the same dump data set. The SMF records of interest are 74-1, 74-5, and 74-8 records, as specified on the OUTDD specification (OUTDD(SMFDATA,TYPE(74(1,5,8)))).

Table 13 SMF records

Activities	SMF Record Type (Long-term Monitor I)
Direct Access Activity	74-1
Cache Subsystem Activity	74-5
Enterprise Server Storage Statistics	74-8

The customer needs to modify the SMF input data set names <input_smfdata_systemX> on the IDD lines to identify the correct data sets in the installation. The last SYSIN line (DATE(2022060,2022066)) specifies a date range from March 1, 2022 to March 7, 2022, inclusive. The line should be modified to specify the desired date range for which data should be dumped. The dates are in Julian format: a 4-digit year followed by a 3-digit Julian day. See Figure 93 on page 107.

Figure 93 Julian dates

Note: If a large amount of data is being dumped, it might be necessary to significantly increase the SPACE allocation on line 10 of the SMFDATA output data set. The JCL example assumes that this output is passed to a subsequent step. The disposition should be changed as needed.

Step 3: Adjusting for different time zones

Step 3 runs the ERBCHGMT program to change the time stamps in the SMF records to use a common time zone.

This step is optional and can be removed from the JCL if all systems for which data was dumped in step 2 run in the same time zone. If this step is included, it ensures that the data can be properly sorted chronologically in Step 4.

//*

//* =================

//* UPDATE GMT OFFSET

//* =================

//CHGGMT EXEC PGM=ERBCHGMT,PARM='-300'

//SMFDATA DD DISP=(OLD,DELETE),DSN=*.SMFDUMP.SMFDATA

//SMFCHGMT DD DISP=(NEW,PASS),DSNTYPE=LARGE,

// SPACE=(CYL,(100,100),RLSE),UNIT=SYSDA,

// DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=0)

//SYSPRINT DD SYSOUT=*

//*

Figure 94 Adjust for different time zones

The CHGGMT program line contains the PARM='-300' parameter, which indicates that a time zone offset of -300 (GMT-5:00 for U.S. Eastern Time Zone) should be used for all SMF records in the dumped data set. This line should be modified to specify the preferred time zone as a number of minutes offset from GMT. The range for the PARM parameter is typically in the range of -720 (GMT-12) to +840 (GMT+14).

The SMFDATA statement in the example above uses the output data set from Step 2. The disposition and data set names should be adjusted as needed.

Note: If a large amount of data is being processed, it might be necessary to significantly increase the SPACE allocation of the SMFCHGMT output data set. The JCL example assumes that the output is passed to a subsequent step. The disposition should be changed as needed.

Step 4: Sorting the SMF Records

Step 4 runs the DFSORT program to sort the SMF records into chronological order.

//*

//* ====================

//* SORT THE SMF RECORDS

//* ====================

//RMFSORT EXEC PGM=ICEMAN

//SORTIN DD DISP=(OLD,DELETE),DSN=*.CHGGMT.SMFCHGMT

//* DD DISP=(OLD,KEEP),DSN=<addtnl_data_set>

//* DD ...uncomment and fill in DSNs...

//* DD DISK=(OLD,KEEP),DSN=<addtnl_data_set>

//SORTOUT DD DISP=(NEW,PASS),DSNTYPE=LARGE,

// SPACE=(CYL,(100,100),RLSE),UNIT=SYSDA,

// DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=0)

//SYSPRINT DD SYSOUT=*

//SYSOUT DD SYSOUT=*

//SYSIN DD *

SORT FIELDS=(11,4,CH,A,7,4,CH,A,15,4,CH,A),EQUALS

MODS E15=(ERBPPE15,36000,,N),E35=(ERBPPE35,3000,,N)

OPTION HIPRMAX=OPTIMAL,DYNALLOC=(SYSDA,6)

//*

Figure 95 Sort the SMF record - DFSORT program

If DFSORT is not available in the installation, a generic SORT program can be used, though the performance options (HIPRMAX and DYNALLOC) might not be supported. In that case, this step can be modified as shown in Figure 96 on page 109.

//*

//* ====================

//* SORT THE SMF RECORDS

//* ====================

//RMFSORT EXEC PGM=SORT

//SORTIN DD DISP=(OLD,DELETE),DSN=*.CHGGMT.SMFCHGMT

//* DD DISP=(OLD,KEEP),DSN=<addtnl_data_set>

//* DD ...uncomment and fill in DSNs...

//* DD DISK=(OLD,KEEP),DSN=<addtnl_data_set>

//SORTOUT DD DISP=(NEW,PASS),DSNTYPE=LARGE,

// SPACE=(CYL,(100,100),RLSE),UNIT=SYSDA,

// DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=0)

//SORTWK01 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(100,100))

//SORTWK02 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(100,100))

//SORTWK03 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(100,100))

//SORTWK04 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(100,100))

//SORTWK05 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(100,100))

//SORTWK06 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(100,100))

//SYSPRINT DD SYSOUT=*

//SYSOUT DD SYSOUT=*

//SYSIN DD *

SORT FIELDS=(11,4,CH,A,7,4,CH,A,15,4,CH,A),EQUALS

MODS E15=(ERBPPE15,36000,,N),E35=(ERBPPE35,3000,,N)

Figure 96 Sort the SMF record - generic SORT program

The SORTIN statement in Figure 95 on page 108 and Figure 96 above uses the output data set from Step 3. The disposition and data set name(s) should be adjusted as needed. If necessary, this Sort step can be used to combine multiple SMF dump data sets from different systems into a single sorted file. If this is needed in the installation, the additional DD statements under SORTIN can be uncommented and adjusted with the appropriate DSN parameters.

Note: If a large amount of data is being sorted, it might be necessary to significantly increase the SPACE allocation of the SORTOUT output data set. The JCL examples assume that the output is passed to a subsequent step. The disposition should be changed as needed.

Explicit definition of SORTWK data sets is needed if DYNALLOC is not supported. Space allocations may need to be increased and/or additional SORTWK data sets might need to be defined for processing a large data set.

The ERBPPE15 and ERBPPE35 sort exit modules are provided by RMF and should be part of every z/OS installation. If they are not available, the MODS statement can be removed. However, this means that sorting occurs on the SMF date/time fields instead of the RMF date/time fields in each SMF record. In most cases there is little or no difference, but sometimes this can result in an incompletely sorted data set.

Step 5: Tersing the sorted SMF data

Step 5 runs the AMATERSE program to terse (compress) the sorted data set.

//*

//* =====================

//* TERSE THE SMF RECORDS

//* =====================

//RMFTERSE EXEC PGM=AMATERSE,PARM=SPACK

//SYSUT1 DD DISP=(OLD,DELETE),DSN=*.RMFSORT.SORTOUT

//SYSUT2 DD DISP=(NEW,CATLG),DSN=<output_tersed_smfdata>,

// DSNTYPE=LARGE,SPACE=(CYL,(100,100),RLSE)

//SYSPRINT DD SYSOUT=*

//*

Figure 97 Terse the sorted SMF data

The SYSUT1 statement in Figure 97 uses the output data set from step 4. The disposition and data set name should be adjusted as needed.

The output data set name <output_tersed_smfdata> on the SYSUT2 line should be modified to identify an appropriate name. This is the data set to be transferred in Step 6.

Note: If a large amount of data is being tersed, it might be necessary to significantly increase the SPACE allocation of the SYSUT2 output data set.

Step 6: Transfer the tersed data set

The last step is for the customer to transfer the data to you, so that the data can be imported into the Storage Modeller tool. This step varies based on the system policies that are in effect in the customer's environment, and they should use the established mechanism with which they are familiar.

Usually this consists of an interactive invocation of the SFTP or FTPS command, from either a TSO or a z/OS Unix System Services terminal session, or from a remote system. However, it is also possible to run it as a batch job using JCL, as shown in Figure 98 on page 111.

Note that the example shown in Figure 98 on page 111 is not necessarily secure, unless security has been enabled on the z/OS Communication Server FTP client in the FTP.DATA and/or SYS1.TPCPARMS(TPCDATA) file. It is beyond the scope of this documentation to describe the necessary setup to configure secure FTP communications on z/OS.

Important: it is critical for the compressed file to be transferred as binary, not as text (ASCII or EBCDIC). Failing to do so will cause the data to be corrupted and to become unusable.

//*

//* ===================

//* FTP THE TERSED DATA

//* ===================

//RMFFTP EXEC PGM=FTP,REGION=4096K,

// PARM='11.11.11.11 (EXIT'

//SYSPRINT DD SYSOUT=*

//SRCFILE DD DISP=(OLD,KEEP),DSN=*.RMFTERSE.SYSUT2

//OUTPUT DD SYSOUT=*

//INPUT DD *

BINARY

CD <remote_directory>

PUT //DD:SRCFILE <remote_file>

QUIT

//*

Figure 98 Transfer the tersed data set

The IP address on the PARM specification (11.11.11.11) should be replaced with the destination address or host name where the data should be transferred to. The SRCFILE statement uses the output data set from step 5. The disposition and data set name should be adjusted as needed.

The <userid> and <password> values should be filled in appropriately. Instead of coding these in the JCL, it is also possible to include a NETRC statement to identify a data set that contains the necessary credential information. The z/OS manuals describe the proper format of the NETRC data set. And lastly, the <remote_directory> and <remote_file> values should be replaced with the destination directory and file name on the remote server.