IBM Support

Setting up data preparation for QRadar SIEM

News


Abstract

You can use IBM Security zSecure Audit to make z/OS event data available for a QRadar appliance, using the Log Event Enhanced Format (LEEF) format .

Content

 Subscribe to My Notifications to be notified of important product support news like this.
  • Follow this link for more information (requires login with your IBM ID)


You can use IBM Security zSecure to make z/OS event data available for a QRadar appliance. For QRadar SIEM, a z/OS image contains a number of Log Sources: for z/OS itself and for RACF, ACF2, or Top Secret. In addition, if DB2 and CICS are active on your z/OS image, the image also contains Log Sources for these products. On the z/OS image, you must set up a zSecure process to transform SMF records into the Log Event Enhanced Format (LEEF) that QRadar expects. This process includes enrichment of the raw SMF data with data from your system configuration and security database.

Device Support Modules (DSMs) on the QRadar appliance retrieve these LEEF files, according to a schedule that is configured on the QRadar console.

Prerequisites

Be sure to meet the following requirements before you set up the data collection process:

  1. Before you set up the data collection process for QRadar, you must complete the basic, shared part of the zSecure installation process. After installing the software, you must also perform activities to create and modify the configuration. The following criteria must be met:
    • PARMLIB member IFAPRDxx must enable at least the zSecure Audit component. For details, see “Enabling the license” in the IBM Security zSecure CARLa-Driven Components: Installation and Deployment Guide.
    • The SCKRLOAD library must be APF-authorized. For details, see “Making the software APF authorized" in the IBM Security zSecure CARLa-Driven Components: Installation and Deployment Guide.
    • You must set up a process to periodically refresh your CKFREEZE and UNLOAD data sets. See “Using a fresh CKFREEZE and UNLOAD each day” in the IBM Security zSecure CARLa-Driven Components: Installation and Deployment Guide.
  2. You must have an active FTP (or SFTP) server on your z/OS image, so that QRadar can download the LEEF files.
  3. The zSecure configuration must contain the specific parameters for QRadar SIEM. For information, see the section "Updating the configuration files."

For instructions for installing and configuring zSecure, see the Program Directory: IBM Security zSecure Suite CARLa-driven components and the first few chapters of the IBM Security zSecure CARLa-Driven Components: Installation and Deployment Guide.

See the IBM Security zSecure information center at
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=/com.ibm.zsecure.doc_1.13/welcome.html.

Selecting the SMF input for the data collection process

Before starting the data collection process, you must:

1. Generate the SMF records.

    SMF processing must be turned on and appropriate records must be created and saved:

    The standard required SMF records are:
    • 0, 7, 9, 11, 14, 15, 17, 18, 22, 26, 30, 36, 41, 42, 43, 45, 47, 48, 49, 52, 53, 54, 55, 56, 57, 58, 59, 61, 62, 64, 65, 66, 80 (RACF and Top Secret), 81 (RACF), 82, 90, most of 92, 118, and 119
    • Selected subtypes of 102 (DB2 IFCids 4, 5, 6, 7, 8, 9, 10, 22, 23, 24, 25, 55, 83, 87, 90, 92, 104, 105, 107, 140, 141, 142, 143, 144, 145, 169, 177, 219, 220, 258, 270, 314, and 319)
    • The CICS monitoring record type 110 subtype 1
    • The ACF2 record type (site-defined number) if you have ACF2
    The exact SMF record selection is specified in the CARLa member C2ELEEF. This member can be updated by regular maintenance.

    To generate SMF records for CICS transactions, CICS monitoring must be enabled and set up. You can set up monitoring by data types and classes. For example, you can monitor the classes for exceptions, performance, and resources.

    To use CICS monitoring:
    1. Create a DFHMCTxy CICS Monitoring Control Table (MCT).
    2. Add MCT=xy to the System Initialization Table (SIT).
    3. Run the CEMT INQ MON command to confirm or set one or both of the following:
    • Monitoring on classes of monitoring data and options
    • Classes of monitoring data and options

    • For DB2, you must activate the DB2 trace in order to generate the required SMF records. Use the following commands. (These commands are intended as an example. In your installation, IFCIDs might already be logged to SMF by other traces. Verify and adapt these examples to meet the requirements of your installation.)
    • If you use installation-defined events, be sure to include the SMF records required by your CARLa member C2EQCES.


2. Make the SMF records available for QRadar.
  • If you are using SMF logstreams, the most convenient way to run data collection is by reading directly from a logstream. Make sure that the data collection for QRadar runs at least as frequently as the SMF retention period that you specified for your logstream, (You use the logstream administrative data utility, IXCMIAPU, to specify a logstream). You might want to set up a dedicated logstream for this purpose.
  • If you are using SMF data sets, you must prepare the input for the data collection during your SMF offload process. That is:

      • a. Add another DD-statement to your IFASMFDP program like:


      b. Update the control statements for IFASMFDP to write simultaneously to your existing accumulation data sets and to the new data sets.

      The collection process for QRadar can then use the DSNPREF parameter to retrieve the additional data sets and, after successful processing, delete these data sets.

      The use of system symbols, as shown in this example, is supported only in started tasks. If you run your SMF offload as a batch job, you can use a generation data group (GDG). However, this approach has disadvantages in serialization. Consider converting to a started task.

    • Do not specify your live SMF data sets as your only input. Doing so results in gaps: SMF records that are written after you run the QRadar job (or started task), but before the next SMF switch, would be missing.
    • If you are creating daily SMF accumulation data sets and you intend to prepare QRadar data once a day, you can use the accumulation data sets as input. However, do not use, for example, a monthly accumulation data set as input for a daily preparation because, in that case, SMF records early in the accumulation are read multiple times. zSecure skips records that were already processed, but the excess reading costs processing resources. Especially when your accumulated SMF is written to tape, and the tape data set becomes multi-volume, reading the SMF accumulation might become prohibitive, both in processing resources and in contention for your tape drive and volume.
    • If your data set contains records from multiple z/OS images, do not feed that data set directly into QRadar; this is not supported. Instead, do one of the following:
        • Specify an EXCLUDE statement in member C2EQ0ES. See the section “Updating the configuration files.”
        • First run a special CARLa job or jobstep against accumulation data sets and use the output from that job or jobstep as input for the QRadar preparation. In the job, use SELECT statements to specify the SMF ID and UNLOAD statements to write the records to separate data sets for each z/OS image.
    • When recovering a lost SMF interval, run a similar job to select the interval and SMF ID from your accumulation data sets.

Setting up the collection process

zSecure supplies job C2EJQRLF and procedure C2ECQRLF.
  • If you want to run the collection process in batch, configure your job scheduling product to submit a copy of job C2EJQRLF. As an alternative, you can arrange for your SMF offload process to submit this job.
  • If you want to run the collection process as a started task, include procedure C2ECQRLF and the zSecure configuration member in a procedure data set for your Job Entry Subsystem (JES), and have it started by your automated operation, or by a JES autocommand.
  • Customize your copies of the JCL according to the conventions used in your installation. In particular, specify the zSecure configuration member of your choice. The default is C2R$PARM, but if you use several functions of the zSecure Suite, you probably want separate configurations for each function.
  • Store your configuration member where the JCL for the job or started task can access it. For a started procedure, this is a JES procedure library. For a job, you can use the JCLLIB statement to specify any other data set.

QRadar-specific parameters that you must supply are:


C2EQCUST
    The name of the data set that contains configuration members C2EQENV, C2EQSPEC and C2EQFIN, see the section “Updating the configuration files.”
C2EQPATH
    The UNIX directory where the collection process is to leave its data.

Notes:
    1. Do not start the process until you finish updating your configuration.
    2. The first run fails with a CKR0945 message because the cutoff file does not yet exist. There is no harm in this failure; the file is created during this first job or started task. Just rerun.

Assigning a userid and preparing a directory to store the LEEF data

You must assign and prepare a UNIX directory where the LEEF data is stored for retrieval by QRadar. Like any UNIX directory, it must have both an owning user and an owning group. You probably want to use the home directory of the userid that runs the collection process (or a subdirectory of the home directory), and you probably want to use a dedicated file system.

zSecure provides three different jobs to create the required user and group, home directory, and file system:
  • Job C2EQAUSA for ACF2
  • Job C2EQAUSR for RACF
  • Job C2EQAUST for Top Secret


    • 1. Select the job that applies to your External Security Manager (RACF, ACF2, or Top Secret).

    2. Adapt the job according to your conventions for users, groups, uids, gids, and data sets.

    3. Depending on your choice for a batch job or a started task, uncomment the actions that create a SURROGAT or a STARTED profile, (for RACF, or the ACF2 or Top Secret equivalent). This step ensures that the process runs under the designated userid.

    4. Specify the size of the file system, based on the amount of SMF data that is generated between two consecutive retrievals by the QRadar appliance. Allow for a margin to accommodate for SMF peaks or QRadar outages.

    5. Run the job.

    Make sure that the file system is mounted after subsequent IPLs. You might want to use the automount facility.

    Updating the configuration files

    If you want to use a new zSecure configuration data set (often called CKRPARM, although you can use any data set name), run job CKRZPOST. For information, see Chapter 6, “Deploying the software,” in the IBM Security zSecure CARLa-Driven Components: Installation and Deployment Guide.

    You can use an existing CKRPARM data set, but if it was created by an older level of zSecure, members C2EQENV, C2EQSPEC ,and C2EQFIN might be missing. If so, copy these members from the SCKRCARL and SCKRSAMP libraries.

    If you want further customization, also copy members C2EQ0ES, C2EQXES, and C2EQCES.

    Then, customize the members:

    1. Adapt member C2EQENV to specify your input: specify the UNLOAD and CKFREEZE data set that you refresh every day with the C2RJPREP job. See “Using a fresh CKFREEZE and UNLOAD each day” in the IBM Security zSecure CARLa-Driven Components: Installation and Deployment Guide.If you are using Top Secret, remove the UNLOAD allocation, because that does not apply to Top Secret.

    2. Adapt member C2EQSPEC to specify your input and output:

      a. Specify the same UNLOAD and CKFREEZE data set that you specified in member C2EQENV.

      b. Specify the SMF that you selected as input, either as the name of a logstream, or by using the DSNPREF parameter. See “Making the SMF records available for QRadar” in the IBM Security zSecure CARLa-Driven Components: Installation and Deployment Guide.If you use the DSNPREF parameter, specify the DELETE parameter, so that the data sets that you created for this purpose during SMF offload are deleted after successful processing.

      c. Specify the absolute path of the LEEF files. For log sources that do not occur in your system (for instance, for ACF2 when your system uses RACF, or the reverse), you can direct the output to /dev/null; this action avoids writing LEEF files that consist of only a zip header. For the other LEEF files, do not change the file name, and make sure that the directory part of the path matches the C2EQPATH parameter.

      d. Specify the absolute path for the time-based cutoff file. This file is the one identified as TYPE=SMFHWIN and TYPE=SMFHWOUT. Make sure that you specify the same file in both cases.

      Note: If you need to recover a lost SMF interval, blank out this file in order to prevent skipping the time period that you want to recover. After recovery is done, edit it back to the previous contents.

      e. If desired, specify output options. The default options are in the zSecure-supplied sample member.

    3. Adapt member C2EQFIN to specify the retention period (in hours) for the LEEF files.

    4. Optionally, adapt members C2EQ0ES, C2EQXES, and C2EQCES as follows:

      C2EQ0ES
        CARLa that is to be processed at the start of processing. You can use this member, for example, when IBM or vendor software writes badly formatted SMF records that cause errors. In this case, IBM Software Support can supply a set of CARLa statements to use until the problems with the OEM vendor are solved, or a more permanent solution is built.

        You can also exploit this member to specify the right CCSID when translating, for example, double byte character set (DBCS) characters to UTF-8. For instance, for Japanese Latin extended Unicode, you can include the following CARLa statement:
        OPTION MY_CCSID=1399

      C2EQXES
        CARLa EXCLUDEs for the SMF selection.

      C2EQCES
        CARLa to customize the z/OS Log Source to also map installation-defined events. For example, you might have a product in place with its own SMF records.


    Configuring QRadar to download the LEEF files

    On the QRadar Console, specify the properties of the Log source. The following figure shows an example screen. See the Configuring DSMs Guide for QRadar for more information.




    Configuring QRadar to show z/OS-specific event properties

    QRadar shows a number of normalized event properties automatically. Additional fields can be added as custom properties in the QRadar user interface.

    A list of appropriate fields that you will want to consider is listed in the document entitled QRadar Custom Event Properties for IBM z/OS in the IBM Security QRadar SIEM version 7.2.0 Product Documentation.

    [{"Product":{"code":"SSRQ8D","label":"IBM Security zSecure Audit for RACF"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Not Applicable","Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"1.13.0","Edition":"","Line of Business":{"code":"","label":""}},{"Product":{"code":"SSRQGZ","label":"IBM Security zSecure Audit for ACF2"},"Business Unit":{"code":"BU008","label":"Security"},"Component":null,"Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"1.13.0","Edition":"All Editions","Line of Business":{"code":"","label":""}},{"Product":{"code":"SUNSET","label":"PRODUCT REMOVED"},"Business Unit":{"code":"BU008","label":"Security"},"Component":null,"Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"1.13.0","Edition":"All Editions","Line of Business":{"code":"","label":""}}]

    Document Information

    Modified date:
    16 June 2018

    UID

    swg21606082

    Page Feedback