Log File log source parameter

Type an IP address, host name, or name to identify the event source. IP addresses or host names are suggested as they allow QRadar to identify a log file to a unique event source.

For example, if your network contains multiple devices, such as multiple z/OS® images or a file repository that contains all of your event logs, you must specify a name, IP address, or host name for the image or location that uniquely identifies events for the DSM log source. This specification enables events to be identified at the image or location level in your network that your users can identify.

From the Service Type list, select the protocol that you want to use when retrieving log files from a remote server. The default is SFTP.

• SFTP - SSH File Transfer Protocol
• FTP - File Transfer Protocol
• SCP - Secure Copy

The underlying protocol that is used to retrieve log files for the SCP and SFTP service type requires that the server that is specified in the Remote IP or Hostname field has the SFTP subsystem enabled.

Type the IP address or host name of the device that stores your event log files.

Type the TCP port on the remote host that is running the selected Service Type. The valid range is 1 - 65535.

The options include ports:

• FTP - TCP Port 21
• SFTP - TCP Port 22
• SCP - TCP Port 22

If the host for your event files is using a non-standard port number for FTP, SFTP, or SCP, you must adjust the port value.

Type the user name or user ID necessary to log in to the system that contains your event files.

• If your log files are on your IBM z/OS image, type the user ID necessary to log in to your IBM z/OS. The user ID can be up to 8 characters in length.
• If your log files are on a file repository, type the user name necessary to log in to the file repository. The user name can be up to 255 characters in length.

Type the password necessary to log in to the host.

Confirm the password necessary to log in to the host.

If you select SCP or SFTP as the Service Type, this parameter gives you the option to define an SSH private key file. When you provide an SSH Key File, the Remote Password field is ignored.

Type the directory location on the remote host from which the files are retrieved, relative to the user account you are using to log in.

If you want the file pattern to search sub folders in the remote directory, select this check box. By default, the check box is clear.

If you configure SCP as the Service Type, the Recursive option is ignored.

If you select SFTP or FTP as the Service Type, you can configure the regular expression (regex) needed to filter the list of files that are specified in the Remote Directory. All matching files are included in the processing.

The IBM z/OS mainframe that uses IBM® Security zSecure Audit writes event files by using the pattern: <product_name>.<timestamp>.gz

The FTP file pattern that you specify must match the name that you assigned to your event files. For example, to collect files that start with zOS and end with .gz, type the following code:

zOS.*\.gz

Use of this parameter requires knowledge of regular expressions (regex). For more information about regex, see Lesson: Regular Expressions (https://docs.oracle.com/javase/tutorial/essential/regex/).

This option displays only if you select FTP as the Service Type. From the list, select Binary.

The binary transfer mode is needed for event files that are stored in a binary or compressed format, such as zip, gzip, tar, or tar+gzip archive files.

If you select SCP as the Service Type you must type the file name of the remote file.

Type the time of day you want the processing to begin. For example, type 00:00 to schedule the Log File protocol to collect event files at midnight.

This parameter functions with the Recurrence value to establish when and how often the Remote Directory is scanned for files. Type the start time, based on a 24-hour clock, in the following format: HH: MM.

Type the frequency, beginning at the Start Time, that you want the remote directory to be scanned. Type this value in hours (H), minutes (M), or days (D).

For example, type 2H if you want the remote directory to be scanned every 2 hours from the start time. The default is 1H.

If you want the Log File protocol to run immediately after you click Save, select this check box.

After the Run On Save completes, the Log File protocol follows your configured start time and recurrence schedule.

Selecting Run On Save clears the list of previously processed files for the Ignore Previously Processed File parameter.

The maximum number of events per second that QRadar ingests.

If your data source exceeds the EPS throttle, data collection is delayed. Data is still collected and then it is ingested when the data source stops exceeding the EPS throttle.

The valid range is 100 to 5000.

From the list, select gzip.

Processors enable event file archives to be expanded and contents are processed for events. Files are processed after they are downloaded to QRadar. QRadar can process files in zip, gzip, tar, or tar+gzip archive format.

Select this check box to track and ignore files that are already processed by the Log File protocol.

QRadar examines the log files in the remote directory to determine whether a file is previously processed by the Log File protocol. If a previously processed file is detected, the Log File protocol does not download the file for processing. All files that are not previously processed are downloaded.

This option applies only to FTP and SFTP service types.

Select this check box to define a local directory on your QRadar for storing downloaded files during processing.

It is suggested that you leave this check box clear. When this check box is selected, the Local Directory field is displayed, which gives you the option to configure the local directory to use for storing files.

From the Event Generator list, select LineByLine.

The Event Generator applies more processing to the retrieved event files. Each line is a single event. For example, if a file has 10 lines of text, 10 separate events are created.