Log File protocol configuration options
To receive events from remote hosts, configure a log source to use the Log File protocol.
The Log File protocol is an active outbound protocol that is intended for systems that write daily event logs. It is not appropriate to use the Log File protocol for devices that append information to their event files.
Log files are retrieved one at a time by using SFTP, FTP, SCP, or FTPS. The Log File protocol can manage plain text, compressed files, or file archives. Archives must contain plain-text files that can be processed one line at a time. When the Log File protocol downloads an event file, the information that is received in the file updates the Log Activity tab. If more information is written to the file after the download is complete, the appended information is not processed.
|Protocol Configuration||Log File|
|Log Source Identifier||
Type a unique name for the log source.
The Log Source Identifier can be any valid value and does not need to reference a specific server. It can also be the same value as the Log Source Name. If you have more than one configured Log File log source, ensure that you give each one a unique name.
Select the protocol to use when retrieving log files from a remote server.
The server that you specify in the Remote IP or Hostname field must enable the SFTP subsystem to retrieve log files with SCP or SFTP.
|Remote IP or Hostname||Type the IP address or hostname of the device that contains the event log files.|
|Remote Port||If the remote host uses a nonstandard port number, you must adjust the port value to retrieve events.|
|Remote User||Type the username that you use to log in to the host that contains the event files.|
|Remote Password||Type the password that you use to log in to the host.|
|Enable Strict Host Key Checking||Enable this option to specify the public key of the target host from a predefined host key list.|
|Host Key List||Provide a list of accepted Base64 encoded host keys to connect to the target
host. Specify multiple keys on separate lines, and use blank lines for formatting. Supported host
key types are:
You can obtain the host keys by running the OpenSSH command
|SSH Key File||
If the system is configured to use key authentication, type the SSH key. When an SSH key file is used, the Remote Password field is ignored.
The SSH key must be located in the /opt/qradar/conf/keys directory.
Important: The SSH Key File field no longer accepts a file path. It can't contain "/" or "~". Type the file name for the SSH key. The keys for existing configurations are copied to the /opt/qradar/conf/keys directory. To ensure uniqueness, the keys must have "_<Timestamp>" appended to the file name.
|Remote Directory||For FTP, if the log files are in the remote user’s home directory, you can leave the remote directory blank. A blank remote directory field supports systems where a change in the working directory (CWD) command is restricted.|
|Recursive||Enable this checkbox to allow FTP or SFTP connections to recursively search subfolders of the remote directory for event data. Data that is collected from subfolders depends on matches to the regular expression in the FTP File Pattern. The Recursive option is not available for SCP connections.|
|FTP File Pattern||The regular expression (regex) that is needed to identify the files to download from the remote host.|
|FTP Transfer Mode||For ASCII transfers over FTP, you must select NONE in the Processor field and LINEBYLINE in the Event Generator field.|
|FTPS TLS Version||The versions of TLS that can be used with FTPS connections. To use the most
secure version, select the TLSv1.2 option. When you select an option with
multiple available versions, the FTPS connection negotiates the highest version available by both
the client and server.
This option can be configured only if you selected FTPS in the Service Type parameter.
|SCP Remote File||For SCP file transfers, type the name of the file on the remote host. You can choose only a single file. This parameter does not support adding multiple files, including methods like file globbing or regular expressions.|
|Start Time||Select the time of day for the log source to start the file import. This parameter works with the Recurrence parameter to establish when and how often the remote directory is scanned for files.|
The time interval to determine how frequently the remote directory is scanned for new event log files. The time interval can include values in hours (H), minutes (M), or days (D). For example, a recurrence of 2H scans the remote directory every 2 hours.
|Run On Save||Starts the log file import immediately after you save the log source configuration. When selected, this checkbox clears the list of previously downloaded and processed files. After the first file import, the Log File protocol follows the start time and recurrence schedule that the administrator defines.|
|EPS Throttle||The number of Events Per Second (EPS) that the protocol cannot exceed.|
|Processor||If the files on the remote host are stored in an archive format, select the processor that is used to decompress the event log. If the files are not stored in an archive format, select None. The default value is None.|
|Ignore Previously Processed File(s)||Select this checkbox to track files that the log source processes. This option prevents duplicate events from files that are processed a second time. This checkbox applies to FTP and SFTP file transfers.|
|Change Local Directory?||Changes the local directory on the Target Event Collector to store event logs before they are processed.|
|Local Directory||The local directory on the Target Event Collector. The directory must exist before the Log File protocol attempts to retrieve events.|
|Event Generator||Choose one of the following file types to use as an event generator for the
|File Encoding||The character encoding that is used by the events in your log file.|
|Message ID Pattern||Type a regular expression (regex) that identifies a common value at the start of each line in a multiline event message.|
|Folder Separator||The character that is used to separate folders for your operating system. Most configurations can use the default value in Folder Separator field. This field is intended for operating systems that use a different character to define separate folders. For example, periods that separate folders on mainframe systems.|
Configure QRadar to use FTPS for the Log File protocol
To configure FTPS for the Log File protocol, you must place server SSL certificates on all QRadar® Event Collectors that connect to your FTP server. If your SSL certificate is not RSA 2048, create a new SSL certificate.
openssl req -newkey rsa:2048 -nodes -keyout ftpserver.key -x509 -days 365 -out ftpserver.crt
Files on the FTP server that have a .crt file extension must be copied to the /opt/qradar/conf/trusted_certificates directory on each of your Event Collectors.