Data Source creation

You create data sources to ingest data from a specific source.

Before you can search a log file, you must use the data source creation wizard to create a data source. To start the wizard, open the Admin UI, click Add > Data Sources. The wizard contains text to help you to complete the task.

You can create three types of data source. Select one of the following types:
  • To stream data from a file that is stored locally on the IBM® Operations Analytics Log Analysis server, create a local data source. The IBM Tivoli® Monitoring Log File Agent on the same server is automatically configured to retrieve local data.
  • To stream data from a file that is stored on a remote server, create a remote data source. Enter a host name, user name, and password. The IBM Tivoli Monitoring Log File Agent on the same server is automatically configured to pull data from a remote server.
    Note: If the remote server uses the Windows operating system, this setting requires that the ssh daemon is configured on the remote server.
  • To stream data from a file that is stored on a remote IBM Operations Analytics Log Analysis server where data delivery is manually configured, or where data ingestion is configured for Logstash or the Data Collector client, create a custom data source. Enter a host name. There is no automatic configuration or verification.

If you select local or remote as the location, IBM Operations Analytics Log Analysis creates a new collection automatically. If you select custom as the location, you can choose to associate it with an existing collection or you can create a new collection.

The log file specified for the File path field is automatically ingested into IBM Operations Analytics Log Analysis for data sources with Local file or Remote file configuration options.

If you want to ingest rolling files, click the Rolling file pattern check box and enter the rolling file pattern. For example, for a WebSphere® Application Server log file the base line file is SystemOut.log. The rolling file pattern is SystemOut_*.log. If you use this setting, the data latency depends on how frequently the files are rolled in the source application.

To complete the process, enter a name for the data source.

You can also edit an existing data source.

Logical and physical data sources

A logical data source is a data source that you create in Log Analysis. A physical data source is source of data in your environment. For example a directory where log files are stored.

You can use Log Analysis to create a logical data source which you can use to stream data from multiple, physical sources of data. For example, if you have multiple servers where you store log files, you can use a data source in Log Analysis to stream data from the servers.

For an example of how to use a logical data source to stream data from multiple physical data sources, see Log File Agent example: Streaming data from a single remote host.