Configuring historical data collection for Performance Management agents

Edit online

Sample history files for agents are available on your Cloud APM server. Use the sample file for your agent as the basis for creating the history configuration xml file on the Cloud APM server. The server propagates the configuration to all agents of this type. The history file specifies the Warehouse Proxy agent address, the data sets to collect samples from, the frequency of data collection, and how long to keep the data locally.

Before you begin

Before configuring any Cloud APM agent to send data to Tivoli Data Warehouse, ensure that the equivalent Tivoli Monitoring agent is installed in your Tivoli Monitoring environment. Otherwise, reporting functions can fail.

About this task

For every agent that can send historical data to Tivoli Data Warehouse, you can find a sample history configuration file on your Cloud APM server. Create your configuration file by copying the sample file and editing the copy.

The file includes the data sets (attribute groups) of the agent that can send historical data to Tivoli Data Warehouse. If a particular data set that you are interested in does not exist in the sample file, it is likely because this exact data set does not also exist in the Tivoli Monitoring V6.3 agent product or it is not available for historical data collection. You can remove some of the data sets if you do not want to collect data for them.

The file also contains other sample or default settings. You must modify these settings to configure historical data collection.

Do not modify the sample history configuration file, because the next Cloud APM server upgrade installation can overwrite it. Instead, create a copy of the file.

Procedure

Locate the product_name_sample_history.xml file on the Cloud APM server system:
```
install_dir/wlp/usr/servers/min/dropins/CentralConfigurationServer.war/data_source/pc
```
where install_dir is /opt/ibm or the directory specified during server installation and pc is the two-character product code. The product codes are shown in parentheses after the product name in Eligible Cloud APM agents for historical data collection.
Create a copy of the product_name_sample_history.xml file in the same directory with the following name:
```
pc_history.xml
```
where pc is the two-character product code. For example, ud_history.xml for the Db2® agent and lz_history.xml for the Linux operating system agent.
Tip: A file with the name pc_history.xml already exists by default in the location. You can overwrite the file with your new configuration file.
Open pc_history.xml in a text editor.
Specify the Warehouse Proxy agent location. The default value is:
```
<WAREHOUSE LOCATION="ip.pipe:#netaddress[port#]"/>
```
Modify the following parts of this value:

ip.pipe:

For non-secure RPC communication between the agent and the Warehouse Proxy Agent, leave at ip.pipe:. For secure RPC communication, change to ip.spipe:.
Attention: Because of certificate compatibility issues, secure communication (ip.spipe) between Cloud APM agents and the Warehouse Proxy agent requires significant additional configuration. For instructions, see Configuring secure communications for historical data collection. If you want to enable integration with Tivoli Data Warehouse without additional configuration procedures, use the non-secure ip.pipe setting.

#netaddress

Set the IP address or fully qualified host name of the system where the Warehouse Proxy agent is installed. All hosts where your Cloud APM agents run must be able to establish a direct outbound connection to the system using this address or host name.
Attention: If you use an IP address, add the # sign before the address. If you use a fully qualified host name, make sure the # sign is not present before the host name.

port#

Enter the listening port of the Warehouse Proxy agent. The default port is 63358 for the ip.pipe protocol and 65100 for the ip.spipe protocol.
Tip: You can find the value of the warehouse location string in the RAS1 log file on the Warehouse Proxy agent host. The RAS1 log file is located in the Install_Home/logs directory. The file name format is hostname_hd_timestamp-#.log (for example, myhost01_hd_56d4db3c-01.log). Search the log file for the register_interface message.
A RAS1 log message can look like this:
```
"register_interface") Registering "Candle_Warehouse_Proxy": ip.pipe:#9.48.147.34[63358]
```
In this example, set the following value in the file:
```
<WAREHOUSE LOCATION="ip.pipe:#9.48.147.34[63358]"/>
```
If you want to specify more than one destination or protocol, separate each with a semi-colon (;).
For example, you can set the value:
```
<WAREHOUSE LOCATION="ip.spipe:#9.11.123.45[65100];ip.pipe:#9.11.123.45[63358];ip.pipe:tdw.example.com[63358]"/>
```
In this case, when an agent initiates communications with the Warehouse Proxy agent, it attempts secure RPC communication, then falls back to non-secure RPC communication.
Optional: Delete the HISTORY EXPORT rows of the data sets that you do not want to collect history from:
```
<HISTORY EXPORT=60 INTERVAL=15 RETAIN=6 TABLE=TABLENAME/>
```
where TABLENAME is the data set name.
For example, if you do not want to send Linux_IP_Address data samples to the Tivoli Data Warehouse, delete the <HISTORY EXPORT=60 INTERVAL=15 RETAIN=6 TABLE=Linux_IP_Address/> row.

Data sets are described in the Attributes section of the agent help and the reference PDF.
In the rows that remain, specify the interval for exporting the data, the interval for collecting the data, and how long to keep the collected samples locally:

EXPORT

This parameter specifies the interval in minutes for exporting historical data to the Tivoli Data Warehouse. Valid export intervals are 15, 30, and values divisible by 60; an interval greater than 60 could be 120, 180, 240, and so on, up to 1440. The export interval must also be divisible by the INTERVAL parameter value. If you enter an invalid value, no historical data is collected nor exported for the specified data set. Default: 60 minutes.

INTERVAL

This parameter specifies the historical data collection interval in minutes. The minimum collection interval is 1 minute and the maximum is 1440 (24 hours). Valid intervals are must divide evenly into 60 or are divisible by 60: 1, 2, 3, 4, 5, 6, 10, 12, 15, 20, and 30; an interval greater than 60 could be 120, 180, 240, and so on, up to 1440. If you enter an invalid value, no history is collected for the specified data set. Default: 15.

RETAIN

This parameter defines the short-term history data retention period in hours, with a one-hour minimum. There is no limit other than that imposed by storage space on the system. After the retention limit has been reached, the agent deletes oldest data samples as new samples arrive. This retention period ensures that, if the agent loses communication with the Tivoli Data Warehouse for some time, history data is not lost. Default: 6 hours.
Save the pc_history.xml file.
Repeat these steps for each agent that you want to configure for historical data collection.

Results

After you save the pc_history.xml file, the Cloud APM server processes the file and distributes the configuration to all online agents of the same type. The time it takes for an agent to receive and process the file and begin historical data collection varies depending on server work load conditions. It might take 15 minutes or more in some cases. As new agents of the applicable type come online, the server automatically distributes the configuration to them.

After your agents receive the configuration, they continue to send history data to the Tivoli Data Warehouse even if connection to the Cloud APM server is disrupted.

Example

The following example is the content of the ud_history.xml configuration file for the Db2 agent. With this configuration, the agent collects samples from the KUDINFO00 attributes every 15 minutes, transmits the collected data every hour to the Warehouse Proxy agent at IP address 9.88.765.432, port 63358, and retains the collected data locally for 6 hours:

<PRIVATECONFIGURATION>
  <WAREHOUSE LOCATION="ip.pipe:#9.88.765.432[63358]"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="KUDINFO00"/>
</PRIVATECONFIGURATION>

This is the linux_os_sample_history.xml sample history file for the Linux OS agent:

<PRIVATECONFIGURATION>
  <WAREHOUSE LOCATION="ip.pipe:#netaddress[port#]"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="KLZ_CPU"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="KLZ_DISK"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="KLZ_VM_STATS"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="KLZ_NETWORK"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="KLZ_SYSTEM_STATISTICS"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="Linux_IP_Address"/>
</PRIVATECONFIGURATION>

The lz_history.xml historical configuration file that you create from the sample might look like this:

<PRIVATECONFIGURATION>
  <WAREHOUSE LOCATION="ip.spipe:#9.11.123.45[65100];ip.pipe:#9.11.123.45[63358];ip.pipe:tdw.example.com[63358]"/>
  <HISTORY EXPORT="60" INTERVAL="15" RETAIN="6" TABLE="KLZ_CPU"/>
  <HISTORY EXPORT="240" INTERVAL="60" RETAIN="24" TABLE="KLZ_DISK"/>
  <HISTORY EXPORT="120" INTERVAL="15" RETAIN="6" TABLE="KLZ_SYSTEM_STATISTICS"/>
</PRIVATECONFIGURATION>