ITCAM for Transactions - RRT (KT6) Agent Data Collection

Answer

This technote explains how data generated by Robotic playbacks is processed on the RRT/KT6 agent and uploaded to the Data Warehouse.

The basic data flow for getting data from the playback to TEP workspaces and the Data Warehouse database WAREHOUS is:

RAW ARM DATA written immediately by playback process
ODI DATA created using data from RAW ARM DATA
SHORT-TERM HISTORY (STH) updated using data from ODI DATA
Data uploaded to the WAREHOUS database from SHORT-TERM HISTORY files

Each row of the following table expands on the above steps by detailing filenames, locations, frequency and configuration.

Filename (TIME=Seconds since the epoch)	Location	...............When................	Data Source	TEMA Config	History Config	.............Comments..............
1. RAW ARM DATA ARM_[PID]_[TIME].dat eg. ARM_1912_1240394624.dat	[ITM_HOME]/tmaitm6/arm/log/kt6	During and/or completion of playback	Playback
2. ODI DATA [TIME]_3_?.dat Where ? = Attribute Group file See Table 1	[ITM_HOME]/tmaitm6/odi/arm	See TEMA Config	1. RAW ARM DATA	Data Analysis Configuration: "Number of minutes to aggregate data"		See Additional Notes (1)
3. SHORT-TERM HISTORY See Table 1	Windows: [ITM_HOME]\tmaitm6\logs Unix/Linux: [ITM_HOME]/INTERP_DIR/t6/hist	See History Config or…..... Override (See comments)	2. ODI DATA		Collection Interval	If Pure data use Override See Table 1 & Override Notes See Additional notes (2)
4. Upload to warehouse		See History Config or…..... Override (See comments)	3. SHORT-TERM HISTORY		Warehouse Interval	If Pure data use Override See Table 1 & Override Notes

Table 1.

Short-Term History file	History Collection Attribute Group	Pure data
T6AGNTMSGS	RRT Agent Messages	Yes
T6APPCS	RRT Application Status
T6PBEVENT	RRT Robotic Playback Event	Yes
T6PBSTAT	RRT Robotic Playback Status
T6SUBTXCS	RRT SubTransaction Status
T6SUBTXINS	RRT SubTransaction Instance	Yes
T6TXCS	RRT Transaction Status
T6TXINS	RRT Transaction Instance	Yes

Override Notes:

If an Attribute Group is used to manage Pure data, the Collection and Warehouse Interval settings seen in the History Configuration GUI does not apply.

The following collection and upload behaviour will be used:

Collection Interval
This is the rate at which the Short-Term History file on the TEMA is updated using ODI data from the agent. The collection will happen as often as the ODI data is updated. This is defined by the "Number of minutes to aggregate data before writing out a data point" entry in the "Data Analysis Configuration" tab of the RRT TEMA configuration.

Warehouse Interval
This is the rate at which the data is uploaded from Short-Term History files to the WAREHOUS database. This will happen 30 minutes after data has been received in the Short-Term History file. If data is received regularly at intervals of less than 30 minutes, then the upload will happen every 30 minutes.

If Short-Term History is updated irregularly and the update happens more than 30 minutes since the previous update, then the next upload to the data warehouse will happen 30 minutes later.


Additional Notes:

(1) ODI DATA files are deleted when older than Data Analysis Configuration:
"Number of hours to save data for viewing in the TEP"

(2) Data more than 24 hours old and successfully inserted in the Data Warehouse is removed from SHORT-TERM HISTORY files when warehousing occurs

(3) ARMLastFile
Located in [ITM_HOME]/tmaitm6/odi/arm
Keeps track of the ARM DATA files being analyzed and the last offset into the files successfully analyzed
Contents of this file are binary
Removing this file will cause analyser to reanalyse ARM DATA

(4) Environment Variables
ARM DATA files are restarted every 5 minutes meaning that data in this initial form is not retained after it has been used to generate ODI data files.

Use the following variable to have the agent copy ARM DATA to the [ITM_HOME]/tmaitm6/arm/log/kt6/delete subdirectory

ARM_KEEP_DATA_FILES=Y

This variable is added to the file:

Windows:
[ITM_HOME]\tmaitm6\kt6env
Unix/Linux
[ITM_HOME]/config/t6.ini

This should only be used when needed to aid troubleshooting or for use in data analysis independently of the usual ITCAMfT reporting options.

(5) Refer to the following technote for additional Historical Data Collection information:

Overview of Historical Data Collection Binary File Usage