Configuring trace spooling

Edit online

RSCT supports multi-page tracing to write the application trace records into trace files. In multi-page tracing, more than one trace file exists and when the existing trace files become full, the trace records are written to the next available trace file. When the last page file associated with a logical trace file is full, the trace records are written to the first page file.

The fixed number of these trace files limits the history information that is available for debugging purposes. You can use trace spooling to copy the completed files to another directory on the system.

In trace spooling, page files are copied to a spool destination.

By using the multi-page tracing, the trace library writes trace records for a specific logical trace file to a set of physical files that have the same base name and suffixes in the form of .number.sp. These files are referred as trace page files. When the last page file associated with a logical trace file is full, the trace records are written to the first page file. Trace spooling can be turned on or off.

Limited only by available disk space, trace spooling allows the system to maintain a continuous history of RSCT actions. You can configure trace spooling control at the daemon level, using the common trace facility configuration file (/var/ct/cfg/trace.conf) or the CT_TR_SPOOL environment variable.

When configured for trace spooling, the trace library manages a client's trace file as several page files and creates a spooling thread to copy full page files to the spooling area. If the copy thread cannot spool page files as quickly as the client can fill them, the trace library favors the retention of the most recent trace records by overwriting older spool page files that could not be spooled in time. When this occurs, the trace library adds a record that looks like the following example to the current page file:


SPOOL: page_file_name not spooled integer consecutive time[s]

This record documents the fact that a page file was not spooled, so its trace records were lost. This circumstance can be mitigated by configuring clients that write many trace records to have a larger page file footprint (that is, more pages, a larger size, or both).

Because the spooling area can become full, leading to a possible loss of trace data that is unaddressed after a long-term retention, you can clean up of the spool area based on the following retention policies:

The size of the spool area
The age of the trace file

You can clean up the spool area by using the cttracecfg command. Consider the following cttracecfg command examples:

```
cttracecfg -S -a -n spoolarea_data -d /data/trc -i 12 -n 14 -o on
```
Result: Any trace files older than two weeks (14 days) are deleted. This check is performed every 12 hours.
```
cttracecfg -S -a -n spoolarea_data -d /data/trc -i 12 -c 50 -o on
```
Result: Saved trace files are deleted, starting with the oldest trace files, until the trace files occupy a maximum of 50 MB of the total space in the /data/trc directory. This check is performed every 12 hours.

Because trace spooling can result in a large number of spooled trace files, and the rpttr command can run out of system resources when attempting to show the content of too many files, use the showtr command to narrow down the scope of spooled trace files to view. Run the showtr --help command to see its filtering options and usage examples.