Configuring the loading of history data

Each history configuration specifies when the loading process is to load history data from the staging area into the database. You can specify thresholds that are based on time intervals, the amount of data in the staging area, or both. Each history configuration also specifies a maximum size for the staging area.

The following history configuration settings determine when the loading process loads history data from the staging area into the database:

LOADINTERVAL: A loading timer that can range from 1 - 60 minutes. Specify 0 to disable the timer. When the load interval timer expires, the system checks for captured data in the staging area. Based on the values of the staging size threshold values, LOADMINTHRESHOLD and LOADMAXTHRESHOLD, and whether the loader is idle, the data in the staging area might or might not be transferred to the loader.
LOADMINTHRESHOLD: The minimum amount of history data, in megabytes, to collect before it transfers the batch to the loading area. Specify 0 to disable the minimum threshold check.
LOADMAXTHRESHOLD: The maximum amount of history data to collect in the staging area before it is automatically transferred to the loading area. Specify 0 to disable the maximum threshold check.

These settings, called loader settings, can have zero or non-zero values, but at least one setting must be non-zero. The following table describes the valid value combinations:

Table 1. History loading settings
Load interval	Min threshold	Max threshold	Loader state	Operation
0	0	Non-zero	Idle	Transfer the captured data in the staging area to the loading area, regardless of the staging size. This combination is typically used for test/demonstration environments. Because of the continuous loading of data, this setting can cause a performance impact on systems.
0	0	Non-zero	Busy	When the captured data in the staging area meets or exceeds the max threshold, transfer it to the loading area.
0	Non-zero	0	Idle	When the captured data in the staging area meets or exceeds the min threshold, transfer it to the loading area.
0	Non-zero	0	Busy	Continue collecting data in the staging area until the loader is idle, then transfer the data to the loading area.
0	Non-zero	Non-zero	Idle	When the captured data in the staging area meets or exceeds the min threshold, transfer it to the loading area.
0	Non-zero	Non-zero	Busy	Continue collecting data in the staging area until the max threshold is reached or until the loader is idle, then transfer the data to the loading area.
Non-zero	0	0	N/A	Transfer any captured data in the staging area to the loading area when the timer expires, regardless of amount of data or loader state.
Non-zero	0	Non-zero	N/A	Transfer captured data in the staging area to the loader when the timer expires, or when the data meets or exceeds the max threshold.
Non-zero	Non-zero	0	N/A	When the timer expires, transfer the captured data in the staging area to the loading area if it meets or exceeds the min threshold.
Non-zero	Non-zero	Non-zero	Idle	When the timer expires, transfer the captured data in the staging area to the loading area if it meets or exceeds the min threshold or the max threshold. Use this combination for a production Netezza Performance Server system. You can “tune” the values of the three loading settings for your environment as described in the text that follows this table.
Non-zero	Non-zero	Non-zero	Busy	If the staging area meets or exceeds the max threshold, transfer the captured data in the staging area to the loading area. Otherwise, continue collecting data until the next timer expiration.

Choose loader settings that balance the need for current history data with the need avoid unduly affecting the system. History data that is still in the staging area is stored in external text files. If necessary, users can review those files to obtain information about recent activity before that information is loaded into the history database.

Depending on the loader settings, how much history data is collected, and the overall utilization of the system, the alcloader process might become busy loading history data. If there are several batch directories in the loading area, this might indicate queued and waiting load requests. You can experiment with different loader settings to tune it for optimal operation.

To ensure that the staging area does not grow indefinitely, use the STORAGELIMIT parameter to set its maximum size, in MB. If the staging area reaches or exceeds this size limit, the system stops collecting history data. An administrator must free up disk space in the storage area to free up disk space so that history collection can resume. This is usually done by adjusting the loader settings, The STORAGELIMIT value must be greater than the LOADMAXTHRESHOLD value.