About poll data aggregation

Data aggregation is the process whereby raw polling data is aggregated into historical poll data and and stored in the database, from where it can be retrieved for presentation in graphs, reports, and dashboards. This is a fully automated system that begins processing raw data when it starts up.

In earlier releases aggregation of polling data for storage as historical data was performed by Tivoli Data Warehouse. From version 4.2 onwards, aggregation of polling data is performed entirely within Network Manager

Note: There is no migration path for historical polling data stored in a legacy Tivoli Data Warehouse integration.

Once set up, the complete system is maintenance free. There are mechanisms built in that protect the system if it starts to receive more data than it can handle, and metrics that make it easy to monitor during the planning phase.

Users can perform a very limited set of configuration tasks, such as defining whether to aggregate data for specific time periods. For example, you can switch off data aggregation for the annual data period.

For many customers, especially those with large networks, the rate at which poll data can be collected can be high. It is important that the mechanism used to perform the task of summarization be able to handle the data load without causing any disruption to the poller. Apache Storm is a distributed real-time computation system that can perform processing on live streams of data efficiently. Apache Storm uses a topology that defines the tasks of aggregating the poll data, and Storm then manages the tasks and the data load. The topology defines how raw data is processed to produce aggregated daily, weekly, and monthly, and annual polling data that can be stored in the NCPOLLDATA database. By default, this topology will take the name NMStormTopology.