Data storage

Database schema that contains various tables to store specific types of data that is collected by Network Performance Insight®.

The following table lists the type of data that is stored in Network Performance Insight database schema:
Table 1. Network Performance Insight database schema
Database schema Type of data
Flow Metric The following Flow metric data is stored in different tables in the Flow Metric schema:
  • Flow Raw data
  • Aggregated data of 1 day, 1 minute, and 30 minutes for all top 10 performers
  • Aggregation status
  • Interface configuration settings for Flow metric data
    Note: Contains interface name, speed, and interface description to enrich the information that is displayed for a particular interface.
  • The Application response time target information
  • All the IP addresses of the various flow-enabled devices
    Note: Includes SNMP community strings of the devices in the network that are discovered. The following information can be obtained from this table:
    • Enrichment state
    • Read Community string
    • Context name
    • SNMP version
    • SNMP port
    • Level that specifies the required level of authentication and privacy.
    • Security name
    • Authorization type for the type of encryption for the authentication password
    • Authorization password
    • Privacy password
    • Privacy type for the type of encryption for the privacy password
  • QoS information such as:
    • QoS class names, class IDs, and class types
    • QoS policy names, policy IDs, and policy types
    • QoS queue IDs for the different entities
Configuration The following Flow configuration data is stored in this schema:
  • Aggregation
  • Autonomous systems
  • Cacti servers
  • Domain Names
  • IP Address Group
  • Site grouping
  • NBAR
  • NBAR2
  • Pack information
  • Resource Type

    The table contains the resource types from the managed devices in the network from Resource Type pages under System Configuration on Dashboard Application Services Hub. It also contains the polling interval from Polling Configuration pages under System Configuration on Dashboard Application Services Hub.

  • Retention Profiles
  • Site
  • Thresholds
  • Type of Service (ToS)
Cacti Configuration data about the devices that are deployed on Cacti is stored in the Cacti schema.
Event Event data for both Flow and entity metrics data is stored in this schema.
Defaults Dashboard Application Services Hub integration-related data is stored in this schema.
NCIM Network interface information that is discovered by Tivoli® Network Manager and available in Network Connectivity and Inventory Model (NCIM) database. The following types of data are stored in this schema:
  • Discovered network interface related data that corresponds to related devices.
  • Data that is polled from Tivoli Network Manager database by using Kafka Connect, which is sent to Kafka message bus and stored within Network Performance Insight database schema.
NCPOLLDATA Resources and metric metadata from the following sources is stored in this schema:
  • The Polling engine, ncp_poller from Tivoli Network Manager uses data from the tables in this schema to determine how to poll. A copy of the same data from Tivoli Network Manager is stored in this schema.
  • Tivoli Network Manager core server contains Apache Storm server that manages poll data aggregation. The data is pushed to Kafka message bus and stored within Network Performance Insight database schema.
Threshold The following threshold-related data for entity metrics is stored in this schema:
  • Threshold evaluation records and static threshold definitions of entity metric data.
  • Static threshold definitions for entity metric data from both Tivoli Network Manager and Cacti deployments.

Flow aggregated data

Network Performance Insight consists of Fact tables and Domain Objects.

All raw NetFlow data is stored up to 5 days as the initial default configuration of Network Performance Insight in the Flow database table.

The data for all interfaces from the NetFlow top talkers on Traffic Details dashboard is aggregated with every 1-minute interval. Aggregated data is progressively stored in 1 minute, 30 minutes, and Daily aggregation tables. The most recent data is available with 1-minute aggregation. Data less than 50 hours are populated from 1-minute aggregation tables while data more than 50 hours but less than 100 days are populated from 30-minutes aggregation tables. All data that is more than 100 days are populated from 1-day aggregation tables.

The following table lists the traffic tables that are queried for the data that is displayed in dashboards in Network Performance Insight for different time intervals:
Time interval Traffic tables
Less than 50 hours 1-minute aggregation
50 hours to less than 100 days 30-minutes aggregation
More than 100 days 1-day aggregation

Inventory data

Starting from V1.3.1, all the inventory data is stored in Cassandra.

Types of inventory-related metadata that is collected from SNMP enabled devices are as follows:
  • SNMP enabled device property names and values
  • SNMP enabled device formulas or metric names
  • SNMP enabled devices entity or resource names and parent entity names
  • IP to host name mappings