Tutorial - Ingesting data and configuring a domain

Device Management Enablement uses a common data ingest mechanism. Sensor or meter data that is exported from data management systems as CSV files is ingested into Device Management Enablement internal data stores. With Device Management Enablement, you can build a customized data consolidation plan to gather and reconcile data from many sources. Use this tutorial to familiarize yourself with importing data.

Figure 1 describes a sample framework showing the components and relationships that are used in data ingest and domain configuration.

Data ingest functional components and their relationships — Figure 1. Sample framework showing data ingest components and relationships

When the CSV file is exported from the data management system, the data is placed in a WebSphere® Message queue. A channel is configured to receive the data.

The data ingestion simulator (at /opt/IBM/water/MLHubAdapter/simulator_csv_receiver on the application server) monitors the incoming folder in simulator_csv_receiver for any CSV file. When a message that contains measurement data is placed in the incoming folder, the data in the CSV file is updated in the Device Management Enablement analytic data stores.

The simulator publishes CSV data to a topic. The topic name is entered in the simulator.properties file in the simulator. This topic name should be same as the topic name entered during channel creation.

Receiving data readings
Each transportation protocol has a receiver that is supplied raw data readings. The raw data readings are parsed and transformed for use in Device Management Enablement.
Setting up a domain model
Creating a domain involves working with the semantic model to add asset types, measurement types, property types, measurement units and relationships.
Ingesting a CSV file
Device Management Enablement uses a simulation tool to monitor incoming folders, parse CSV files, and send data readings. You can use the Simulator to check operational data flows sourced from external systems.
Creating derived value measurements
Creating derived value measurements involves setting up a derived value data source and channel, setting properties and rules for processing measurements, and checking that the data flow is operational. Derived values are calculated by the system based on other measurement readings.

Parent topic: Device Management Enablement tutorials