IBM Support

Historical Database Gateways - An Overview

Technical Blog Post


Abstract

Historical Database Gateways - An Overview

Body

 

 

 

Historical Database Gateways

 

The Netcool OMNIbus suite offers two database schemas as default historical database repositories – the Audit Schema and the Reporter Schema, with two corresponding modes for the gateway operation itself.

 

The Database Schemas

The database schemas – Audit and Reporter – are created using scripts included in the nco-g-jdbc-reporting-scripts and nco-g-jdbc-audit-scripts packages. Each package comes with the respective scripts to create the required supported database schema and the gateway configuration files - i.e:

 

  1. DB Schema creation scripts for Reporter  Mode:                        Oracle, Sybase, MSSQL, DB2, Informix
  2. DB Schema creation scripts for Audit Mode:                               Oracle, Sybase, MSSQL, DB2, Informix, MySQL

 

Version Changes to Schemas

Newer releases of the Reporter Database Schema creation scripts now have a foreign key reference included to facilitate the removal of linked outdated records and feature improved indexing. In another departure from older practice, the end user is now expected to have a valid database instance and user configured prior to deploying the script.

 

The Difference between Audit and Reporter Mode

In Audit Mode, a new record is created in the database for each ObjectServer record insert, update or deletion transferred by the gateway. In Reporter Mode, all the changes to an alert through its lifecycle in the ObjectServer are stored by amending a single record, using the event's ServerName and ServerSerial values as unique record identifiers.

 

The JDBC Gateway

The Netcool Database Gateways have historically come in two database flavours:

Oracle (native/instant client connections) and non-Oracle (via ODBC).

The JDBC Gateway now combines the functionality of both types of gateways in one.

In addition, the gateway architecture previously only allowed individual record updates and inserts. The High Performance Oracle Gateway and subsequent gateways introduced bulk data processing.

To achieve this, the JDBC gateway splits work up into units to be processed by generic thread pool workers (count configurable using a property). Thus, the throughput can be scaled up by increasing the number of thread pool threads, up to the scalability threshold provided by the target database. As a result, no single type of operation acts as a bottleneck.

The gateway generally requires little if any tuning to achieve good performance. The default count of threads (three) provide good concurrency on most target databases. Increasing the thread count may place more burden on the target database however and cause throughput to plateau. It is a good idea to increase thread count if needed to the point where performance is good enough, then stop.

The JDBC Gateway has 3 modes of synchronisation, set by the Gate.Jdbc.ResyncMode parameter in the gateway.props file.

 

UNI - Resynchronisation is in one direction only - from OMNIbus to the target database. This is the regular resync supported by the old gateways, and processes alerts as if processing IDUC. In Reporting Mode, records which do not exist in the database are treated as new entries, and created as INSERTs, while those which already exist added are treated as UPDATEs.

BI - As UNI, but the gateway retrieves the set of existing alerts from the target database first. This selects alerts from the target database that are considered "open". In Reporting Mode, these are simply the alerts for which the DeletedAt value is null. In Audit Mode, these are alerts for which no 'D' (delete) record exists.

AUTO - If the cache contains 0 alerts, then BI, else UNI only. This catches the case of new gateway deployments, and the BI resync reads from the target database the set of existing alerts, so that the subsequent resync from OMNIbus doesn't generate constraint errors trying to INSERT alerts over existing alerts (existing alerts will now correctly be handled using UPDATEs)

 

Installing the Gateway

The latest gateway binary patch also contains the associated libngtk and libngjava libraries, so there is only one installation procedure required.

Connectivity is via the JDBC .jar driver available from the database vendor and needs to be copied to $OMNIHOME/gates/java. Your DBA will be able to confirm where these drivers are located or how they can be obtained.

 

Java Considerations

Currently none – the JDBC drivers currently used by the gateway are compatible with the latest release of Java 8.

 

Some JDBC Gateway Online References

 

Operational Guide

List of Supported Databases

Download information for the Database Schema Scripts

Replace the ODBC or Oracle Gateway with the JDBC Gateway

Support’s Guide to the JDBC Gateway

Create Historical Reporter Database on DB2 - Video

JDBC Gateway Installation on OMNIbus v8.1 - Video

Gateway Configuration - Video

Understanding the JDBC Gateway Database Record Update Rate

 

Other blogs in this series:

 

Reporter Database Schema 1 – The Dynamic Tables https://ibm.biz/BdjgV8
Reporter Database Schema 2 – The Static Tables https://ibm.biz/Bdjh3Z
Reporter Database Schema 3 – The Audit Tables https://ibm.biz/BdjAHH

 

 


image

 

Subscribe and follow us for all the latest information directly on your social feeds:

 

 

image

 

image

 

image

 

 

  

Check out all our other posts and updates:

Academy Blogs:https://goo.gl/eZjStB
Academy Videos:https://goo.gl/kJeFZE
Academy Google+:https://goo.gl/HnTs0w
Academy Twitter :https://goo.gl/DiJbvD
 


image
 

 

 

[{"Business Unit":{"code":"BU004","label":"Hybrid Cloud"},"Product":{"code":"","label":""},"Component":"","Platform":[{"code":"","label":""}],"Version":"","Edition":""}]

UID

ibm11081545