Technical Blog Post
Abstract
Historical Database Gateways - An Overview
Body
Historical Database Gateways
The Netcool OMNIbus suite offers two database schemas as default historical database repositories – the Audit Schema and the Reporter Schema, with two corresponding modes for the gateway operation itself.
The Database Schemas
The database schemas – Audit and Reporter – are created using scripts included in the nco-g-jdbc-reporting-scripts and nco-g-jdbc-audit-scripts packages. Each package comes with the respective scripts to create the required supported database schema and the gateway configuration files - i.e:
- DB Schema creation scripts for Reporter Mode: Oracle, Sybase, MSSQL, DB2, Informix
- DB Schema creation scripts for Audit Mode: Oracle, Sybase, MSSQL, DB2, Informix, MySQL
Version Changes to Schemas
Newer releases of the Reporter Database Schema creation scripts now have a foreign key reference included to facilitate the removal of linked outdated records and feature improved indexing. In another departure from older practice, the end user is now expected to have a valid database instance and user configured prior to deploying the script.
The Difference between Audit and Reporter Mode
In Audit Mode, a new record is created in the database for each ObjectServer record insert, update or deletion transferred by the gateway. In Reporter Mode, all the changes to an alert through its lifecycle in the ObjectServer are stored by amending a single record, using the event's ServerName and ServerSerial values as unique record identifiers.
The JDBC Gateway
The Netcool Database Gateways have historically come in two database flavours:
Oracle (native/instant client connections) and non-Oracle (via ODBC).
The JDBC Gateway now combines the functionality of both types of gateways in one.
In addition, the gateway architecture previously only allowed individual record updates and inserts. The High Performance Oracle Gateway and subsequent gateways introduced bulk data processing.
To achieve this, the JDBC gateway splits work up into units to be processed by generic thread pool workers (count configurable using a property). Thus, the throughput can be scaled up by increasing the number of thread pool threads, up to the scalability threshold provided by the target database. As a result, no single type of operation acts as a bottleneck.
The gateway generally requires little if any tuning to achieve good performance. The default count of threads (three) provide good concurrency on most target databases. Increasing the thread count may place more burden on the target database however and cause throughput to plateau. It is a good idea to increase thread count if needed to the point where performance is good enough, then stop.
The JDBC Gateway has 3 modes of synchronisation, set by the Gate.Jdbc.ResyncMode parameter in the gateway.props file.
UNI - Resynchronisation is in one direction only - from OMNIbus to the target database. This is the regular resync supported by the old gateways, and processes alerts as if processing IDUC. In Reporting Mode, records which do not exist in the database are treated as new entries, and created as INSERTs, while those which already exist added are treated as UPDATEs.
BI - As UNI, but the gateway retrieves the set of existing alerts from the target database first. This selects alerts from the target database that are considered "open". In Reporting Mode, these are simply the alerts for which the DeletedAt value is null. In Audit Mode, these are alerts for which no 'D' (delete) record exists.
AUTO - If the cache contains 0 alerts, then BI, else UNI only. This catches the case of new gateway deployments, and the BI resync reads from the target database the set of existing alerts, so that the subsequent resync from OMNIbus doesn't generate constraint errors trying to INSERT alerts over existing alerts (existing alerts will now correctly be handled using UPDATEs)
Installing the Gateway
The latest gateway binary patch also contains the associated libngtk and libngjava libraries, so there is only one installation procedure required.
Connectivity is via the JDBC .jar driver available from the database vendor and needs to be copied to $OMNIHOME/gates/java. Your DBA will be able to confirm where these drivers are located or how they can be obtained.
Java Considerations
Currently none – the JDBC drivers currently used by the gateway are compatible with the latest release of Java 8.
Some JDBC Gateway Online References
Download information for the Database Schema Scripts
Replace the ODBC or Oracle Gateway with the JDBC Gateway
Support’s Guide to the JDBC Gateway
Create Historical Reporter Database on DB2 - Video
JDBC Gateway Installation on OMNIbus v8.1 - Video
Understanding the JDBC Gateway Database Record Update Rate
Other blogs in this series:
Reporter Database Schema 1 – The Dynamic Tables https://ibm.biz/BdjgV8
Reporter Database Schema 2 – The Static Tables https://ibm.biz/Bdjh3Z
Reporter Database Schema 3 – The Audit Tables https://ibm.biz/BdjAHH
Subscribe and follow us for all the latest information directly on your social feeds:
|
|
 |
|
| Check out all our other posts and updates: | |
| Academy Blogs: | https://goo.gl/eZjStB |
| Academy Videos: | https://goo.gl/kJeFZE |
| Academy Google+: | https://goo.gl/HnTs0w |
| Academy Twitter : | https://goo.gl/DiJbvD |
UID
ibm11081545