Netcool event load management
The event rate that the ObjectServer can handle is greater than a Netcool connector can handle. As a consequence, you may need to run more than one Netcool connector instance and distribute the event load amongst the them.
The need to distribute the event load arises when the Netcool connectors deployed are showing signs of the SAF file count growing constantly. This indicates that Netcool event load is breaching the connector's processing limit. Unmitigated piling of SAF files will cause issues in the pod's file system and the database for Netcool data files encompassing cache file, outbound SAF, and inbound SAF. This can cause an event delay or missing events in IBM Cloud Pak for AIOps.
To prevent storage issues, high event load must be distributed across multiple connectors.
The best practice is to split the event load and enable each connector's IDUC query on alternate alert rows to achieve evenness in event sourcing. In the Netcool connector UI configuration, append the following clause to the filter definition.
(mod(ServerSerial,X)=Y)
Where X
is the total number of connectors deployed and Y
is the index for each connector.
For example, if there are five connectors to be used, X=5 and Y ranges from 0 to 4:
connector_1: FILTER AND (mod(ServerSerial,5)=0)
connector_2: FILTER AND (mod(ServerSerial,5)=1)
connector_3: FILTER AND (mod(ServerSerial,5)=2)
connector_4: FILTER AND (mod(ServerSerial,5)=3)
connector_5: FILTER AND (mod(ServerSerial,5)=4)
Important: The filter definition must be same across this group of connectors.
Measuring event load and SAF file growth manageable by one connector
A connector's IDUC occurs every 30 seconds to fetch Netcool events from ObjectServer. In one IDUC cycle the number of SAF files generated is proportional to the events fetched. Each SAF file can buffer up to 1000 entries of events. The connector's SAF processor consumes SAF files for event data mapping and forwarding. SAF processor can complete processing two 100%-filled SAF files in a minute. Complexity of custom mapping rules and Kafka endpoint performance will be to some degree lengthning the processing time to get every event mapped and forwarded, but this SAF per minute baseline should serve as a guideline for planning the number of connectors to be deployed.
Clean up for a fresh start
You must clean up is mandatory on Netcool connector and AIOps when there are changes in any of the following items:
- Number of Netcool connector instances
- Filter definition
For details of cleaning up the deployment see the steps for resynchronizing the alerts from Netcool and AIOps described in this procedure