Audit logging

Edit online

The audit logging capabilities within IBM® Netcool® Operations Insight® on Red Hat® OpenShift® provide you with the capability to track changes that were made to your system, when the changes were made, and by whom. The recorded audit logs include the record of actions that were completed or attempted by users or services of your system. This auditing helps you with ensuring accountability, traceability, and regulatory compliance for data access, modification, and security for your IBM Netcool Operations Insight on OpenShift deployment.

Components used for audit logging

Audit logging is done by using the following components:

Dashboard Application Services Hub

Logging of user logon and logoff activity in IBM Netcool Operations Insight on OpenShift can be enabled by running a script that is provided as part of both the Tivoli® Integrated Portal and Dashboard Application Services Hub. The same script can be used to later disable security auditing, if wanted, since security logging can have an impact on performance.

What is audited

  • Account Management events

    Covers account management tasks, such as adding, changing, removing users, and managing their credentials and RBAC.

  • Authorization Checks

    Validates whether a user has a given role-based permission.

  • Privilege functions

    Covers the management of administrator's access controls (also known as access policies), and their ability to run admin level tasks.

  • Permission Changes

    Covers the management of user permissions.

How to enable, configure, and disable logging

To turn on security auditing, run the configureConsoleAudit.sh script. Pass in true as the value for the parameter that indicates whether security auditing is enabled or disabled.
cd <JazzSMHOME>/ui/bin
./configureConsoleAudit.sh smadmin <password> true
Note: You must restart Jazz® for Service Management after you run the script.
/opt/IBM/JazzSM/profile/bin/stopServer.sh
/opt/IBM/JazzSM/profile/bin/startServer.sh
For more information about how audit logs can be enabled, see Enable Security Auditing in Dashboard Application Service Hub (DASH).

How to access the logs

WebSphere® generates a Binary Audit log file, which contains the audit records for various actions that are performed in Dashboard Application Services Hub. The log file is created in the following directory.
<JazzSMHOME>/profile/logs/server1

The log file is named as BinaryAudit_JazzSMNode01Cell_JazzSMNode01_server1.log. The binary audit log file can be signed or encrypted for protection of audit data. For more information, see Security Auditing detail external icon in the WebSphere Application Server documentation.

IBM Tivoli Netcool/Impact

Records of user activity and report history are contained in the IBM Netcool Operations Insight audit log files.

What is audited

  • Audit log

    The audit log is a record of all user interactions with Netcool Operations Insight.

  • Report history log

    The report history log records the following data for Event Analytics reports:

    • Run ID
    • Date
    • Report Name
    • Report Type
    • Status
    • Start Date
    • End Date
    • Duration
    • Number of Events
    • Seasonal Events
    • Seasonality-Related Events Count
    • Related Events
    • Related Events Groups
    • Related Events Group Size
    • Suggested Patterns
    • Filter
    • Additional Related Events Filter
  • For drilling down to see content changes - what changed in the audit logs - read Viewing the differences between versions of a configuration file in Using restorative IBM Subversion(SVN) commands external icon topic in the Netcool/Impact documentation.
  • For more information about security auditing in Netcool/Impact, see Auditing Liberty events external icon.

How to enable, configure, and disable logging

For more information about event analytics from Netcool/Impact, see Netcool Operations Insight audit log files.

How to access the logs

The log files can be found at the following locations:

  • Audit log $IMPACT_HOME/logs/NCI_0_NOI_Audit.log
  • Report history log $IMPACT_HOME/logs/NCI_0_NOI_Report_History.log

Log example

  • Monitoring configuration changes (who, when, and what changed?)
      15 Mar 2023 16:23:11,108: DataSource 'ImpactDB' has been requested for editing by user 'impactadmin'
  15 Mar 2023 16:23:11,118: DataSource 'ImpactDB' has been opened for editing by user 'impactadmin'
  15 Mar 2023 16:23:17,991: DataSource 'ImpactDB' has been requested to be saved by user 'impactadmin'
  • Report history 
  •   "RunId_1519402261","2023-02-23 16:37:43.000","Seasonal_and_Related_events","Combined","COMPLETED","2022-07-01 00:00:00.0","2022-07-04 23:59:59.0","00:08:15","459418","3141","193731","244","3569","93","96","38","10","7","0","","((AlertGroup != 'Synthetic Event - Parent') OR AlertGroup IS NULL)"
  "RunId_1519405376","2023-02-23 17:30:53.000","Related_Events_1","RelatedEvents","COMPLETED","2022-07-01 00:00:00.0","2022-07-04 23:59:59.0","","459418","0","0","244","3569","93","96","38","10","7","0","","((AlertGroup != 'Synthetic Event - Parent') OR AlertGroup IS NULL)"
  "RunId_1523626088","2023-04-13 09:30:56.000","Seasonallity_only","Seasonality","COMPLETED","2021-02-01 00:00:00.0","2021-03-01 23:59:59.0","00:02:09","116961","3540","0","0","0","0","0","0","0","0","0","",""

Event management

Event management micro services share a common logging system. Log entries are JSON objects in the bunyan schema. Audit and diagnostic log entries are both streamed to stdout, which is collected by the Kubernetes log handling system. Audit log entries can be identified by their name property with the access value. Diagnostic log entries have another value for this property.

What is audited

  • Authentication or authorisation of incoming HTTP requests

  • Down stream HTTP requests to other micro services

    Requests from outside the event management service are usually processed by several micro services. Communication between micro services is logged allowing the full flow of requests to be identified.

  • Access to the CouchDB databases

    The CouchDB databases are accessed by HTTP requests. These requests are logged, which allows the documents that are accessed, the operation, and the result to be identified.

  • Account management events

    This covers account management tasks, such as adding, changing, removing users, and managing their credentials and RBAC.

  • Policy change

    Covers the management of user's access controls (also known as access policies). It does NOT include IBM Netcool Operations Insight on OpenShift policies, such as event suppression.

  • Permission changes

    Covers the management of user's permissions.

How to enable, configure, and disable logging

Logging is on by default. Audit log messages are logged at INFO level. The container’s LOG_LEVEL environment variable can be set to 31 or higher to turn off audit logging. This setting also disables most diagnostics logs.

How to access the logs

Use standard Kubernetes logging tools, including kubectl logs, stern, and log archiving systems. Log entries are JSON objects. Before you process the logs, remove any extraneous text that is introduced by the logging system.

The bunyan tool can be used to help remove extra text and to select just the audit logs, as in the following example.
stern ibm-cem-cem-users -o raw | bunyan --strict -c ‘this.name == “access”’

Logs are output in human readable form. Optionally, --bunyan can be used to keep the entries in single-line JSON format, or --json to return entries in formatted JSON.

Log examples

  • Authentication or authorisation of incoming HTTP requests:
      {
  "name": "access",
  "hostname": "evtmanager-ibm-cem-cem-users-6cfdd9bf84-2q7sq",
  "pid": 24,
  "level": 30,
  "req_id": "f9d83490-ae0c-11ed-aef6-c967b96d4fb8",
  "method": "GET",
  "url": "/api/usermgmt/v1/tenants/cfd95b7e-3bc7-4006-a4a8-a73a79c71255/userinfo",
  "username": "icpadmin",
  "statusCode": 200,
  "msg": "Audit",
  "time": "2023-02-16T15:17:07.351Z",
  "v": 0
  }
    • msg is Audit to identify an entry of this type.
    • method and url indicate the operation. See the HTTP requests table.
    • statusCode indicates the result. 2xx codes are successful. 401/403 codes indicate authentication or authorisation failures. Other codes are possible.
    • username indicates the authenticated user. The username is omitted on service to service requests.
    • req_id can be used to find other log entries for the same request within this micro service.
  • Down stream HTTP requests to other micro services:
      {
  "name": "access",
  "hostname": "evtmanager-ibm-cem-event-analytics-ui-6bdd8fbb8d-wnsvj",
  "pid": 24,
  "level": 30,
  "method": "GET",
  "url": "http://evtmanager-ibm-cem-incidentprocessor.noi.svc:6006/api/incidents/v1/user/icpadmin?eventsummary=true&includecounts=true&queue=individual&stream=true",
  "req_id": "49e3bb70-ae0e-11ed-a552-074876e36465",
  "server_req_id": "49e4cce0-ae0e-11ed-b03d-b9855fee0e59",
  "statusCode": 200,
  "msg": "Down stream request",
  "time": "2023-02-16T15:26:31.137Z",
  "v": 0
  }
    • msg is Down stream request to identify an entry of this type.
    • method and url indicate the downstream request.
    • statusCode indicates the result return.
    • req_id can be used to find other log entries for the same request within this micro service.
    • server_req_id can be used to find log entries for the request in the down stream micro service.
  • Access to the CouchDB databases:
      {
  "name": "access",
  "hostname": "evtmanager-ibm-cem-cem-users-6cfdd9bf84-2q7sq",
  "pid": 24,
  "level": 30,
  "req_id": "0bdeebc0-ae0d-11ed-aef6-c967b96d4fb8",
  "server_req_id": "de52b80b5f",
  "method": "POST",
  "host": "evtmanager-couchdb.noi.svc",
  "path": "/collabopsuser/_design/all_v18/_view/userSubscriptions",
  "user": "root",
  "statusCode": 200,
  "msg": "Database access",
  "time": "2023-02-16T15:17:37.570Z",
  "v": 0
  }
    • msg is Database access to identify an entry of this type.
    • path and method identify the document and the operation that is performed.
    • statusCode indicates the outcome.
    • req_id can be used to find other log entries for the same request within this micro service.
    • server_req_id indicates the request ID assigned by CouchDB.

Topology management

Topology management provides operations teams with complete up-to-date visibility and control over dynamic infrastructure and services.

What is audited

Data access
  • Search for applications, groups, and resources.
  • View applications and resource groups and related details.
  • View resources and their properties, comments, and status.
  • View topologies and their timelines.
  • View business criticality levels.
  • View resource group templates.
  • View resource types and their customized icons and styling.
  • View relationship types and their customized styling.
  • View right-click topology tools.
  • View topology rules.
  • View observer jobs.
  • View topology data administration routines.
  • View advanced topology settings.
Data changes
  • Modify user preferences.
  • Favorite applications and groups.
  • Create or modify applications.
  • Assign business criticality to an application, group, or resource.
  • Add comments to a resource.
  • Create or modify business criticality levels.
  • Create or modify resource group templates.
  • Modify resource types icons and styling.
  • Modify relationship types styling.
  • Create or modify right-click topology tools.
  • Create or modify topology rules.
  • Create or modify observer jobs.
  • Run topology data administration routines.
  • Create or modify advanced topology settings.
Data deletions
  • Delete applications.
  • Delete business criticality levels.
  • Delete resource group templates.
  • Delete resource types icons and styling.
  • Delete relationship types styling.
  • Delete right-click topology tools.
  • Delete topology rules.
  • Delete observer jobs.
  • Remove topology data administration routines.

How to enable, configure, and disable logging

The audit log messages are on by default, as they are logged at AUDIT log level, which is greater than the INFO level, the default logging level.

To disable audit logging, the administrator just sets the logging to a higher log level, for example, WARNING.

For more information, see Viewing the logs .

How to access the logs

Log messages are sent to stdout by the UI API, and hence can be accessed in the same way as for other microservices.

In Red Hat OpenShift Container Platform, use the oc logs command for the UI API pod.

Log example

AUDIT [2023-02-28 17:35:00.587] [REQ-000277] [HNAUA0169A] proxyRequest.send - Received "getObserverJobs" request from user ui-api-user (GET /1.0/ui-api/observers/jobs). Return status is 200 OK.
Audit log messages appear within the Agile Service Manager container or pod log output (stdout) alongside log messages of other levels, such as SEVERE, WARNING, INFO, and DEBUG.

IBM Netcool/OMNIbus Web GUI

The Web GUI has several log files to provide information on operations.

What is audited

  • Create, copy, modify filters.

  • Create, copy, modify views.

  • Create, modify relationships.

  • Create, modify tools.

  • Create, modify tool prompts.

  • Create, modify menus.

  • Create, modify metrics.

  • Register, modify CGI Registry.

  • Create, modify map.

  • Create, modify data sources.

  • Runs Event Database Query.

How to enable, configure, and disable logging

Set the log level to AUDIT or higher that is, ALL, FINEST, FINER, PROFILE, FINE, CONFIG, INFO, or AUDIT. For more information, see Setting the log level external icon in the Web GUI documentation.

How to access the logs

See Web GUI log files external icon in the Web GUI documentation.

Log example

Web GUI would write these log messages.
[2023-02-28T11:03:54]:AUDIT:HEMJS0457A:[WebContainer : 5]:User 'ldapadmin' was granted access to 'RAAPIServlet' via 'ncw_admin' role.
Audit messages carry the prefix code HEMxxnnnnA=, where A is for Audit, in JazzSM_HOME/profile/logs/ncw/ncw*.log or ncw*.trace log files. For instance, HEMFV is filters or views, HENHA is for HA, and HEMJS is user access. For more information, see Web GUI message codes external icon in the Web GUI documentation.

Runbook Automation

What is audited

Information related to running Ansible®, HTTP, or SSH automation on an external system.

How to enable, configure, and disable logging

  • Security audit logger is enabled by default.
  • No special configuration is needed to disable audit logging for RBA.

How to access the logs

Security audit log statements are included in the stdout stream of the rba-as pod, and can be obtained with the following command.
oc logs [-f] <rba-as-pod-name>

Log examples

  • Run Ansible automation
      {"name":"Security Audit Logger","hostname":"aiops-ir-core-rba-as.cp4waiops.svc","pid":60,"attachments":{"content":{"correlation_id":"d356a0b0-a5ef-11ed-92e0-b5dba43eda17"}},"initiator":{"id":"cfd95b7e-3bc7-4006-a4a8-a73a79c71255","name":"thgraser@de.ibm.com","typeURI":"security/account/user"},"level":30,"target":{"typeURI":"service"},"requestData":{"type":"POST","actionId":"AWX:job:fvt_job_simple"},"outcome":"success","reason":{"reasonCode":201,"message":"Created"},"eventTime":"2023-02-06T07:28:19.573Z","msg":"Security Audit Log Record","time":"2023-02-06T07:28:20.426Z","v":0}
  • Run Script automation
      {"name":"Security Audit Logger","hostname":"aiops-ir-core-rba-as.cp4waiops.svc","pid":60,"attachments":{"content":{"correlation_id":"43bac0c0-a5f0-11ed-92e0-b5dba43eda17"}},"initiator":{"id":"cfd95b7e-3bc7-4006-a4a8-a73a79c71255","name":"thgraser@de.ibm.com","typeURI":"security/account/user"},"level":30,"target":{"name":"rba701.fyre.ibm.com","typeURI":"system"},"requestData":{"actionId":"f30d2e0c-5517-4b9e-90ae-cdd3c41da008"},"action":"script.execute","outcome":"success","eventTime":"2023-02-06T07:31:28.309Z","msg":"Security Audit Log Record","time":"2023-02-06T07:31:28.978Z","v":0}
  • Run HTTP automation
      {"name":"Security Audit Logger","hostname":"aiops-ir-core-rba-as.cp4waiops.svc","pid":60,"attachments":{"content":{"correlation_id":"879338e0-a5f0-11ed-92e0-b5dba43eda17"}},"initiator":{"id":"cfd95b7e-3bc7-4006-a4a8-a73a79c71255","name":"thgraser@de.ibm.com","typeURI":"security/account/user"},"level":30,"target":{"name":"rba19-1.fyre.ibm.com","typeURI":"service"},"requestData":{"path":"/version.json","type":"GET","actionId":"80701bd5-9109-4e64-a2f3-83c5ecc34c45"},"outcome":"success","reason":{"reasonCode":200,"message":"OK"},"eventTime":"2023-02-06T07:33:20.509Z","msg":"Security Audit Log Record","time":"2023-02-06T07:33:20.865Z","v":0}