Mapping similar ticket data from ServiceNow
If you need to use historical incidents from ServiceNow as similar ticket data, you need to map the data to an expected format and insert the data into Elasticsearch.
Historical incidents are required to first be inserted into Elasticsearch for similar ticket data. The data format consists of one incident record per line, in JSON format, with the original raw data. Raw data must be normalized to the IBM Cloud Pak for AIOps target format. This normalization includes mapping fields, converting date-time formatting, and more. Incidents are then inserted in batch. When the insertion is done, IBM Cloud Pak for AIOps tries a simple search to make sure everything runs as expected.
For more information about training similar tickets models in IBM Cloud Pak for AIOps, see Setting up training for similar tickets.
Raw similar ticket data
IBM Cloud Pak for AIOps ingests incidents from standard or custom data sources, for the incidents to be analyzed for similarities and action recommendations. Offline historical raw incidents are first uploaded to the Elastic database to extract indexes and set the stage for finding incidents that are similar to incoming live incident data.
You do not need to transform raw incident data into a normalized format. For more information about normalization, see Normalized output data schema.
Normalization is automatically done in similar incidents model training step. For example, the following raw JSON incident data comes from ServiceNow:
{
"source": "https://dev82395.service-now.com/incident.do?sysparm_query=number=INC0010062",
"number": "INC0010066",
"short_description": "Containers being killed in ts-payment-mongo service. Users may notice issues during payment",
"description": "",
"opened_at": "2020-01-08T22:48:23Z",
"sys_created_on": "2020-01-08T22:48:23Z",
"sys_updated_on": "2020-01-08T22:48:23Z",
"resolved_at": "2019-12-11T07:36:00Z",
"closed_at": "2019-12-11T07:36:00Z",
"business_duration": "",
"severity": 3,
"priority": 5,
"impact": 3,
"state": "closed",
"comments_and_work_notes": "",
"close_notes": "This issue has been resolved by the DBA team, which SQL queries were updated to reboot proxy agent to prevent the monitoring notification system from failing.",
"parent_incident": ""
}
In this example, you can extract data from ServiceNow. The source and format of your exported data doesn't matter.
For more information about exporting data from ServiceNow, see Exporting data .
For more information about managing incident data in ServiceNow, see Incident management .
Normalized similar ticket data
The following sample illustrates the result of mapping data from an external source for use with IBM Cloud Pak for AIOps:
{
"application_group_id": "1",
"application_id": "1",
"timestamp": 1570221600,
"utc_timestamp": "2019-10-04T20:40:00.038Z",
"type": "incident",
"incident": {
"incident_id": "fed6b7bf-f741-49d1-ab36-5d04e9a5c128",
"title": "Users are not able to access Discovery and Conversation in EU",
"description": "Unable to connect to email using my laptop.",
"created_at": "2019-09-11T06:39:48.000Z",
"updated_at": "2019-09-11T07:57:58.000Z",
"resolved_at": "2019-09-11T07:57:58.000Z",
"closed_at": "2019-09-11T07:57:58.000Z",
"started_at": "2019-09-11T07:57:58.000Z",
"business_duration_ms": 15347,
"severity": 1,
"priority": 2,
"impact": 3,
"state": "new",
"source": {
"source_name": "ServiceNow",
"source_url":"https://ibm.servicenow.com/INC0000060",
"source_incident_id": "INC0000060",
"source_application_id": "tokenizer124"
},
"comments": [
{
"comment_text":"Rebooted it."
}
],
"related_incidents":[
{
"source_incident_id":"INC0000059",
"relationship":"parent"
}
],
"resolution": {
"rca_id": "RCA00013547",
"resolution_summary": "The problem was resolved by rebooting the laptop."
},
"features": [
]
},
"meta_features": []
}
Description of similar ticket normalized attributes
Attribute | Description |
---|---|
title |
Short description or title of the incident. |
description |
Long description of the incident. |
timestamp/utc_timestamp |
Epoch timestamp of the event in the log entry and the utc_timestamp formatted as yyyy-mm-ddTHH:MM:SSZ . |
created_at |
Coordinated Universal Time at which the incident was created formatted as yyyy-mm-ddTHH:MM:SSZ . |
updated_at |
Coordinated Universal Time at which the incident was last updated formatted as yyyy-mm-ddTHH:MM:SSZ . |
resolved_at |
Coordinated Universal Time at which the incident was resolved formatted as yyyy-mm-ddTHH:MM:SSZ . |
closed_at |
Coordinated Universal Time at which the incident was closed formatted as yyyy-mm-ddTHH:MM:SSZ . |
started_at |
Coordinated Universal Time at which the incident started formatted as yyyy-mm-ddTHH:MM:SSZ . |
severity |
Severity of the issue (integer) |
priority |
Priority of the issue (integer) |
impact |
Impact of the issue (integer). |
source_name |
Data source in which the incident is stored (for example, ServiceNow). |
source_url |
URL of the incident in the source. |
source_incident_id |
incident_id from the source (for example, NUMBER in ServiceNow). |
source_application_id |
Application from which the incident came (for example, the configuration item in ServiceNow). |
comments |
Comments or closed notes that talk about the list of actions that are taken, or comments that are added by the user who is attending to the incident. |
related_incidents |
Any related incidents, such as parent incidents, child incidents, or CIE documentation that is attached with the incident. |
resolution |
Any RCA ID or resolution that is provided for the incident. |
Attribute | Description |
---|---|
business_duration |
Duration for which the incident was alive without resolution. |
Attribute | Description |
---|---|
application_group_id |
The application_group_id to support multitenancy. This value is provided as part of the initial configuration. |
application_id |
Application that IBM Cloud Pak for AIOps is monitoring. This value is provided as part of the initial configuration. |
type |
This value is hardcoded to the incident. |
timestamp |
Epoch time of the incident normalization, in seconds. |
utc_timestamp |
Normalization time in the Coordinated Universal Time time zone. |
incident_id |
The unique UUID. |
state |
Current state of the incident. |
Attribute | Description |
---|---|
features |
Placeholder for any features. |
meta_features |
Placeholder for service analysis. |