Creating a ServiceNow connection

Prerequisites

Before you create a ServiceNow connection, you must create users and configure ServiceNow. For more information about configuring ServiceNow, see Configuring the ServiceNow App for IBM Cloud Pak for Watson AIOps.

Important: Starting with IBM Cloud Pak® for Watson AIOps version 4.1.0, similar incidents, such as from a ServiceNow integration, are called similar tickets. Similar tickets are grouped with other ticket artifacts from BMC and Jira as input to the IBM Cloud Pak for Watson AIOps AI algorithms.

Important: Basic update functionality for incidents through the connection is added. Previously, incidents were opened in both IBM Cloud Pak for Watson AIOps and ServiceNow. Updates to incidents in IBM Cloud Pak for Watson AIOps did result in corresponding updates to the incidents in ServiceNow. Now, the connection supports live updates to incident state, priority, and description through the connection. This update support helps to ensure that incidents in both IBM Cloud Pak for Watson AIOps and ServiceNow remain synchronized. When an incident is closed in IBM Cloud Pak for Watson AIOps, the incident is closed in ServiceNow. This update support requires that your connected ServiceNow application is version 4.1 or later. If your ServiceNow application version is older than version 4.1, errors will occur.

Important: You can have only one ServiceNow connection per instance.

Note: ServiceNow instances with self-signed certificates are not supported for connecting with IBM Cloud Pak for Watson AIOps.

For more information about working with a ServiceNow connection, see the following sections:

• Creating a ServiceNow connection
• Enabling a ServiceNow connection
• Editing a ServiceNow connection
• Deleting a ServiceNow connection
• Troubleshooting a ServiceNow connection
• Known issue with a ServiceNow connection

For information about REST APIs and ServiceNow, see Configuring ServiceNow Observer jobs and Topology REST Observer APIs.

Creating a ServiceNow connection

To create a ServiceNow connection, complete the following steps:

1. Log in to IBM Cloud Pak Automation console.

2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.

3. On the Data and tool connections page, click Add connection.

4. From the list of available connections, find and click the ServiceNow tile.

   Note: If you do not immediately see the connection that you want to create, you can filter the tiles by type of connection. Click the type of connection that you want in the Category section.

5. On the side-panel, review the instructions and when ready to continue, click Connect.

6. On the Add connection page, define the general connection details:

   • Name: The display name of your connection.

   • Description: An optional description for the connection.

   • URL: The URL of your ServiceNow instance.

     Important: If you have any restrictive EgressNetworkPolicies in place, ensure that they are updated to allow for this outbound connection. For more information, see Postinstallation tasks.

   • User ID: ServiceNow user with roles x_ibm_waiops.admin.

     Important: You must create a new, unique user in ServiceNow specifically for your ServiceNow connection before you create your connection. Make sure that you are using the correct user role, the admin user, for your connection. You must also make sure that the time zone set for this user matches your system time zone in ServiceNow. For more information about ServiceNow users, see Managing roles.

   • Password: Your ServiceNow password for x_ibm_waiops.admin.

   • Select the orchestration or deployment type as Local or Remote.

     Note: If you select 'Remote' you will be provided with a bootstrap command to finish the orchestration.

   Click Test Connection to test the connection.

   

7. Click Next.

8. (Optional) Enter Field mapping information.

   

   With this field, you can map custom data fields to default fields in ServiceNow. The change_request and incident tables are queried from ServiceNow. These fields are used for AI training. These tables can be modified to include additional columns.

   • The following change_request fields are extracted:

     "u_environment", "u_environment_id", "cmdb_ci", "short_description", "u_purpose", "description", "backout_plan", "close_notes", "state", "close_category", "sys_created_by", "assigned_to", "sys_domain", "category", "business_service", "priority", "approval", "type", "contact_type", "production_system", "impact", "reason", "justification", "start_date", "end_date", "work_start", "work_end", "calendar_duration", "close_code","delivery_task", "implementation_plan", "test_plan", "work_notes_list"
     

   • The following incident fields are extracted:

     "short_description", "description", "u_caused_by_change", "caused_by", "close_code", "u_detection_source", "close_notes", "u_environment","cmdb_ci", "comments_and_work_notes", "impact", "problem_id", "priority", "severity", "state", "resolved_at", "closed_at", "opened_at", "parent", "made_sla", "watch_list", "upon_reject","sys_updated_on", "child_incidents", "hold_reason", "approval_history", "resolved_by", "sys_updated_by","opened_by", "user_input", "sys_created_on", "sys_domain", "sys_created_by", "knowledge", "order","calendar_stc", "delivery_plan", "contract", "active", "business_service", "sys_domain_path", "rfc","time_worked", "expected_start", "business_duration", "group_list", "work_end", "caller_id", "reopened_time", "approval_set", "subcategory", "correlation_display", "delivery_task", "work_start",
     "assignment_group", "additional_assignee_list", "business_stc", "calendar_duration", "notify", "service_offering", "sys_class_name", "closed_by", "follow_up", "parent_incident", "contact_type", "reopened_by", "incident_state", "urgency", "company", "reassignment_count", "activity_due", "assigned_to", "comments", "approval", "sla_due", "due_date", "sys_mod_count", "reopen_count", "sys_tags", "escalation", "upon_approval", "correlation_id", "location", "category", "work_notes_list"
     

   Example: If you add a custom column (named my_custom_field1_name) into the table, the prefix u_ is added so the field must be queried as u_my_custom_field1_name. If this field contains useful description information for training, you might want the existing description field to be updated for both change requests and incidents to use this data. To support this mapping, you need to add the following JSON to the Field mapping:

   {
     "codec": "servicenow",
     "description": "u_my_custom_field1_name"
   }
   

   With this mapping, the description value is replaced with the value from u_my_custom_field1_name. For example, assume the default description field has the value Server down for 2 hours due to an upgrade and the u_my_custom_field1_name field had the value Impact: server down, Duration: 2 hours, Cause: server upgrade. With the above mapping in place, the description now has the value Impact: server down, Duration: 2 hours, Cause: server upgrade. This description value is also now used for AI training. If the mapping is not in place, the u_my_custom_field1_name value is not used for AI training as it is not examined throught the connection.

   Adding the field mapping can give you flexibility to add your own custom fields into AI training. Also, if an existing field should not be used for training, you can add a new custom column with a default value to use for training instead of the existing field value.

   Notes:

   • The field mapping is applied to both historic and live data. Once the mapping is set in the connection and training is completed, the mapping should not be changed. Otherwise, if new change requests or incidents are queried, the mapping is not applied properly, which can cause results to be less accurate.

   • Currently, the field mapping applies to all tables. If a custom field does not exist in a table, a workaround is to create that custom field with default values to ensure that the mapping does not fail. Otherwise, the data is not be retrieved properly for that table.

9. (Optional) Enter the Collect topology data and Set advanced options information:

   • Collect inventory and topology data: Set to On for inventory and topology data collection. You can also set advanced options.

   • Schedule request: Set to On and this allows you set the Job Schedule (optional)

     • Start date: The start date of when the job is to run.

     • Time: The time to run the observer job.

     • Time interval (period): How frequently to run the job (either by hour, or by minute).

     • Interval: The duration of time between runs based on the Time interval (period). For example, if you wanted the job to run every 2 hours, set the interval to 2 and the Time interval (period) to hours. Enter 0 for the Interval to run the observer job one time (and manually through the interface otherwise).

   • Maximum number of records to retrieve per API call (100-10000): Number of records to retrieve per API call (100 - 10000). The default value is 10000.

   • Resource type mapping: Map ServiceNow resource types to Topology Manager resource types in the form c_type_1:asm_type_1,ci_type_2:asm_type_2. Topology Manager resource types should not include spaces.

   • Discoverable CI types: Specify ServiceNow Configuration Items (CI) types to discover in the form sys_class_name_1,sys_class_name_2. To discover all CI types, leave the field blank.

   • Connection timeout: The amount of time before a connection times out in milliseconds. The default value is 5000 milliseconds.

   • Read timeout: The amount of time before a read operation times out in milliseconds. The default value is 5000 milliseconds.

   • Proxy Host: The proxy host for your ServiceNow instance.

   • Proxy Port: The proxy port for your ServiceNow instance. The default value is 8080.

     

10. Click Next.

11. (Optional) Enter the Collect ticket data information. Set the Data collection toggle set to On to collect data. Then, select how you want to collect data:

    • Live data for continuous ticket data collection: To generate incidents and change risk assessments (after a model is trained).

      Note: Before you select this option, you must first set Mode to Historical data for ticket data collection to collect a minimum amount of data, and then run AI training on that data. For more information, see Planning data loading and training.

    • Live data for initial ticket data collection: A single set of training data used to define your AI model. Data collection takes place over a specified time period that starts when you create your connection. <!---not mentioned in Kevin's docs--->

      Note: If you select this option, you must disable your connection when you collect enough data for training. If you do not disable the connection, it continues to collect data. For more information about AI model training, including minimum and ideal data quantities, see Configuring AI training. For more information about disabling and enabling a connection, see Enabling and disabling a ServiceNow connection.

    • Historical data for ticket data collection: To train a change risk model (required to enable change risk assessment functions). A single set of training data used to define your AI model. You complete the 'Start' and 'End' date fields.

    • Ticket types: Specify the types of ServiceNow tickets that you want to pull in to the system. By default, change request, incident, and problem tickets are included and data is queried from those ServiceNow tables. You can edit this list to remove any types you don't want to sample. For change requests, the default sampling rate is 10-second intervals, to support the real-time requirements for change risk.

    • Sampling Rate: You must enter a sampling rate between 1 and 60 minutes. The default value is 1.

Note: If you selected Historical data for ticket data collection, you also need to enter a Start and End date.

Important: Different types of AI models have different requirements to properly train a model. Make sure that your settings satisfy minimum data requirements. For more information about how much data you need to train different AI models, see Configuring AI training.

12. Click Done.

You have created a ServiceNow connection in your instance.

After the connection creation is complete, and if you selected the Remote option for connector deployment, you can download the bootstrap script by selecting the created connection, and clicking download from the options menu (three vertical dots). If a script is regenerated, it invalidates the credentials of previously downloaded scripts for that connection.

Note: You can either copy the script or download as an sh file to run it on a remote cluster.

Prerequisites before running the bootstrap script on remote cluster:

• Podman needs to be installed on the remote cluster.
• Log in to the Image registry for the script to pull in the image that uses the Podman login command.

After you create your connection, you must enable the data collection to connect your connection with the AI of IBM Cloud Pak for Watson AIOps. For more information about enabling your connection, see Enabling a ServiceNow connection. To verify that your job completed successfully, view the schedule status of your ServiceNow connection in the Schedule connections tab.

To create more connections (such as a ChatOps connection), see Configuring data and tool connections.

For more information about working with the insights provided by your connections, see ChatOps insight management. For more information about change risk from ServiceNow, see Change risk.

Training a Change Risk model

Note: Make sure ServiceNow Integration is first set to Historical mode.

1. Go to IBM Cloud Pak Automation console.

2. Click AI Model Management.

3. Find the Change risk algorithm under the Training tab.

4. Click Set up training.

5. Verify your ServiceNow integration is listed under Data & tool connections.

   

6. Click Next for Schedule training. No need to schedule the training. It starts training in a few moments.

7. Click Next for Deploy. Set to deploy model On completion.

8. Click Done.

9. View details on the Change risk algorithm.

   

10. Click Start training. IBM Cloud Pak® for Watson AIOps will first check data quality before training.

    

11. Deploy model to enable change risk assessment functionality. Remember to change ServiceNow Integration to Live data mode

    

Navigation from an incident in Slack to ServiceNow

When a new anomaly is detected, IBM Cloud Pak for Watson AIOps generates an incident and sends the incident to users via Slack. A new incident is also created in ServiceNow and added to the incident shortly afterward in the form of an incident number and a link to the incident in ServiceNow (in the example INC0010072):

• WebUI view of the incident

  

• Slack view of the incident

  

Clicking the link opens the ServiceNow view of the incident:

Viewing a IBM Cloud Pak for Watson AIOps incident

1. Incident section.

   

   Number: Incident number used to identify the incident and match incident to incident across IBM Cloud Pak for Watson AIOps, Slack, and ServiceNow.

   Caller: User that was used to set up ServiceNow Integration in IBM Cloud Pak for Watson AIOps.

   Impact and Urgency: Impact and urgency are set to map the severity of the incident to the priority field of the incident in ServiceNow.

   Short description and Description: Set to the title and long description of the incident. The user can change the values in the ServiceNow ticket as well, but the changes are not sent back to IBM Cloud Pak for Watson AIOps.

2. IBM Cloud Pak for Watson AIOps section.

   

   Note: References to "story" or "stories" are synonymous with incidents. Any references to "story" in field names or details is equivalent to incident in all respects.

   Story number and id: Used to identify the incident in IBM Cloud Pak for Watson AIOps.

   Story URL: Link to incident in IBM Cloud Pak for Watson AIOps UI. Clicking the url opens the IBM Cloud Pak for Watson AIOps UI to the corresponding incident.

   Applications: List of names and urls of the applications that the incident is associated with. N/A if no applications detected.

   Topology URL: URL that links back to IBM Cloud Pak for Watson AIOps UI to view topology details.

3. IBM Cloud Pak for Watson AIOps tab.

   

   Same information from IBM Cloud Pak for Watson AIOps section above. Clicking the incident number links out to the IBM Cloud Pak for Watson AIOps table to view all IBM Cloud Pak for Watson AIOps stories in ServiceNow

4. IBM Cloud Pak for Watson AIOps Events tab.

   

   IBM Cloud Pak for Watson AIOps stories are made from a collection of alerts. View alerts that contributed to this incident in the IBM Cloud Pak for Watson AIOps Events tab.

Creating a Change Risk Assessment

1. Make sure ServiceNow Integration is set to Live data for continuous ticket data collection mode.

2. Go to Change > All to view all existing change requests.

3. Click New > Create Normal Change> Add a short description for your change. Then, click Submit.

   

4. View Change Request and see Change Risk Assessment tab.

   

5. Give feedback to IBM Cloud Pak for Watson AIOps by clicking thumbs up or thumbs down icons.

Enabling and disabling a ServiceNow connection

If you didn't enable your data collection during creation, you can enable your connection afterward. You can also disable a previously enabled connection the same way. If you selected Live data for initial AI training when you created your connection, you must disable the connection before AI model training. To enable or disable a created connection, complete the following steps:

1. Log in to IBM Cloud Pak Automation console.

2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.

3. On the Manage connections tab of the Data and tool connections page, click the ServiceNow connection type.

4. Click the connection that you want to enable or disable.

5. Go to the Collect ticket data section. Set Data collection to On or Off to enable or disable data collection. Disabling data collection for a connection does not delete the connection.

You enabled or disabled your connection. For more information about deleting a connection, see Deleting a ServiceNow connection.

Editing a ServiceNow connection

After you create your connection, your can edit the connection. For example, if you specified Historical data for initial AI training but you now want your connection to sample live data for continuous monitoring, you can edit it. To edit a connection, complete the following steps:

1. Log in to IBM Cloud Pak Automation console.

2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.

3. Click the ServiceNow connection type on the Manage connections tab of the Data and tool connections page.

4. On the ServiceNow connections page, click the name of the connection that you want to edit. Alternatively, you can click the options menu (three vertical dots) for the connection and click Edit. The connection configuration opens.

5. Edit your connection. Click Next to go through the connection configurationn pages. Click Save when you are done editing.

You edited your connection. If you did not enable ticket data collection when creating your connection, you can enable the connection directly from the interface. For more information about enabling and disabling your connection, see Enabling and disabling a ServiceNow connection. For more information about deleting a connection, see Deleting a ServiceNow connection.

Note: After you upgrade, you might find that the Next run field in the Scheduled connections tab is showing the last run time of the scheduled job, instead of the next run time. If this happens, the next run time should be regarded as the last run time.

Deleting a ServiceNow connection

If you no longer need your ServiceNow connection and want to not only disable it, but delete it entirely, you can delete the connection from the console.

Note: You must disable ticket data collection before you delete your ServiceNow connection. For more information about disabling ticket data collection, see Enabling and disabling a ServiceNow connection.

To delete a connection, complete the following steps:

1. Log in to IBM Cloud Pak Automation console.

2. Expand the navigation menu (four horizontal bars), then click Define > Data and tool connections.

3. Click the ServiceNow connection type on the Manage connections tab of the Data and tool connections page.

4. On the ServiceNow connections page, click the options menu (three vertical dots) for the connection that you want to delete and click Delete.

5. Enter the name of the connection to confirm that you want to delete your connection. Then, click Delete.

Your connection is deleted.

Note: Alerts that are generated from the ServiceNow connector are automatically cleared when the connection is deleted.

Troubleshooting a ServiceNow connection

If the ServiceNow change request, incident, or problem contains a large amount of text, such as work notes with close to 150,000 characters, the ticket is dropped and a warning is logged in the pod log. Dropping the ticket affects change risk for that ticket, as it will not occur or that ticket will not be used for similar ticket detection.

Known issue with a ServiceNow connection

If there is an active ServiceNow connection with data collection enabled and the ServiceNow credentials change, the ServiceNow user account can get locked out. ServiceNow has an automatic login locking script that is called "SNC User Lockout Check", which locks users out after more than five failed attempts (including any failed API calls).

If you check the Stories and alert page, you see also an alert saying "ServiceNow instance authentication failed".

When this problem occurs, unlock the user in ServiceNow. Then, change the password in the ServiceNow connection and save. When authentication fails in the ServiceNow connector, there is a 1-minute wait time before you can access it, to prevent a lockout from occurring quickly.