IBM Support

QRadar SOAR: QRadar Plug-in v5.0.0 - Template changes needed

Troubleshooting


Problem

In v5.0.0 templates that were working with earlier versions of the plug-in do not work for offense.local_destination_addresses and offense.source_addresses fields.
This problem has been resolved in 5.0.3.

Symptom

After upgrading the plug-in to 5.0.0, you might find that incidents are not created when an offense is escalated or incidents are not updated when an offense is updated. Errors such as the following taken from the circuits.log indicate a problem with the template.
2023-04-13 12:13:00,783 [soar_inbound] [Thread-1] INFO A new offense was found. Creating a case for offense ID 102775
2023-04-13 12:13:00,988 [actions_component] [MainThread] ERROR Traceback (most recent call last):
  File "/opt/app-root/app/components/soar_inbound.py", line 102, in _inbound_soar_escalator
    self.create_case_flow(qradar_offense, matched_rule, org_id=org_id)
  File "/opt/app-root/app/components/soar_inbound.py", line 234, in create_case_flow
    case_to_escalate = self.escalator.transform_offense(qradar_offense, matched_rule)
  File "/opt/app-root/app/apis/escalation_helper.py", line 452, in transform_offense
    case = template_functions.render_json(template, mapdata)
  File "/opt/app-root/app/lib/template_functions.py", line 339, in render_json
    result = render(template, data)
  File "/opt/app-root/app/lib/template_functions.py", line 320, in render
    stringvalue = jtemplate.render(data)
  File "/usr/local/lib/python3.6/site-packages/jinja2/environment.py", line 1291, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.6/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 67, in top-level template code
  File "/opt/app-root/app/lib/template_functions.py", line 200, in list_filter
    return ', '.join(val)
TypeError: sequence item 0: expected str instance, dict foundAs a result, the case_updates to follow fail and show IntegrationErrors.
File "/usr/local/lib/python3.6/site-packages/circuits/core/manager.py", line 874, in processTask
    raise value.extract()
  File "/usr/local/lib/python3.6/site-packages/resilient_circuits/actions_component.py", line 90, in _on_task
    yield result.get()
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.6/site-packages/resilient_circuits/decorators.py", line 176, in _invoke_inbound_app
    for r in ia_results:
  File "/opt/app-root/app/components/soar_inbound.py", line 220, in _inbound_soar_escalator
    raise IntegrationError(str(e))
resilient_lib.components.integration_errors.IntegrationError: ':  {"success":false,"title":null,"message":"The specified IP Address is invalid:  {&#39;id&#39;:","hints":[],"error_code":"generic"}'
2023-04-20 17:32:37,739 [escalation_helper] [Thread-23] INFO Token gathered and set
2023-04-20 17:32:37,740 [resilient_helpers] [Thread-23] DEBUG has_qradar_id() request
2023-04-20 17:32:37,740 [resilient_helpers] [Thread-23] DEBUG get_types_incident_fields() request
2023-04-20 17:32:37,740 [resilient_client] [Thread-23] DEBUG org_id 201 was passed to a SingleOrgClient.
2023-04-20 17:32:38,153 [connectionpool] [Thread-23] DEBUG https://<IP ADDRESS>:443 "GET /rest/orgs/201/types/incident/fields HTTP/1.1" 200 None
2023-04-20 17:32:39,611 [connectionpool] [Thread-23] DEBUG https://<IP ADDRESS>:443 "POST /rest/orgs/201/incidents HTTP/1.1" 400 None
2023-04-20 17:32:39,615 [api] [Thread-23] WARNING BasicHTTPException: 'resilient' API Request FAILED:
Response Code: 400
Reason: Unknown Reason. {"success":false,"title":null,"message":"The specified IP Address is invalid:  {&#39;id&#39;:","hints":[],"error_code":"generic"} in resilient.co3base.BaseClient.post.<locals>.__post, retrying in 2 seconds...
......
2023-04-20 17:33:12,077 [actions_component] [MainThread] ERROR Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/resilient/co3.py", line 474, in post
....
    raise BasicHTTPException(response)
resilient.co3base.BasicHTTPException: 'resilient' API Request FAILED:
Response Code: 400
Reason: Unknown Reason. {"success":false,"title":null,"message":"The specified IP Address is invalid:  {&#39;id&#39;:","hints":[],"error_code":"generic"}

Cause

Previous versions of the plug-in called the QRadar API as opposed to reading messages added to the inbound destination on SOAR, by the QRadar console server. The format of the data is different in messages added to SOAR's inbound destination by QRadar. These messages, for some offense properties, returns results as lists of dictionaries instead of the expected lists of strings.
Was
"source_addresses": [
"100.77.77.77"
]

Now

"source_addresses": [
{
"id": 8,
"ip": "100.77.77.77"
}
This problem has been reported to effect offense.local_destination_addresses and offense.source_addresses. These offense fields are often used in the artifact section of templates.
There is also a problem with offense.categories, which cannot be resolved with template changes. The fix requires a new version of the plug-in to be made available on the App Exchange.

Environment

v5.0.0 of the QRadar SOAR plug-in

Diagnosing The Problem

If offenses are not being escalated
  1. Enable DEBUG in the UI of the plug-in
  2. Download the logs
  3. Extract the zip file and look for errors in the circuits.log when an offense is created or updated

Resolving The Problem

Version 5.0.3 of the plug-in introduced fixes to many template related problems. Upgrade the plug-in to this version before you attempt manual configuration of the templates.
The following templates show the two affected offense fields, offense.local_destination_addresses and offense.source_addresses.
Was
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or (True and offense.local_destination_addresses|length > 0)  or True) %}
	   {%- for value in offense.source_addresses %}
     {{ artifact("IP Address", value, "QRadar Offense Source Address", properties=[{"name": "source", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
	   {%- endfor -%}
     {% set more_artifacts = (False or True) %}
	   {%- for value in offense.local_destination_addresses %}
     {{ artifact("IP Address", value, "QRadar Offense Local Destination Address", properties=[{"name": "destination", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
Now
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or (True and offense.local_destination_addresses|length > 0)  or True) %}
	   {%- for value in offense.source_addresses %}
     {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Source Address", properties=[{"name": "source", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
	   {%- endfor -%}
     {% set more_artifacts = (False or True) %}
	   {%- for value in offense.local_destination_addresses %}
     {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Local Destination Address", properties=[{"name": "destination", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
To make it easier to distinguish the changes, the changes are in red.
       {%- for value in offense.source_addresses %}
     {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Source Address", properties=[{"name": "source", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
       {%- endfor -%}
     {% set more_artifacts = (False or True) %}
       {%- for value in offense.local_destination_addresses %}
     {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Local Destination Address", properties=[{"name": "destination", "value": "true"}]) }}
To amend the template, perform the following steps.
  1. Download the affected template from the list of templates under the heading Template Files
  2. Edit the template in a text editor
  3. Upload the template to the plug-in to create a new template
  4. In Automatic Escalation Conditions, update Template to Use to point to the new template
  5. Create or update an offense that matches the Value Match Expression for the new template
  6. Was the offense escalated or was the incident updated?
  7. Download the logs and check circuits.log for errors if it is not escalated automatically
Further examples of changes made to templates.
Before
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or (True and offense.local_destination_addresses|length > 0)  or True) %}
       {%- for value in offense.source_addresses %}
     {{ artifact("IP Address", value, "QRadar Offense Source Address", properties=[{"name": "source", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
       {%- endfor -%}
     {% set more_artifacts = (False or True) %}
       {%- for value in offense.local_destination_addresses %}
     {{ artifact("IP Address", value, "QRadar Offense Local Destination Address", properties=[{"name": "destination", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
       {%- endfor %}{% set description = "QRadar Offense Source: " ~ offense.offense_type_name %}
     {%- if offense.offense_type in (0, 10) -%}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type in (1, 11) %}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_type == 3 %}
     {{ artifact("User Account", offense.offense_source, description) }}
     {% elif offense.offense_type == 4 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type == 5 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_type == 7 %}
     {{ artifact("System Name", offense.offense_source, description) }}
     {% elif offense.offense_type == 8 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type == 9 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% else %}
     {{ artifact("String", offense.offense_source, description) }}
     {%- endif -%}
     
  ],
After
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or (True and offense.local_destination_addresses|length > 0)  or True) %}
     {%- for value in offense.source_addresses %}
        {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Source Address", properties=[{"name": "source", "value": "true"}]) }} 
     {%- if not(loop.last) or more_artifacts %},{% endif %}
     {%- endfor -%}
     {% set more_artifacts = (False or True) %}
     {%- for value in offense.local_destination_addresses %}
        {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Local Destination Address", properties=[{"name": "destination", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
     {%- endfor %}{% set description = "QRadar Offense Source: " ~ offense.offense_type_name %}
     {%- if offense.offense_type in (0, 10) -%}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type in (1, 11) %}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_type == 3 %}
     {{ artifact("User Account", offense.offense_source, description) }}
     {% elif offense.offense_type == 4 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type == 5 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_type == 7 %}
     {{ artifact("System Name", offense.offense_source, description) }}
     {% elif offense.offense_type == 8 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type == 9 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% else %}
     {{ artifact("String", offense.offense_source, description) }}
     {%- endif -%}
     
  ],
Changes
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or (True and offense.local_destination_addresses|length > 0)  or True) %}
     {%- for value in offense.source_addresses %}
        {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Source Address", properties=[{"name": "source", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
     {%- endfor -%}
     {% set more_artifacts = (False or True) %}
     {%- for value in offense.local_destination_addresses %}
        {% set ip_address = value.ip if value.ip is defined else value %}
     {{ artifact("IP Address", ip_address, "QRadar Offense Local Destination Address", properties=[{"name": "destination", "value": "true"}]) }}
     {%- if not(loop.last) or more_artifacts %},{% endif %}
Before
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or True) %}{% set description = "QRadar Offense Source: " ~ offense.offense_type_name %}
     {%- if offense.offense_type in (0, 10) -%}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type in (1, 11) %}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_type == 3 %}
     {{ artifact("User Account", offense.offense_source, description) }}
     {% elif offense.offense_type == 4 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type == 5 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_type == 7 %}
     {{ artifact("System Name", offense.offense_source, description) }}
     {% elif offense.offense_type == 8 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_type == 9 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% else %}
     {{ artifact("String", offense.offense_source, description) }}
     {%- endif -%}
     
  ],
After
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or True) %}{% set description = "QRadar Offense Source: " ~ offense.offense_indexer.name %}
     {%- if offense.offense_indexer.offense_type_id in (0, 10) -%}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id in (1, 11) %}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 3 %}
     {{ artifact("User Account", offense.offense_source, description) }}
     {% elif offense.offense_indexer.offense_type_id == 4 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 5 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 7 %}
     {{ artifact("System Name", offense.offense_source, description) }}
     {% elif offense.offense_indexer.offense_type_id == 8 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 9 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% else %}
     {{ artifact("String", offense.offense_source, description) }}
     {%- endif -%}
{%- }
     
  ],
Changes
{%- endmacro %}{
  "artifacts": [
     {% set more_artifacts = (False or True) %}{% set description = "QRadar Offense Source: " ~ offense.offense_indexer.name %}
     {%- if offense.offense_indexer.offense_type_id in (0, 10) -%}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id in (1, 11) %}
     {{ artifact("IP Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 3 %}
     {{ artifact("User Account", offense.offense_source, description) }}
     {% elif offense.offense_indexer.offense_type_id == 4 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 5 %}
     {{ artifact("MAC Address", offense.offense_source, description, properties=[{"name": "destination", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 7 %}
     {{ artifact("System Name", offense.offense_source, description) }}
     {% elif offense.offense_indexer.offense_type_id == 8 %}
     {{ artifact("Port", offense.offense_source, description, properties=[{"name": "source", "value": "true"}]) }}
     {% elif offense.offense_indexer.offense_type_id == 9 %}

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSA230","label":"IBM Security QRadar SOAR"},"ARM Category":[{"code":"a8m0z0000001gyGAAQ","label":"Integrations-\u003EQRadar app"}],"ARM Case Number":"TS012871839","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEGM63","label":"IBM Security QRadar SOAR on Cloud"},"ARM Category":[{"code":"a8m0z0000001gyGAAQ","label":"Integrations-\u003EQRadar app"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSTDPP","label":"IBM Cloud Pak for Security"},"ARM Category":[{"code":"a8m0z0000001h8pAAA","label":"Support-\u003ECases"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
31 July 2023

UID

ibm16989129

Page Feedback