Extension and customization

There are two approaches to extending and customizing this feature:
  • Run multiple scripts for the same email.
  • Modify the supplied script.

Adding more scripts is better than adding more complexity to one script. The Orchestration & Automation application can ingest multiple categories of email messages from different apps. Some of the processing of the email messages might be common, and some processing could be category or app specific. Keeping the common processing in one script, and the specialized processing in others allows a cleaner and more maintainable implementation.

Each script execution is run within defined computational quota limits, which is 5 seconds of execution time or 50,000 lines of Python executed. Regular Expression processing is performed by the "re" (regular expression) Python module, execution of which is considered part of the quota. It is possible to create a complex regular expression whose execution requires a great many lines of Python to be interpreted on a particular email message. Execution of many such complex regular expressions could overrun the 50,000 line limit.

Example: Dealing with phishing reports

Scenario: Emails arriving in a particular mailbox reflect individuals forwarding suspected phishing messages. The scripts operating on these email messages should, in addition to the common processing in the sample script, record the reporter's email address as possibly having been the target of a phishing attack, and record the sender of the forwarded phishing email as suspicious.

A solution: Add the following Python 3 script to the Orchestration & Automation application:
import re

def addArtifact(regex, artifactType, description):
    """This method adds new artifacts to the incident derived from matches of the the regular expression  parameter within the email body contents.
    Parameter "regex" - a regular expression to match against the email body contents.  Parameter "artifactType" - the type of the artifact(s).
    Parameter "description" - the description of the artifact(s).  """
    # Using a set to enforce uniqueness
    dataList = set(re.findall(regex, emailmessage.body.content))
    if dataList is not None and len(dataList) > 0:
        map(lambda theArtifact: incident.addArtifact(
            artifactType, theArtifact, description), dataList)
###
# Mainline starts here
###
# Add "Phishing" as an incident type for the associated incident
incident.incident_type_ids.append("Phishing")
# Add the email sender information to the incident as the recipient of the phishing attempt
reportingUserInfo = emailmessage.from.address
if emailmessage.from.name is not None:
    reportingUserInfo = "{0} <{1}>".format(emailmessage.from.name, emailmessage.from.address)
    incident.addArtifact("Email Recipient", reportingUserInfo,
                         "Recipient of suspicious email")
# Extract email sender information on the assumption that a fishing email is being forwarded
if not emailmessage.body.content is None:
    addArtifact(r"From: (.*)\n", "Email Sender", "Suspicious email sender")
    addArtifact(r"Reply-To: (.*)\n", "Email Sender",
                "Suspicious email sender (Reply-To)")
It is important that the phishing-specific script above should run after the common script because the common script causes the incident variable to be set, and the phishing-specific script expects this to have been done already. The two simplest alternative ways to do this are:
  • The phishing-specific script should run as the second script of a multi-script rule that first runs the standard script.
  • The phishing-specific script should run in a separate rule that runs afterward.

In either case, the phishing-specific script should only run on the condition that the email message that was created indicates that it is a phishing report.

If you choose to implement the solution as one script, not two, then be sure that you add the Phishing incident type to the incident at a point in the script when the incident object exists. For example, the following command should occur after the incident is created or found.
incident.incident_type_ids.append("Phishing")

Example: Campaign identifier

Scenario: The email message subject alone might not be enough to collect related emails into one incident.

It might be that the email message subject is not specific or reliable enough to use as the way to collect related emails. In particular, there may be an attack taking place where multiple attack vectors are being employed in a single campaign, which may result in many different kinds of email messages being received for this one campaign.

One solution to the problem is to create a new field in an incident to contain a campaign identifier. This identifier could be either derived from the content of the email messages, or chosen from a hard-coded list when the campaign is recognized by the parsing script.

A solution:
  1. Create a new incident custom field for the campaign identifier, which must be of type Text.
  2. Copy the sample parsing script into a new script.
  3. Modify the new script to create a value for the campaign identifier either by selecting some text from the content of the email messages, or selecting from a hard-coded list of campaign identifiers if certain criteria are met.
  4. To associate the email message with an existing incident, search for incidents whose campaign identifier field is the same as the campaign identifier value for the email message. This would replace the search based on email message subject.
  5. If no suitable incident is found, create a new incident and set its campaign identifier field to be the campaign identifier value.
  6. Modify the rules so that the new script runs instead of the sample script.