Email Listener: Removing Unwanted Text From Incoming Emails via Scripting

Body

Summary: In this blog entry we will present a method to customize the removal of patterns of characters from incoming email HTML message bodies before they become part of free form SR long descriptions.  We'll do this using the Automation Scripting application. 

Here's a collection of non-IBM Maximo HTML weirdness -- mostly email, some from the web -- collected without actively seeking it out.  It comes from sources such as LinkedIn, Google Translate, USA Cycling, WSJ and Outlook.  We see weird alignments and extraneous characters that we expect to see hidden as part of the raw HTML of the document.

When the Email Listener receives incoming free form emails, the content is written directly to the SR long description without any processing.  The raw bodies of HTML emails are like snowflakes, but there may be a pattern of characters or formatting common to all or some of those emails, which based on the tolerance of the customer, may range from a minor annoyance to a problem that they want resolved.  There is no one size fits all approach and in the listener application we will not hard code HTML processing. The listener is the thing-in-the-middle subjected to the vagaries of an email system and the rich text editor; enter the Maximo Automation Scripting application, the perfect tool for customized solutions.

Automation Scripting supports various script languages, so bring whatever expertise you possess to the fore.  In this example, we will be removing the text via a custom Java class and calling that class via script.  If you are on 7.5.0.X or 7.6.0.X, this class should be compiled with IBM SDK 1.6 or 1.7, respectively.  There will be *no* dependencies to Maximo in this utility class; we'll be using pure Java API.  Also, since this entails using a class file which will be staged in the Maximo directory structure, a rebuild and redeploy of the EAR file (maximo.ear) will be required.  If this is onerous, pursue an alternate scripting solution in a jython, javascript, etc.  Caveat:  do not perform this in production for the first time; use a test environment to do your initial implementation and to refine your script and class.

Below is sample source code for the simple utility class that we'll call from a Maximo automation script. Ensure the compiled class is placed in the binary folder indicated below as this will ensure when the script calls the class (via  import statement) it will be in the correct location.  Update the path and the Java package declaration to suit your desired package name. Here's a quick introduction to compiling a program Essentials, Part 1, Lesson 1: Compiling & Running a Simple Program (ignore the "running" section as this class is not a runnable application).

     Source placement: applications\maximo\businessobjects\src\com\yourcompany\html\ProcessRichTextEmail.java 

     Binary  placement: applications\maximo\businessobjects\classes\com\yourcompany\html\ProcessRichTextEmail.class

 

package com.yourcompany.html;

public class ProcessRichTextEmail {
    
    private static final String NONBREAKING_SPACE = " ";
    private static final String SINGLE_SPACE = " ";
    private static final String NOTHING = "";
    private static final String MATCH_EVERYTHING = ".*?";
    private static final String COMMENT_LEFT = "<!--";
    private static final String COMMENT_RIGHT = "-->";
    private static final String STYLE_BEGIN = "<style>";
    private static final String STYLE_END = "</style>";
    private static final String IMAGE = "<img ";
    
    public static String processMessage(String message)
    {    
        if ( message.contains(COMMENT_LEFT) && message.contains(COMMENT_RIGHT) )
        {
            message = message.replaceAll(COMMENT_LEFT + MATCH_EVERYTHING + COMMENT_RIGHT, NOTHING);            
            message = message.replaceAll("(?s)"+COMMENT_LEFT + MATCH_EVERYTHING + COMMENT_RIGHT, NOTHING);    
        }
        
        if ( message.contains(NONBREAKING_SPACE) )
        {
            message = message.replaceAll(NONBREAKING_SPACE,SINGLE_SPACE);
        }
        
        if ( message.contains(STYLE_BEGIN) && message.contains(STYLE_END) )
        {
            message = message.replaceAll(STYLE_BEGIN + MATCH_EVERYTHING + STYLE_END, NOTHING);
            message = message.replaceAll("(?s)"+STYLE_BEGIN + MATCH_EVERYTHING + STYLE_END, NOTHING);    
        }
        
        if ( message.contains(IMAGE) )
            message = message.replaceAll(IMAGE + MATCH_EVERYTHING + ">",NOTHING);
 
        return message;
    }
}

Once the class is compiled and the new ear is deployed, we're ready to log in, set up the script in the Automation Scripting application and perform an email test.

Create a Script with Object Launch Point as follows:

Step 1. Launch Point: HTML_LD
Object: SR
Events: Add & Update
Next

Step 2: Script: CLEAN_HTML
Script Language: jython
Add Variable:
Variable: LD
Launch Point Attribute: select DESCRIPTION_LONGDESCRIPTION
Next

Step 3: In description field note:  This is a script for custom HTML email cleanup.
Source Code (note indentation in final statement, python is sensitive white space):

from psdi.util import HTML
from com.yourcompany.html import ProcessRichTextEmail

if not interactive and HTML.isHtml(LD):
   LD = ProcessRichTextEmail.processMessage(LD)

Next we'll send a test email.  This email contains an example of each of the items we want stripped out of the email body before it hits the long description:  a style section, a comment, some non-breaking spaces and an image tag which will be orphaned (though an image specified as attachment via email header will be attached as a document to the record - see Email LIstener Image Handling).

Return-Path: pete@maximo.dev
Received: from [9.32.231.202] (lemond.swg.usma.ibm.com [9.32.231.202])
    by LEMOND with ESMTPA
    ; Mon, 19 Oct 2015 09:37:05 -0400
To: List Ener <listener@maximo.dev>
From: pete <pete@maximo.dev>
Subject: HTML Test
Message-ID: <5624F201.3050809@maximo.dev>
Date: Mon, 19 Oct 2015 09:37:05 -0400
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101
 Thunderbird/38.3.0
MIME-Version: 1.0
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8">
    <style><!-- Here's a comment --></style>
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <p>This is a test email.</p>    
    <p>Some nonbreaking space entity characters were here:     &nbsp;  &nbsp;  &nbsp; </p>    
    <p>An image tag was here:   <img src="image.png"></p>     
  </body>
</html>

Once the listener picks up the message, we'll go directly to the database to see the fruits of our labors.  Voilà!