Integrate i2 COPLINK with the Intelligence Operations Center

Make information exchange easier

This article describes a method of integrating IBM® i2® COPLINK® data with the IBM Intelligent Operations Center. To achieve this we will take advantage of existing XML specifications, such as NIEM and LEXS, and tooling from the products including the COPLINK File Exporter from COPLINK and WebSphere Message Broker Toolkit from the Intelligent Operations Center.

Share:

Brian Daly (briandal@ie.ibm.com), Software Developer, IBM

Photo of Brian DalyBrian Daly is a Software Developer and a member of IBM Industry Solutions Development group at the Technology Campus at Mulhuddart, Ireland. He develops new content that extends the Industry Solutions offerings.



15 January 2013

Also available in Chinese

Overview

IBM i2 COPLINK is a database application that consolidates policing data, aids collaboration, and helps generate tactical leads. The IBM Intelligent Operations Center monitors and manages city services. It provides operational insight into daily city operations through centralized intelligence. Integrating the two systems helps enrich the information available to city managers, which helps them to determine and address potential problems quickly and effectively. This integration helps a city use its resources in a cohesive manner, helping make it a smarter city that is better equipped to manage minor incidents to major catastrophes.


Introducing NIEM, IEPD, LEXS, and CAP

NIEM

The National Information Exchange Model (NIEM) is an information sharing framework that allows national agencies to map their systems to the building blocks it provides. It reduces the complexity of sharing important information between agencies in a timely fashion without significant adjustment of the systems they maintain.

IEPD

An Information Exchange Package Documentation (IEPD) is a formal and structured XML-based representation of the rules that are required for a successful exchange of information between producers and consumers of information. An information producer uses NIEM to build an IEPD (for example, the IBM i2 COPLINK IEPD) to easily facilitate information exchange.

LEXS

The Logical Entity Exchange Specification (LEXS) builds on NIEM to provide a further framework to build IEPDs that both conform to NIEM and support a consistent means of performing publication, search, and retrieval operations. Whereas a NIEM IEPD is used for a specific information exchange between a producer and consumer of two specific systems, a LEXS IEPD presents a more fluid solution to the moving requirements and demands of different producers and consumers of information. This fluidity is achieved by having a base level of data known as the digest that all LEX implementations (producers and consumers) will comprehend.

CAP

The Common Alerting Protocol is an open XML based data format designed for the easy exchange alerts and notifications in a format that is easy to understand. The Intelligent Operations Center processes CAP messages, which must be converted from the LEXS messages sent by the IBM COPLINK File Exporter.


Transformation message flow

The following is a brief overview of the message flow, as shown in Figure 1:

  • The message flow begins with the File Input node, which reads in IBM i2 COPLINK dispatches from a remote file system using the sftp protocol.
  • The mapping node, doPub-CopAug, then strips out the structured payload from the LEXS message to retrieve the augmented IBM i2 COPLINK information. Extra fields are retrieved from the baseline information of the LEXS message, and then stored in the local environment so they can be accessed throughout the message flow.
  • The CopAug-CAP mapping node is used to map the IBM i2 COPLINK elements of the CoplinkAugmentation message to the CAP elements of the CAP message.
  • The filter node, checkAddress, is used to verify if the IBM i2 COPLINK dispatch contains valid address information since it will be required to geocode the address into GPS coordinates so they can be displayed upon the map portler in the IBM Intelligent Operations Center.
  • The Java™ compute node, geocodeCoords, queries an ESRI provided web service that will return GPS coordinates upon receipt of valid address information.
  • The filter node, checkEvent, checks that the CAP has a valid event assigned to it (Event data maps from DispatchTypeText from the IBM i2 COPLINK message).
  • The final filter node, checkCoords, checks that the GPS coordinates returned by the web service are valid. If the web service is provided with address information that is bad or incomplete it will return 0.0 for the longitude and latitude of the GPS coordinates.

With the completion of this final check, the validated CAP message is moved to the Input CAP message queue of Intelligent Operations Center, which is IOC.CAP.IN.

Figure 1. Complete transformation message flow
Complete transformation message flow

Message flow steps

Now, let's examine the separate steps of the message flow in greater detail.

The File Input node, the first node of the message flow, connects to a remote directory on the IBM i2 COPLINK File Exporter server using the sftp protocol to allow for the transfer of COPLINK information updates that the COPLINK File Exporter will export to the remote directory. Figure 2 shows an example for configuring this node.

Figure 2. Configuring File Input node for SFTP transfer
Configuring a File Input node for SFTP transfer

The File Input node listens for new files that have been exported by the IBM i2 COPLINK File Exporter, and securely transfers these to the message flow running on the Intelligent Operations Center Application and Integration server to begin the process of transforming the IBM i2 COPLINK dispatch data into the CAP format.

You can create the two mapping nodes, doPub-CopAug and CopAug-CAP (see Figure 3) using the CAP 1.2 schema and the IBM i2 COPLINK IEPD, which you can import into two separate message set projects. You can generate the CAP message definition file manually by clicking on File -> New -> Message Definition File From... -> XML schema file and choosing the CAP-v1.2.xsd file. The only global element that can be chosen is an alert element due to CAP being a relatively simple schema. The result is a CAP-v1.2.mxsd, or message definition file, and an alert message object that can be used in a message map by the mapping node.

Figure 3. Part of the message flow containing two mapping nodes to transform COPLINK IEPD output to CAP format
Message flow section

The IBM i2 COPLINK schema or IEPD (built from LEX 3.1.4 and NIEM 2.0) involves a number of .xsd files. You could import these manually just like the CAP specification but that would be a laborious process due to the many .xsd files involved. Instead, we can run a batch tool found in the WebSphere Message Broker Toolkit to process all of the .xsd files together. It will analyze the dependencies between them and generate the required message objects to use in a message map. To run the batch tool, unzip the LEX 3.1.4 schema into a folder somewhere on your system and put the coplink.xsd into this folder, as shown in Figure 4. An example of running this command is:

mqsicreatemsgdefs -p coplinkxsdauto -data
/home/briand/projects/con_coplink_ioc/workspaces/transformation26061012 -d
/home/briand/projects/con_coplink_ioc -rmp -rmd -ns

Note: To run this command successfully you must first shut down the WebSphere Message Broker Toolkit.

Find an explanation of this command and the parameters it uses in the section of the WebSphere Message Broker Information Center entitled, Importing from the command line.

Figure 4. Setting up batch creation of message definition files
Setting up batch creation of message definition files

The message flow depicted in Figure 3 illustrates the two-step process for assembling the required CAP message. The doPub-CopAug mapping node is the first step in this process and is composed of a message map called doPubCopAug.msgmap and a submap called doPubCopAug_submap0.msgmap.

You drag and drop the mapping node onto your message flow just as you would any other node. The message map can then be created in one of two ways, by double-clicking on the mapping node, or by File -> New -> Message Map. You are then asked to choose the source and target messages for the message map. Choose the doPublish message as the source as this will provide the means of getting at the structured payload as described earlier. Choose the CoplinkAugmentation message as the target for the message map. The CoplinkAugmentation message will be contained in a wildcard element within the structured payload of the LEX message. To reach it navigate through the source as follows: lexspd:doPublish -> lexs:PublishMessageContainer -> lexs:PublishMessage -> substitutions for lex:PublishMessageItemAbstract -> lexs:DataItemPackage -> specializations for lexs:StructuredPayload -> lexs:StructuredPayload -> Wildcard Element, then drag the Wildcard Element to the CoplinkAugmentation Type in the target. This is illustrated in Figure 5.

Figure 5. Message map of Wildcard Element -> CoplinkAugmentation
Message map of Wildcard Element

Figure 6 illustrates the Submap that will result from this action and the submap can be completed by dragging the CoplinkAugmentation Type from the source to the CoplinkAugmentation Type in the target. This will successfully map the IBM i2 COPLINK data from the LEX message but access is also required to some other LEX message elements for the CAP message from the parent message map. These can be stored in the local environment of the message flow for easy access by the submap. Save the submap and go back to the original message map which is doPubCopAug.msgmap in our example. Right click on the target pane of the message map -> Add or Remove Headers and Folders -> Selected headers and other folders, and then select LocalEnvironment.

Figure 6. Submap of Wildcard Element to CoplinkAugmentation
Submap of Wildcard Element

The extra LEX elements required in this instance were lexs:MessageDateTime, nc:AddressFullText, and lex:DataItemID. Figure 7 illustrates dragging one of the variables to the Wildcard Element of variables in the local environment and shows the map script for all three variables.

Figure 7. Adding variables to the local environment
Adding variables to LocalEnvironment

The IBM i2 COPLINK message and the extra LEX data stored in the local environment must be transformed into the CAP format that can be interpreted by the Intelligent Operations Center. Figure 7 is a screen capture of the message map CopAug_CAP.msgmap, which was created for the second mapping node of the message flow from Figure 3. Some of the CAP message elements are mapped from COPLINK, LEXS, and NIEM data, and some are defined as constants. Listing 1 shows the detailed mapping. Note: Although we made the GPS coordinates constant for this example but that will not be the case for actual IBM i2 COPLINK data.

Listing 1. Detailed mapping of the CAP message elements
cap:identifier -> $source/LocalEnvironment/Variables/lexs:DataItemID

cap:sender ->
$source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
/coplink:ReportingOrganizationText

cap:sent -> $source/LocalEnvironment/Variables/lexs:MessageDateTime 

cap:status -> 'Actual' 

cap:msgType -> 'Alert' 

cap:scope -> 'Public' 

cap:code -> 'Event' 

cap:info 

    cap:category ->       'Security' 
    
    cap:event -> $source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
    /coplink:DispatchTypeText
    
    cap:urgency ->        'Immediate' 
    
    cap:severity ->       'Severe' 
    
    cap:certainty ->      'Observed' 
    
    cap:headline -> '$source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
    /coplink:DispatchTypeText'
                
    cap:description -> $source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
    /coplink:DispatchTypeText
    
    cap:area 
                cap:areaDesc ->
    $source/LocalEnvironment/Variables/nc:AddressFullText
    
                cap:circle ->     '30.199192440061,-97.763281799772 0'
Figure 8. Message map CoplinkAugmentation -> CAP
Message map CoplinkAugmentation -> CAP

A Filter node is used to check that a full address has been provided. The following ESQL was written in the Filter node to provide this, and is a piece of validation to see if the areaDesc field of the CAP message is null. If the address is null then the message gets put in ADDRESS_NOT_EXIST queue. If the Filter node throws an error, the result is that the message goes to the ADDRESS_FAIL queue.

Listing 2. ESQL code to check for a valid address
CREATE FILTER MODULE CoplinkTransform_CheckAddress 
    CREATE FUNCTION Main() RETURNS BOOLEAN 
    BEGIN
        IF Root.XMLNSC.ns:alert.ns:info.ns:area.ns:areaDesc IS NULL THEN 
        RETURN FALSE; 
        ELSE
                RETURN TRUE; 
        END IF; 
    END;
END MODULE;

The next step of the messageFlow involves using a Java compute node, geocodeCoords, to use the address provided by COPLINK to geocode a valid longitude and latitude. Java code was written to read in the full address from the CAP message and split it out into Street, City, State, and ZIP code. A connection is made to the ESRI web service, which accepts an address as input and attempts to return valid GPS coordinates in the form of a JSON object. An IBM JSON parser is used to parse out the GPS coordinates and the piece of the code edits the CAP message inserting the GPS coordinates into the correct part of the message.

There are two other filter nodes performing validation checks similar to the address filter. Filter node checkEvent ensures that the CAP message isn't missing an event.

Listing 3. ESQL code to check for a valid event
CREATE FILTER MODULE CoplinkTransform_CheckEvent 
        CREATE FUNCTION Main() RETURNS BOOLEAN 
        BEGIN 
            IF Root.XMLNSC.ns:alert.ns:info.ns:event IS NULL THEN 
                RETURN FALSE; 
            ELSE 
                RETURN TRUE;
            END IF; 
        END;       
END MODULE;

Filter node checkCoords checks that any GPS coordinates that come back from the ESRI web service as 0.0 are not sent on to the Intelligent Operations Center for further processing.

Listing 4. ESQL code to check for valid GPS coordinates
CREATE FILTER MODULE CoplinkTransform_checkCoords 
        CREATE FUNCTION Main() RETURNS BOOLEAN 
        BEGIN 
            IF Root.XMLNSC.ns:alert.ns:info.ns:area.ns:circle = '0.0,0.0 0' THEN 
            RETURN FALSE; 
            ELSE
                RETURN TRUE; 
            END IF; 
        END; 
END MODULE;

The result of the message flow is valid CAP messages have been transformed from incoming COPLINK dispatch messages. The final stage is for them to go to the Intelligent Operations Center for processing, and this is achieved by placing them on a message queue that the Intelligent Operations Center has for processing CAP messages. The MQOutput node in the message flow in Figure 3 should be configured with queue IOC.CAP.IN on queue manager IOC.MB.QM to be processed by the Intelligent Operations Center. Figure 9 displays a number of sample IBM i2 COPLINK dispatches that have put through the message flow and have been successfully processed and displayed by the Intelligent Operations Center.

Figure 9. Sample COPLINK dispatches displayed in the Intelligent Operations Center
Sample COPLINK dispatches being displayed

Conclusion

This article demonstrates a flexible means of exchanging data between IBM i2 COPLINK and IBM Intelligent Operations Center using IBM WebSphere MQ message flows and taking advantage of messaging standards such as CAP, NIEM, and LEXS. The systems now are successfully integrated and users can now incorporate information from IBM i2 COPLINK. City managers can use features of the Intelligent Operations Center such as the executive dashboard to quickly get a powerful overview as to what is presently happening in the city, and to respond promptly and effectively to whatever arises.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Industries
ArticleID=838774
ArticleTitle=Integrate i2 COPLINK with the Intelligence Operations Center
publish-date=01152013