Overview

IBM i2 COPLINK is a database application that consolidates policing data, aids collaboration, and helps generate tactical leads. The IBM Intelligent Operations Center monitors and manages city services. It provides operational insight into daily city operations through centralized intelligence. Integrating the two systems helps enrich the information available to city managers, which helps them to determine and address potential problems quickly and effectively. This integration helps a city use its resources in a cohesive manner, helping make it a smarter city that is better equipped to manage minor incidents to major catastrophes.

Back to top

Introducing NIEM, IEPD, LEXS, and CAP

NIEM

The National Information Exchange Model (NIEM) is an information sharing framework that allows national agencies to map their systems to the building blocks it provides. It reduces the complexity of sharing important information between agencies in a timely fashion without significant adjustment of the systems they maintain.

IEPD

An Information Exchange Package Documentation (IEPD) is a formal and structured XML-based representation of the rules that are required for a successful exchange of information between producers and consumers of information. An information producer uses NIEM to build an IEPD (for example, the IBM i2 COPLINK IEPD) to easily facilitate information exchange.

LEXS

The Logical Entity Exchange Specification (LEXS) builds on NIEM to provide a further framework to build IEPDs that both conform to NIEM and support a consistent means of performing publication, search, and retrieval operations. Whereas a NIEM IEPD is used for a specific information exchange between a producer and consumer of two specific systems, a LEXS IEPD presents a more fluid solution to the moving requirements and demands of different producers and consumers of information. This fluidity is achieved by having a base level of data known as the digest that all LEX implementations (producers and consumers) will comprehend.

CAP

The Common Alerting Protocol is an open XML based data format designed for the easy exchange alerts and notifications in a format that is easy to understand. The Intelligent Operations Center processes CAP messages, which must be converted from the LEXS messages sent by the IBM COPLINK File Exporter.

Back to top

Transformation message flow

The following is a brief overview of the message flow, as shown in Figure 1:

• The message flow begins with the File Input node, which reads in IBM i2 COPLINK dispatches from a remote file system using the sftp protocol.
• The mapping node, doPub-CopAug, then strips out the structured payload from the LEXS message to retrieve the augmented IBM i2 COPLINK information. Extra fields are retrieved from the baseline information of the LEXS message, and then stored in the local environment so they can be accessed throughout the message flow.
• The CopAug-CAP mapping node is used to map the IBM i2 COPLINK elements of the CoplinkAugmentation message to the CAP elements of the CAP message.
• The filter node, checkAddress, is used to verify if the IBM i2 COPLINK dispatch contains valid address information since it will be required to geocode the address into GPS coordinates so they can be displayed upon the map portler in the IBM Intelligent Operations Center.
• The Java™ compute node, geocodeCoords, queries an ESRI provided web service that will return GPS coordinates upon receipt of valid address information.
• The filter node, checkEvent, checks that the CAP has a valid event assigned to it (Event data maps from DispatchTypeText from the IBM i2 COPLINK message).
• The final filter node, checkCoords, checks that the GPS coordinates returned by the web service are valid. If the web service is provided with address information that is bad or incomplete it will return 0.0 for the longitude and latitude of the GPS coordinates.

With the completion of this final check, the validated CAP message is moved to the Input CAP message queue of Intelligent Operations Center, which is IOC.CAP.IN.

Back to top

Message flow steps

Now, let's examine the separate steps of the message flow in greater detail.

The File Input node, the first node of the message flow, connects to a remote directory on the IBM i2 COPLINK File Exporter server using the sftp protocol to allow for the transfer of COPLINK information updates that the COPLINK File Exporter will export to the remote directory. Figure 2 shows an example for configuring this node.

The File Input node listens for new files that have been exported by the IBM i2 COPLINK File Exporter, and securely transfers these to the message flow running on the Intelligent Operations Center Application and Integration server to begin the process of transforming the IBM i2 COPLINK dispatch data into the CAP format.

You can create the two mapping nodes, doPub-CopAug and CopAug-CAP (see Figure 3) using the CAP 1.2 schema and the IBM i2 COPLINK IEPD, which you can import into two separate message set projects. You can generate the CAP message definition file manually by clicking on File -> New -> Message Definition File From... -> XML schema file and choosing the CAP-v1.2.xsd file. The only global element that can be chosen is an alert element due to CAP being a relatively simple schema. The result is a CAP-v1.2.mxsd, or message definition file, and an alert message object that can be used in a message map by the mapping node.

The IBM i2 COPLINK schema or IEPD (built from LEX 3.1.4 and NIEM 2.0) involves a number of .xsd files. You could import these manually just like the CAP specification but that would be a laborious process due to the many .xsd files involved. Instead, we can run a batch tool found in the WebSphere Message Broker Toolkit to process all of the .xsd files together. It will analyze the dependencies between them and generate the required message objects to use in a message map. To run the batch tool, unzip the LEX 3.1.4 schema into a folder somewhere on your system and put the coplink.xsd into this folder, as shown in Figure 4. An example of running this command is:

mqsicreatemsgdefs -p coplinkxsdauto -data
/home/briand/projects/con_coplink_ioc/workspaces/transformation26061012 -d
/home/briand/projects/con_coplink_ioc -rmp -rmd -ns

Note: To run this command successfully you must first shut down the WebSphere Message Broker Toolkit.

Find an explanation of this command and the parameters it uses in the section of the WebSphere Message Broker Information Center entitled, Importing from the command line.

The message flow depicted in Figure 3 illustrates the two-step process for assembling the required CAP message. The doPub-CopAug mapping node is the first step in this process and is composed of a message map called doPubCopAug.msgmap and a submap called doPubCopAug_submap0.msgmap.

You drag and drop the mapping node onto your message flow just as you would any other node. The message map can then be created in one of two ways, by double-clicking on the mapping node, or by File -> New -> Message Map. You are then asked to choose the source and target messages for the message map. Choose the doPublish message as the source as this will provide the means of getting at the structured payload as described earlier. Choose the CoplinkAugmentation message as the target for the message map. The CoplinkAugmentation message will be contained in a wildcard element within the structured payload of the LEX message. To reach it navigate through the source as follows: lexspd:doPublish -> lexs:PublishMessageContainer -> lexs:PublishMessage -> substitutions for lex:PublishMessageItemAbstract -> lexs:DataItemPackage -> specializations for lexs:StructuredPayload -> lexs:StructuredPayload -> Wildcard Element, then drag the Wildcard Element to the CoplinkAugmentation Type in the target. This is illustrated in Figure 5.

Figure 6 illustrates the Submap that will result from this action and the submap can be completed by dragging the CoplinkAugmentation Type from the source to the CoplinkAugmentation Type in the target. This will successfully map the IBM i2 COPLINK data from the LEX message but access is also required to some other LEX message elements for the CAP message from the parent message map. These can be stored in the local environment of the message flow for easy access by the submap. Save the submap and go back to the original message map which is doPubCopAug.msgmap in our example. Right click on the target pane of the message map -> Add or Remove Headers and Folders -> Selected headers and other folders, and then select LocalEnvironment.

The extra LEX elements required in this instance were lexs:MessageDateTime, nc:AddressFullText, and lex:DataItemID. Figure 7 illustrates dragging one of the variables to the Wildcard Element of variables in the local environment and shows the map script for all three variables.

The IBM i2 COPLINK message and the extra LEX data stored in the local environment must be transformed into the CAP format that can be interpreted by the Intelligent Operations Center. Figure 7 is a screen capture of the message map CopAug_CAP.msgmap, which was created for the second mapping node of the message flow from Figure 3. Some of the CAP message elements are mapped from COPLINK, LEXS, and NIEM data, and some are defined as constants. Listing 1 shows the detailed mapping. Note: Although we made the GPS coordinates constant for this example but that will not be the case for actual IBM i2 COPLINK data.

 
cap:identifier -> $source/LocalEnvironment/Variables/lexs:DataItemID

cap:sender ->
$source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
/coplink:ReportingOrganizationText

cap:sent -> $source/LocalEnvironment/Variables/lexs:MessageDateTime 

cap:status -> 'Actual' 

cap:msgType -> 'Alert' 

cap:scope -> 'Public' 

cap:code -> 'Event' 

cap:info 

    cap:category ->       'Security' 
    
    cap:event -> $source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
    /coplink:DispatchTypeText
    
    cap:urgency ->        'Immediate' 
    
    cap:severity ->       'Severe' 
    
    cap:certainty ->      'Observed' 
    
    cap:headline -> '$source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
    /coplink:DispatchTypeText'
                
    cap:description -> $source/coplink:CoplinkAugmentation/coplink:DispatchAugmentation
    /coplink:DispatchTypeText
    
    cap:area 
                cap:areaDesc ->
    $source/LocalEnvironment/Variables/nc:AddressFullText
    
                cap:circle ->     '30.199192440061,-97.763281799772 0'
            

A Filter node is used to check that a full address has been provided. The following ESQL was written in the Filter node to provide this, and is a piece of validation to see if the areaDesc field of the CAP message is null. If the address is null then the message gets put in ADDRESS_NOT_EXIST queue. If the Filter node throws an error, the result is that the message goes to the ADDRESS_FAIL queue.

 
CREATE FILTER MODULE CoplinkTransform_CheckAddress 
    CREATE FUNCTION Main() RETURNS BOOLEAN 
    BEGIN
        IF Root.XMLNSC.ns:alert.ns:info.ns:area.ns:areaDesc IS NULL THEN 
        RETURN FALSE; 
        ELSE
                RETURN TRUE; 
        END IF; 
    END;
END MODULE;

The next step of the messageFlow involves using a Java compute node, geocodeCoords, to use the address provided by COPLINK to geocode a valid longitude and latitude. Java code was written to read in the full address from the CAP message and split it out into Street, City, State, and ZIP code. A connection is made to the ESRI web service, which accepts an address as input and attempts to return valid GPS coordinates in the form of a JSON object. An IBM JSON parser is used to parse out the GPS coordinates and the piece of the code edits the CAP message inserting the GPS coordinates into the correct part of the message.

There are two other filter nodes performing validation checks similar to the address filter. Filter node checkEvent ensures that the CAP message isn't missing an event.

 

CREATE FILTER MODULE CoplinkTransform_CheckEvent 
        CREATE FUNCTION Main() RETURNS BOOLEAN 
        BEGIN 
            IF Root.XMLNSC.ns:alert.ns:info.ns:event IS NULL THEN 
                RETURN FALSE; 
            ELSE 
                RETURN TRUE;
            END IF; 
        END;       
END MODULE;
            

Filter node checkCoords checks that any GPS coordinates that come back from the ESRI web service as 0.0 are not sent on to the Intelligent Operations Center for further processing.

 

CREATE FILTER MODULE CoplinkTransform_checkCoords 
        CREATE FUNCTION Main() RETURNS BOOLEAN 
        BEGIN 
            IF Root.XMLNSC.ns:alert.ns:info.ns:area.ns:circle = '0.0,0.0 0' THEN 
            RETURN FALSE; 
            ELSE
                RETURN TRUE; 
            END IF; 
        END; 
END MODULE; 
            

The result of the message flow is valid CAP messages have been transformed from incoming COPLINK dispatch messages. The final stage is for them to go to the Intelligent Operations Center for processing, and this is achieved by placing them on a message queue that the Intelligent Operations Center has for processing CAP messages. The MQOutput node in the message flow in Figure 3 should be configured with queue IOC.CAP.IN on queue manager IOC.MB.QM to be processed by the Intelligent Operations Center. Figure 9 displays a number of sample IBM i2 COPLINK dispatches that have put through the message flow and have been successfully processed and displayed by the Intelligent Operations Center.

Back to top

Conclusion

This article demonstrates a flexible means of exchanging data between IBM i2 COPLINK and IBM Intelligent Operations Center using IBM WebSphere MQ message flows and taking advantage of messaging standards such as CAP, NIEM, and LEXS. The systems now are successfully integrated and users can now incorporate information from IBM i2 COPLINK. City managers can use features of the Intelligent Operations Center such as the executive dashboard to quickly get a powerful overview as to what is presently happening in the city, and to respond promptly and effectively to whatever arises.

Resources

Learn

• IBM Intelligent Operations Center (developerworks, March 2012): Read a good introduction to IBM Intelligence Operations Center.
  
  
• Creating a NIEM IEPD (developerWorks, May 2010): Learn more about NIEM from this four-part series of articles.
  
  
• NIEM website: Find documentation, and download the NIEM specifications.
  
  
• LEXS website Find similar information about LEXS.
  
  
• IBM IOC (developerworks, March 2012): Read a good introduction to IBM Intelligence Operations Center.
  
  
• IBM i2 COPLINK: This product page provides detailed information about the product and how to get it.
  
  
• IBM developerWorks Industries: Get all the latest industry-specific technical resources for developers.
  
  
• developerWorks on Twitter: Join today to follow developerWorks tweets.
  
  
• developerWorks podcasts: Listen to interesting interviews and discussions for software developers.
  
  
• developerWorks on-demand demos: Watch demos ranging from product installation and setup for beginners to advanced functionality for experienced developers.
  
  

Get products and technologies

• IBM Smarter City Solutions on Cloud: IBM Intelligent Operations Center on IBM SmartCloud offers a straightforward, user-based subscription service at a single price that includes all costs, including hardware, software, maintenance, support, and networking.
  
  
• IBM product evaluation versions: Download or explore the online trials in the IBM SOA Sandbox and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  

Discuss

• developerWorks profile: Create your profile today and set up a watchlist.
  
  
• developerWorks community: Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  
  

About the author

Brian Daly is a Software Developer and a member of IBM Industry Solutions Development group at the Technology Campus at Mulhuddart, Ireland. He develops new content that extends the Industry Solutions offerings.

Rate this article

Comments

Back to top

Table of contents

• Overview
• Introducing NIEM, IEPD, LEXS, and CAP
• Transformation message flow
• Message flow steps
• Conclusion
• Resources
• About the author
• Comments

Next steps from IBM

Rational products offer a complete collaborative lifecycle management set of tools for the IBM i platform for existing and new developers.

---

• Try: Download a free trial of Rational Developer for Power Systems, which provides an integrated development environment across the major platforms.
• Buy: Rational Developer for Power Systems

Dig deeper into Industries on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers