Case study: Invoice processing with IBM Business Process Manager, Part 2: System integration

This two-part series provides a case study of a real world IBM Business Process Manager V8.0.1 invoice processing solution, using web services and Java™ integration to automate a process spanning multiple vendors and operational systems. Part 2 focuses on the Java and web service integration, and user interface and image manipulation techniques involved in the solution. This content is part of the IBM Business Process Management Journal.

Share:

Scott Glen (scott.glen@uk.ibm.com), Certified IT Architect, IBM

Scott Glen photoScott Glen is a Certified IT Architect in IBM Software Group, and the Lead Architect in Europe, the Middle East, and Africa (EMEA) for the Global Business Services SWG Smarter Process Centre of Competence. He has over 20 years of experience in the architecture and design of complex solutions, providing leadership to clients in the finance, government, telecommunications and media sectors across EMEA.


developerWorks Contributing author
        level

22 May 2013

Also available in Chinese

Solution recap

In the first part of this two-part series, we introduced an invoice processing solution, based on IBM Business Process Manager (BPM) Standard V8.0.1. We described the business challenges encountered when interoperating with multiple third party providers, and defined a business process model, with supporting data structures, that formed the backbone of a reusable solution. Figure 1 shows an overview of the solution.

Figure 1. Solution overview
Solution overview

Integration techniques

In this article, we'll focus more on the technical aspects of the solution, specifically looking at the Java and web service integration, as well as the user interface and image manipulation techniques involved in the solution. In particular, we will examine how the TWObject and TWList objects support complex integration with Java, which can expand the capabilities of a BPM solution.

Java integration

Java code is used throughout the solution to interrogate and manipulate the contents of the invoice files provided by the scanning and indexing providers. The most interesting aspect of this involves mapping between complex business objects in BPM, and their Java representations, which have been derived from associated XML structures.

Part 1 introduced TEAPPS, an XML-based industry standard for describing invoices, and demonstrated how JAXB can be used to marshal TEAPPS data between Java and XML representations. In this article, we examine the relationship between Java and BPM, specifically the representation of BPM data structure in Java, through the use of the TWObject and TWList classes. These are generic Java classes, provided by BPM, which can be used to facilitate complex integration with Business Objects.

First, let's have a quick recap on the TEAPPS structure, shown in Listing 1.

Listing 1. TEAPPS XML segment
<?xml version="1.0" encoding="ISO-8859-1"?>
<INVOICE_CENTER>
	<TRANSPORT_FRAME>
		<TF_CODE>TF01</TF_CODE>
		<TIMESTAMP>20130119142648753</TIMESTAMP>
		<BATCH_ID>13777339</BATCH_ID>
		<CONTENT_RECEIVER>
			<RECEIVER_REF>BWEISE-0012928</RECEIVER_REF>
			<CONTENT_REF>BPMS-Invoice11</CONTENT_REF>
		</CONTENT_RECEIVER>
		<SENDER>BW003705925424</SENDER>
		<FB_REQUEST>1</FB_REQUEST>
	</TRANSPORT_FRAME>
	…

This section of a TEAPPS file shows the <TRANSPORT_FRAME> header element, which contains a nested <CONTENT_RECEIVER> element. The JAXB class that maps to this element is shown in Listing 2.

Listing 2. TEAPPS_ContentReceiver class
publicclass TEAPPS_ContentReceiver 
{
	private String receiverRef;
	private String contentRef;

	@XmlElement (name="RECEIVER_REF")
	public String getReceiverRef() {
		return receiverRef;
	}

	@XmlElement (name="CONTENT_REF")
	public String getContentRef() {
		return contentRef;
	}

	publicvoid setReceiverRef(String receiverRef) {
		this.receiverRef = receiverRef;
	}

	publicvoid setContentRef(String contentRef) {
		this.contentRef = contentRef;
	}
}

The BPM process requires access to this information because the contentRef attribute contains a unique value that is used to identify a TEAPPS formatted invoice. A Java integration is used to access the data through the checkForIndexedResponse method which is contained within the InvoiceManager class. This integration is configured in BPM as shown in Figure 2.

Figure 2. CheckForIndexingResponse API
CheckForIndexingResponse API

The method monitors the incoming directory and extracts the content of any TEAPPS files that are found there, before transferring them to the processed directory. The results are then returned to BPM and stored in the indexedDocuments variable, which is an instance of the IndexedDocuments business object shown in Figure 3.

Figure 3. IndexedDocuments business object
IndexedDocuments business object

This object contains a list of IndexedDocument (singular) objects, each of which is essentially a wrapper around the central Invoice object introduced in Part 1. The key elements of interest in the checkForIndexingResponse method that populates this data structure are shown in Listing 3.

Listing 3. CheckForIndexingResponse method
public TWObject checkForIndexingResponse (String incomingDir, String processedDir)
{
  // create BPM results object
  TWObject results = TWObjectFactory.createObject ("IndexedDocuments");
	
  // create list of document objects 
  TWList docList = TWObjectFactory.createList ("IndexedDocument");
					
  // for each incoming response file
  for (int i = 0; i < listOfFiles.length; i++)
  {
    // get top level TEAPPS element				
    TEAPPS_InvoiceCenter tic = tm.getInvoiceCenter (listOfFiles[i].getName());
	
    // move file to processed dir
    FileUtilities.moveFile (incomingDir, processedDir, listOfFiles[i]);
								
    // format for BPM objects
    TWObject indexedDoc = TWObjectFactory.createObject ("IndexedDocument");
    TWObject invoice = TWObjectFactory.createObject ("Invoice");
					
    // populate invoice - get key field for correlation
invoice.setPropertyValue ("inputDocId", 
	tic.getTransportFrame().getContentReceiver().getContentRef());
						
    // populate remaining invoice fields
invoice.setPropertyValue ("receiverRef", 
	tic.getTransportFrame().getContentReceiver().getReceiverRef());
		…
		…	
		
    // populate invoice line details	
    TWList invoiceLines = TWObjectFactory.createList ("InvoiceLine");
    if (tic.getContentFrame().getInvoices().getInvoice().getRows() != null)
    {
      // for each row in the invoice					 
      List<TEAPPS_Row> rows = tic.getContentFrame().getInvoices().
					getInvoice().getRows().getRows();
      for (int j = 0; j < rows.size(); j++)
      {
        // get line data
        TWObject line = TWObjectFactory.createObject("InvoiceLine");
        line.setPropertyValue("lineNumber",rows.get(j).getRowNumber());
		…
		…
	 // add (line) TWObject to (lines) TWList  
        invoiceLines.addArrayData(line);
      }
    }
	
    // vendor data
    TWObject vendor = TWObjectFactory.createObject ("Vendor");
    vendor.setPropertyValue ("number", 
	tic.getContentFrame().getInvoices().getInvoice().
	getPayee().getCustomerInformation().getCustomerId());
		…
		…					
    // add lines (TWList) to invoice (TWObject) 
    invoice.setPropertyValue ("invoiceLines", invoiceLines);
						
    // add invoice to indexed Doc
    indexedDoc.setPropertyValue ("invoice", invoice);  		
						
    // add indexed doc (TWObject) to docList (TWList)
    docList.addArrayData(indexedDoc);
  }
					
  // finally add indexed doc list to single results object
  results.setPropertyValue ("indexedDocumentList", docList);			
  return results;
}

First note that the method signature matches the configuration of the Java integration in BPM. It accepts two strings and returns a generic TWObject, which is in fact an instance an IndexedDocuments (plural) object, held in the local results variable. As shown in Figure 2, this business object contains a list of IndexedDocument (singular) objects. Therefore, the code creates a TWList, called docList, to represent this data structure.

A number of different TWObject and TWList objects are subsequently created within the method, representing the nested structure of the TEAPPS file and the associated BPM objects. The relationship between this code and the data structures is illustrated in Figure 4.

The outermost IndexedDocuments structure is essentially a wrapper, holding a list of Indexed Document objects, each of which maps to a single Invoice within the batch. This Invoice element is itself a wrapper, containing a Vendor details element as well as a list of Invoice Line elements, which represent the individual products or services related to the Invoice.

Figure 4. IndexedDocuments structure
IndexedDocuments structure

Each element defined in Figure 4 has a matching line of code in Listing 3 (highlighted in red), which creates the associated object and gradually builds up the overall structure. This represents one of the most complex Java to BPM mappings we have seen, but highlights a pattern where nested TWObject and TWList objects can be used to represent a repeating and hierarchal data structure.

Web service integration

By comparison the web service Integration is relatively straightforward. Due to the unknown length of time it may take a business partner to perform its portion of the invoice processing, the web services model is asynchronous. The pattern used involves a one way web service request, followed by a wait activity, which is implemented by an intermediate message event. A separate BPM web service is then exposed by BPM to facilitate the response, which correlates with the waiting process through an Undercover Agent (UCA).

Figure 5. Post Invoice process
Post Invoice process

In the Post Invoice process shown in Figure 5, the outgoing web service request (in the Update ERP task) is straightforward, passing an Invoice business object, which includes a correlation identifier. The structure of the incoming web service, which handles the response, is shown in Figure 6.

Figure 6. Anatomy of incoming web service
Anatomy of incoming web service

The web service is essentially a wrapper around a General System Service, which invokes a UCA that uses the correlation information passed in the web service to find the right instance of the PostInvoice process. In Figure 5, this UCA would trigger the "Wait for post notification" event, enabling the Post Invoice process to continue executing.

Image manipulation

Aside from supporting multiple technical integration options, the solution also has to display the digital images received from the Scanning Provider. This allows the user to verify the results of the Optical Character Recognition (OCR) activity and manually amend any invoice lines if necessary. The images are typically provided in JPEG or PDF format, but are not known to the BPM solution at design time and so cannot be an integral part of the deployed Process App.

Instead, the images are encoded at runtime using a Java integration to transform them from a binary format to Base64 ASCII streams, which are them rendered as part of a Coach UI using a custom HTML control. The key elements of this pattern are shown in Figure 7.

Figure 7. Verify Invoice process
Verify Invoice process

The Verify Invoice process is again straightforward; in fact, all processes in the solution are small and maintainable, minimizing the impact of change. The process initially establishes some variables because the location of the images is configured through Exposed Process Variables (EPV), enabling them to be easily changed depending on the deployment environment. The Base64 encoding routine is then invoked via a Java integration, with the API shown in Figure 8.

Figure 8. Encode image definition
Encode image definition

This API accepts the fully-qualified filename of the image and returns a Base64-encoded character stream, which is stored locally in an image parameter shown in Figure 9.

Figure 9. Encode image data mapping
Encode image data mapping

The image variable is not a special structure, but is simply an instance of a Base64Image business object, which is a wrapper containing the image stream and the associated filename as shown in Figure 10.

Figure 10. Base64Image business object
Base64Image business object

The scanned images are fairly small, typically around 30K bytes in size, and from the results of our testing appear to be handled successfully by BPM. Clearly, if larger images were supplied, or if there was a requirement to concurrently process a large volume of invoices, this approach might have to be revisited, potentially in conjunction with an increased server heap size.

Now that we have the image stored within a BPM variable, it can be manipulated just as if it were any other string value. Indeed one of the integrations between BPM and the Workflow Provider requires that the entire image be passed as a parameter to a web service. This can now be achieved using the standard web service integration within BPM.

The encoded image can also now be rendered within a standard BPM Coach. Since the introduction of the new Coach architecture in BPM V8, there have been some changes to the way Custom HTML controls can be used. In versions prior to V8, it was possible to use the syntax shown in Figure 11 in a Custom HTML component.

Figure 11. Custom HTML Control (pre V8)
Custom HTML Control (pre V8>

The BPM expression within the <#= #> tags would be evaluated prior to rendering the control, enabling the data associated with the image variable to be displayed in the HTML <img> tag, which in this case would display the image. However, in BPM V8 this syntax is no longer supported. However, a number of workarounds are possible to achieve the same result. One option is to use Javascript to dynamically generate an entire HTML string within the Human Service, as shown in Listing 4.

Listing 4. Runtime HTML generation
tw.local.imageHTML = "<div id='imageDiv'><img id='invoiceImage' 
	src='data:image/jpg;base64," + tw.local.image.image + "'></div>";

This initializes a local string variable within the VerifyInvoice Human Service to contain a dynamic HTML statement that will render the encoded invoice image inside a <div> element. You can then assign this variable to a Custom HTML element in the Coach, as shown in Figure 12.

Figure 12. Assignment of a variable to a Custom HTML control
Assignment of a variable to a Custom HTML control

The end result is that the scanned image appears nested within the Coach as required, as shown in Figure 13.

Figure 13. Figure 13. Image nested within Coach
Image nested within Coach

Another "gotcha" to be aware of with new Coaches relates to rendering of list variables that are bound to a repeating table within a Horizontal or Vertical Section. This is a fairly common pattern in which a list variable is used to represent rows of information relating to a single parent object.

For example, in the Coach in Figure 13, the Invoice Lines area displays a table of individual line items within a Horizontal Section. If you drag the invoiceLines list variable from the palette and drop it on the canvas within a Section, it will not only bind the variable to the resulting Table, but also to the Section itself. This means that if you have three elements in the invoiceLines list, you would have three entire Sections, each of which has a Table with three rows! In some scenarios this is the desired behavior, but in this case to avoid repeating sections, simply remove the binding between the Section and the list variable, but keep the binding between the Table and the variable. Thanks to Avijit Banerji for spotting that one!


Business monitoring

The final element of the solution involves creating some reports to provide business insight into the status and performance of the process. The capabilities of Business Activity Monitoring (BAM) have been outlined in a number of resources, notably in a series of developerWorks articles by John Alcorn, therefore they will not be covered in detail here.

The solution leverages the out-of-the-box integration between BPM and BAM, where Process Designer is used to identify a set of variables that will be monitored throughout the execution of the process. These "tracked" fields, which are shown in Figure 14, are used to generate and publish an IBM Business Monitor model by selecting the File => Update Tracking Definitions option in Process Designer.

Figure 14. Tracked fields
Tracked fields

As the process executes, the values associated with these variables are sent to Business Monitor. Using this information, a number of predefined and custom reports were created in the Business Space environment, such as the one shown in Figure 15, which shows the cumulative value of invoices processed, broken down by business unit. Note that the names of the actual business units have been removed from the x-axis for reasons of confidentiality.

Figure 15. Invoice Value report
Invoice Value report

Developing the monitor model in Process Designer is very convenient, but it offers only a basic level of integration with Business Monitor. If a more detailed model is required, you would need to use the Monitor Toolkit in Integration Designer. This enables finer-grained control over the automatically generated content and the criteria that can be applied to the dimension of the reports.


Conclusion

This article series provided an end-to-end case study of a real world BPM project, involving multiple cooperating third parties and a variety of technical integration techniques. From the initial analysis through to the generation of business reports, you've seen how IBM BPM Standard V8.0.1 can be placed at the core of a complex business solution, choreographing a range of both automated and human activities. Specifically, this series covered:

  • Business analysis and process mapping in Blueworks Live
  • Development of a process model, involving multiple subprocess instances
  • Accessing external XML-based data structures
  • Java and web service integration
  • Process correlation
  • Image manipulation, including encoding and rendering in Coaches
  • Business activity monitoring

This entire solution was designed, developed and deployed to an externally hosted client environment taking approximately fifteen person weeks of effort, showing how IBM BPM can reduce the development cycle and can accelerate the time to value of complex BPM solutions.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Business process management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Business process management, WebSphere
ArticleID=930331
ArticleTitle=Case study: Invoice processing with IBM Business Process Manager, Part 2: System integration
publish-date=05222013