Skip to main content

skip to main content

developerWorks  >  Information Management | WebSphere  >

Building Portals with Enterprise Information Integration Technology

A Design and Coding Comparison

developerWorks
Document options

Document options requiring JavaScript are not displayed


Rate this page

Help us improve this content


Level: Introductory

C. M. Saracco, Senior Software Engineer, IBM
Jacques Labrie (jlabrie@us.ibm.com), Senior Software Engineer, IBM
Julien Muller, Software Engineering Intern, IBM

19 Mar 2004

This article will explain why using Enterprise Information Integration Technology to develop portal application components that integrate data from disparate sources can simplify design issues and cut coding requirements by 50 percent or more. It will also review a project in which we built functionally equivalent portal components with and without EII.

It's a design issue that portal application developers often face: what's the best way to build software components that access and integrate data from a variety of disparate sources?

The most obvious approach -- write the data integration code yourself -- may cost you more time and effort than you think. Indeed, for one of our projects, we found our code more than doubled when we managed data integration work ourselves than when we delegated this work to an Enterprise Information Integration (EII) software layer. We'll take you through some of our work in this article so you can understand the trade-offs involved with each approach.

In some respects, this article is a follow-on to "Extending Portals with DB2 Information Integrator". That article explored how developers can extend the reach and capabilities of ready-made portal components (such as IBM's My Query Reports and JDBC Business Object Builder) with EII. While that article focused on building data integration portals without writing Java code, this article focuses on undertaking similar work when coding your own portal components. To get started, we'll tell you a bit more about our sample development project and its underlying environment.

A quick look at our project

To help us investigate how EII technology might benefit portal developers, we built a simple portal application designed to simulate support for a customer service center of an insurance agency. Our portal enables call center representatives to field questions from customers, such as determining the status of a claim or reviewing policy information.

Like most portal developers, we wanted our users to have a single point of access to all the information they needed to do their jobs. In our case, this meant providing access to a DB2 UDB database, a Lotus Domino database, and XML files. DB2 managed customer contact and policy information. Domino stored claims information, including the nature of the claim, date of the claim, and a determination of who was at fault. An XML file contained police reports of incidents, which could be traced to a claim through an incident number. Fig. 1 shows a simplified view of this data schema.


Figure 1. Customer, claims, and incident data
Customer, claims, and incident data

We explored design and development options for supporting a straightforward request: view all the claims and incident reports associated with a given customer. Technically, this request was quite simple. For starters, it involved a single equality search predicate (a specific customer ID). Furthermore, it lacked aggregations, date/time expressions, or more complex data filtering operations. Finally, it didn't involve integration of sets of data, which could have required sorting or more complex join logic. Indeed, the simplicity of this request was due, in part, to arbitrary restrictions we placed on our data schema, such as a 1:1 relationship between claims and incident reports.

You might wonder why we chose such a simple case. Previously, we'd explored the design and development trade-offs of implementing more complex queries over disparate data using servlets and entity Enterprise JavaBeans (EJBs). These projects, described in "Using DB2 Information Integrator for J2EE Development: A Cost/Benefit Analysis" and "Our Experience with Developing Entity EJBs over Disparate Data Sources", used a wider variety of data objects, all of which were accessible through a JDBC interface. For this project, we wanted to understand the impact of integrating data -- even in a simple manner -- using the native application programming interfaces (APIs) of each source.

In addition, we elected to hand-code our own custom portlet, or portal application component, to support our work. One implementation used DB2 Information Integrator (DB2 II) to provide EII services and manage our data access work. Another accessed each data source directly to provide equivalent function. In the end, our DB2 II-based scenario required about 180 lines of code, while our hand-coded version required about 380. You'll begin to see why when we discuss the connection management, data retrieval, and data consolidation issues we confronted later in this article.

However, if you're not already familiar with EII technology, federated data services, or DB2 II, we recommend you read "Extending Portals with DB2 Information Integrator" first. It will give you a quick overview of DB2 II, as well as explain how federated data management technology can be useful in a portal environment.



Back to top


Software configurations

To support our work, we installed several software products:

  • WebSphere Portal Version 5.0
  • WebSphere Application Server Version 5.0.1
  • WebSphere Studio Application Developer Version 5.0.2 (We used this for development and test only.)
  • Portal Toolkit Version 5 for WebSphere Studio. (We used this for development and test only.)
  • DB2 Information Integrator Version 8.1 (We used this for our EII-based portlets only and separately tested against fixpacks 3 and 5.)
  • DB2 Universal Database Version 8.1 (We tested against fixpacks 3 and 5 in different scenarios.)
  • Lotus Domino 5.0.10
  • XML parser capable of supporting XML 1.0 versioned documents

This left us with two runtime configurations, both shown in Fig. 2.


Figure 2. Runtime architectures
Runtime architectures


Back to top


Portlet architecture

Like all portlets designed to run in WebSphere Portal, our portlet was implemented as a subclass of a Java servlet. Thus, we adopted many of the "best practices" guidelines for servlet development as part of our work. This included implementing a Model-View-Controller (MVC) architecture in which JavaBeans represented our data model, Java Server Pages (JSPs) provided the view or presentation layer, and our portlet functioned as the controller object. If you're not familiar with the MVC design pattern, a number of books and Web sites provide tutorial information about it. (One such source is Chapter 4 of the IBM Redbook Legacy Modernization with WebSphere Studio Enterprise Developer.)

With this design, our JavaBeans contained the logic for accessing each required data source and retrieving the appropriate data. When we used DB2 II, we only wrote one such bean. Without DB2 II, we wrote one bean per data source (one for DB2 UDB, one for Domino, and one for the XML document).

Our portlet code called methods in the JavaBeans as appropriate to execute the data retrieval work and ensure a consolidated result set was available for use by a JSP, which displayed the data for the user. With DB2 II, this involved a few lines of code in the portlet's doView method to establish a single data source connection and execute a single query. Without DB2 II, it involved a number of lines of code to

  • Build (and later populate) a hash table to hold the final results.
  • Invoke appropriate method(s) for each JavaBean that wrapped a data source. These methods connect to or open the necessary data source object and retrieve the required data.
  • Invoke a JavaBean to integrate the result sets.

We'll step through key aspects of our portlet logic in the subsequent sections.



Back to top


Connection management and data access initialization issues

Retrieving data from any given source usually requires some initialization activities, such as connecting to a data source and providing valid authentication information. For example, multi-user relational database management systems (DBMSs) require programmers to connect to a database with a valid user ID and password. Domino database access requires programmers to initialize a Notes session and connect to a database. And before searching for data within a XML file, programmers must first open and parse the file.

Because DB2 II presents a virtual database image of remote data sources, we didn't need to write code for each of these activities. Instead, we connected to our DB2 II database using a DataSource object (a pooled database connection defined in WebSphere) and we were ready to go. We used a Java 2 Connector (J2C) alias associated with our DB2 II DataSource object to encapsulate the user ID/password information. Here's our code:


Listing 1. Connection management with DB2 II
// This is in the portlet's doView() method
. . . 
// If we haven't already looked up data source, do it now
if(ds==null){
	
 //Retrieve a datasource through the JNDI name service
 java.util.Properties parms = new java.util.Properties();
 parms.setProperty(javax.naming.Context.INITIAL_CONTEXT_FACTORY,
   "com.ibm.websphere.naming.WsnInitialContextFactory");

  try{
    //Create the initial name context
    javax.naming.Context ctx = new javax.naming.InitialContext(parms);

    //Look up the DataSource object
    ds = (javax.sql.DataSource) ctx.lookup(dsName);
    ctx.close();
    . . .

    // 	get a connection to the DataSource
    con = ds.getConnection();

  } catch(Exception e){
    . . .
  }
. . . 
}

. . . 
if(!((Query1)dataBean).connect()){
  System.out.println("Not able to connect to the database");
  log.error("cotton:JDBCPortlet:Could not connect to the database" + 
    "Please check user, password and url");
}
. . . 


// This is in an abstract class used by our DB2 II JavaBean
/ *
* connects to the datasource:
* 1. directly over jdbc
* 2. uses a connection instance from the connection pool
* @return boolean
*/	

. . . 
public boolean connect(){
  if(isDS == false){
    try { 
      Class.forName(driver).newInstance();
      con = DriverManager.getConnection(url,user,pass);
    }catch(Exception e){System.out.println("direct JDBC connection error: " + e);
      return false;
    }
    return true;
    }
    else{
      try{
        con = ds.getConnection();
        return true;
      }catch(Exception e){
        System.out.println("datasource connection error: " + "check jndi setup");
        return false;
      }
  }
}	


When we accessed each data source directly, we naturally had more code to write. This included connecting to the DB2 UDB database using a DataSource object, connecting to the Domino database (without a DataSource object), and parsing the XML document of interest. Of course, we had to use different Java classes and methods to support each activity, which required more programming skill than our earlier approach. And we were also more aware of the distributed nature of our data.

Finally, it's worth noting that if we had needed to connect to multiple data sources with different authentication requirements, we would have needed to acquire (and specify) valid user IDs and passwords for each of these data sources, possibly hardcoding multiple security objects in our application to do so. We were able to avoid specifying a user ID and password for the Notes database in our test scenario because we built it as a local, unprotected database. However, this is not typical of a production environment.

Here's the code we used to connect to the DB2 UDB database directly:


Listing 2a. DB2 UDB connection
// This is in the portlet's doView() method
. . . 
// If we haven't already looked up data source, do it now
if(ds==null){
	
 //Retrieve a datasource through the JNDI name service
 java.util.Properties parms = new java.util.Properties();
 parms.setProperty(javax.naming.Context.INITIAL_CONTEXT_FACTORY,
   "com.ibm.websphere.naming.WsnInitialContextFactory");

  try{
    //Create the initial name context
    javax.naming.Context ctx = new javax.naming.InitialContext(parms);

    //Look up the DataSource object
    ds = (javax.sql.DataSource) ctx.lookup(dsName);
    ctx.close();
    . . .

    // 	get a connection to the DataSource
    con = ds.getConnection();

  } catch(Exception e){
    . . .
  }
. . . 
}
. . . 
if(!db2.connect()){
  System.out.println("Not able to connect to the datasource: DB2");
  log.error("cotton:JDBCPortlet:Could not connect to the database" + 
    "Please check user, password and url");
}



// This is in an abstract class used by our DB2 UDB JavaBean
/ *
* connects to the datasource:
* 1. directly over jdbc
* 2. uses a connection instance from the connection pool
* @return boolean
*/	

. . . 
public boolean connect(){
  if(isDS == false){
    try { 
      Class.forName(driver).newInstance();
      con = DriverManager.getConnection(url,user,pass);
    }catch(Exception e){System.out.println("direct JDBC connection error: " + e);
      return false;
    }
    return true;
    }
    else{
      try{
        con = ds.getConnection();
        return true;
      }catch(Exception e){
        System.out.println("datasource connection error: " + "check jndi setup");
        return false;
      }
  }
}	


Here's the code we used to connect to the Domino server and request access to the Lotus Notes database:


Listing 2b. Domino connection
// This is in the portlet's doView() method
/* The call to the Notes constructor specifies the database of interest
*  Actual connection to this database will be done as part of the 
*  access() method in the JavaBean
*/
. . .
// getRealPath() provides the fully qualified name of Notes file.
// In our test scenario, this file was stored within our portal project  
// because we wanted to ship the application and most of its data as 
// a single unit.  In a production environment, access to the 
// Notes database would probably need to be specified in a different manner. 
NotesAccess notes = new NotesAccess
  (getPortletConfig().getContext().getRealPath("Data/Claims2.nsf"));
notes.access(CUSTOMER_ID,incidents);
. . . 
				

// This is part of the access() method in the Domino Notes JavaBean
. . . 
try{
  // Initialize a notes session
  NotesThread.sinitThread();
  }

catch (Exception e){   // cannot create Domino thread
  . . .
}

try{
  Session s = NotesFactory.createSession();

  // Connect to the database
  Database claims = s.getDatabase(null,notesurl);

  // Test if we could open the database
  if (!claims.isOpen()){
    System.out.println("Not able to connect to the server notes database");
    . . .
  }
  . . .

catch (Exception e) {
  . . .
}

Since the XML document is a locally referenced file, there is no need for any connection code; however, the portlet must specify the XML file of interest, as well as invoke methods on the JavaBean to prepare for a subsequent search of the content within an XML file:


Listing 2c. XML file preparation
// This is in the portlet's doView() method  
/* The call to the XML JavaBean constructor specifies the file of interest
*  The JavaBean's access() method includes code to prepare for searching the file
*/
. . . 

// getRealPath() provides the fully qualified name of the XML file.
// In our test scenario, this file was stored within our portal project
// because we wanted to ship the application and most of its data as 
// a single unit.     
XMLAccess xml = new XMLAccess
  (getPortletConfig().getContext().getRealPath("Data/PoliceReports.xml"));
xml.access(incidents);
. . . 


// This is in the XML JavaBean's access() method
// Shown below is only the code to prepare for search of the file
. . . 

// Create a DOM parser
DOMParser parser= new DOMParser();

try{
  // Parse the file and get it as a tree
  parser.parse(fileURL);
  Document doc = parser.getDocument();

  // Establish elements of the tree
  Node reports = doc.getDocumentElement();
  NodeList reportList = reports.getChildNodes();
  NodeList reportElemList;
  . . .
}

catch (Exception e) {
  . . .
}



Back to top


Data retrieval issues

With preparatory work completed (primarily connection management), we then had to retrieve our data. Recall that our objective was quite simple -- for a given customer, find all his/her claims and any police reports associated with them. Of course, we had to express this business request in appropriate data access code.

Writing this code with DB2 II required little work. We simply issued one SQL statement and processed the results. DB2 II took care of decomposing this query into pieces that could be processed by each of the target data sources. It also determined and executed an appropriate data access strategy on our behalf in a manner that was transparent to us.

We wrote only a few lines of code in the portlet's doView method to call a method to execute the query:


Listing 3. Portlet data access code with DB2 II
//try to execute the SQL query
if(!((Query1)dataBean).executeQuery()){	
  System.out.println("Not able to execute the query");
  log.error("cotton:SQLPortlet: Could not execute the sql query");
}


The code in the executeQuery method verifies that a database connection exists and executes our query. Here's the data access code:


Listing 4. JavaBeans and related code for data access with DB2 II

// This code is in our DB2 II JavaBean
. . .
public boolean executeQuery( . . .) {
. . .
if(!cust_id.equals(""))
// ***********************************************************************************************
// Execute the following query to get all reports, claims and customer info based on policy#
// ***********************************************************************************************
  return super.executeQuery("
    SELECT
       DEMO.CUSTOMER_DB2.CUSTOMER_ID, DEMO.CUSTOMER_DB2.FIRST_NAME || ' ' ||
       DEMO.CUSTOMER_DB2.LAST_NAME, DEMO.CUSTOMER_DB2.ADDRESS, DEMO.CUSTOMER_DB2.CITY,
       DEMO.CUSTOMER_DB2.STATE, DEMO.CUSTOMER_DB2.POLICY_TYPE, DEMO.CUSTOMER_DB2.POLICY_PREMIUM,
       DEMO.CLAIMS_DOMINO.ODATE, DEMO.CLAIMS_DOMINO.CDATE, DEMO.CLAIMS_DOMINO.INCIDENT_ID,
       DEMO.CLAIMS_DOMINO.DOC_ID,
       DEMO.REPORTS_XML.OFFICERNAME, DEMO.REPORTS_XML.INCIDENTDESC
    FROM
       DEMO.CLAIMS_DOMINO, DEMO.REPORTS_XML,DEMO.CUSTOMER_DB2
    WHERE
       DEMO.CLAIMS_DOMINO.INCIDENT_ID = DEMO.REPORTS_XML.INCIDENTID AND
       DEMO.CUSTOMER_DB2.CUSTOMER_ID = INTEGER(DEMO.CLAIMS_DOMINO.CUSTOMER_ID) AND
       (DEMO.CLAIMS_DOMINO.INCIDENT_TYPE = 'Fault' OR DEMO.CLAIMS_DOMINO.INCIDENT_TYPE = 'No Fault') AND
       DEMO.CUSTOMER_DB2.CUSTOMER_ID=" + cust_id
    );

else return false;
. . .

public String getCustID(){
  try{
    return rs_c.getString(1);
  }catch(SQLException e){ . . . }
}

public String getName(){
  try{
    return rs_c.getString(2);
  }catch(SQLException e){. . . }
}

. . .

}

// This code is included in another class (an abstract class used by our JavaBean).
// After a SQL query string is defined, this method is called to execute it.
protected boolean executeQuery(String sql){

  try{
    stmt_c = con.prepareStatement(sql,ResultSet.TYPE_SCROLL_SENSITIVE,ResultSet.CONCUR_READ_ONLY);
    rs_c = stmt_c.executeQuery();
    }

  catch(SQLException e){
       . . .
  }
       . . .
}


Without DB2 II, we needed to retrieve data from each source individually using native APIs. We also needed to determine how to break up our one logical query into individual data retrieval activities for each data source. And finally, we needed to determine our overall data access strategy -- which data to retrieve first, how we would filter data at each source, etc.

The latter two items turned out to be relatively straightforward since our query was so simple. However, past experience has shown that this isn't always the case. Indeed, when we did our comparative servlet work, we found that decomposing more complex queries and developing an efficient data access strategy for them took considerable time, effort, and expertise. Indeed, preserving our original query semantics while providing for reasonable performance often required iterative design, coding, and testing. As we mentioned earlier in this paper, we deliberately avoided revisiting this issue for our portlet work and therefore confined our focus to a very simple business task.

Our portlet's doView method invokes methods from multiple JavaBeans to initiate data retrieval activities at each data source. First, it retrieves appropriate customer information from the DB2 UDB database. Next, it retrieves claims information for the target customer from Domino. Finally, it retrieves incident reports (police reports) for the appropriate claims from an XML file. All this data is then combined and prepared for final presentation by a JSP.


Listing 5. Portlet data access code without DB2 II

//Execute the query without ii:
// Create an Hashtable to store the results
Hashtable incidents = new Hashtable();

// Connect to DB2 UDB, execute the query and get the result
// using the DB2 access class		
. . . 
			
db2.setCust_id(((Query1)dataBean).getCust_id());
//try to execute the queries against each data source
if(!db2.executeQuery()){	
  System.out.println("Not able to execute the query");
  log.error("cotton:SQLPortlet: Could not execute the sql query");
}
db2.getNextResult();
String CUSTOMER_ID = db2.getCustID();
String CUSTOMER_NAME = db2.getName();
String ADDRESS = db2.getAddr(); 
String CITY = db2.getCity();
String STATE = db2.getState();
String POLICY_TYPE = db2.getType();
String POLICY_PREMIUM = db2.getCost();
			
// Get the results from Notes and store all the matching
// incidents from notes into the hashtable 
NotesAccess notes = new NotesAccess
  (getPortletConfig().getContext().getRealPath("Data/Claims2.nsf"));
notes.access(CUSTOMER_ID,incidents);

// Include the XML data in the hashtable
XMLAccess xml = new XMLAccess
  (getPortletConfig().getContext().getRealPath("Data/PoliceReports.xml"));
xml.access(incidents);

// Put the result in a bean to pass it to the jsp page ...
JoinBean join = new JoinBean
  (incidents.values(),CUSTOMER_ID,CUSTOMER_NAME,ADDRESS,CITY,STATE,POLICY_TYPE,POLICY_PREMIUM);		s.setAttribute(dataBeanName,join);

. . . 

The code in the portlet's doView method relies on three JavaBean classes to handle the actual data retrievals. Excerpts from the DB2 UDB JavaBean are included here:


Listing 6a. JavaBean for DB2 UDB access
// This code is in the DB2 UDB JavaBean
. . .
private String sql = "
  SELECT DEMO.CUSTOMER_DB2.CUSTOMER_ID,DEMO.CUSTOMER_DB2.FIRST_NAME || 
    ' ' || DEMO.CUSTOMER_DB2.LAST_NAME, DEMO.CUSTOMER_DB2.ADDRESS, 
    DEMO.CUSTOMER_DB2.CITY, DEMO.CUSTOMER_DB2.STATE, DEMO.CUSTOMER_DB2.POLICY_TYPE, 
    DEMO.CUSTOMER_DB2.POLICY_PREMIUM 
  FROM 
    DEMO.CUSTOMER_DB2 
  WHERE DEMO.CUSTOMER_DB2.CUSTOMER_ID =";
. . . 

/**
 * executes the query
 * @return boolean
 */	
public boolean executeQuery(){
  return super.executeQuery(sql + cust_id);
}		
	

public String getCustID(){
  try{
    return rs_c.getString(1);
  }catch(SQLException e){return "";}
}

public String getName(){
  try{
    return rs_c.getString(2);
  }catch(SQLException e){return "";}
}
. . . 

// This code is included in another class (an abstract class used by our JavaBean).
// After a SQL query string is defined, this method is called to execute it.
protected boolean executeQuery(String sql){

  try{
    stmt_c = con.prepareStatement(sql,ResultSet.TYPE_SCROLL_SENSITIVE,ResultSet.CONCUR_READ_ONLY);
    rs_c = stmt_c.executeQuery();
    }

  catch(SQLException e){
       . . .
  }
       . . .
}



After retrieving the data associated with a given customer from DB2, we use the customer's ID to search our Domino database for claims filed by the customer. Claims may reference police reports, and we also retrieved the identifiers for these reports (incident IDs) from Domino. We employ a helper class (Incident) to assist us with processing incident data. In the interest of brevity, we've omitted the code for that class.

A key feature of DB2 II is the ability to hide the actual storage of the data from the application developer. As such, the application developer does not need to know where the data is stored. However, without DB2 II, the data's location is not transparent to the developer, as our code demonstrates.


Listing 6b. JavaBean for Domino access

// establish a connection to the database 
. . .
Item it;
DateTime dt=null;
// Get a view (List of the documents)
View v = claims.getView("By Policy");
			
// Take all the documents one by one to check the  Customer_ID
Document doc = v.getFirstDocument();
while(doc!=null){
  if(doc.isValid()){   // if:  level 1
    
    // Check if the doc has a matching cust id, 
    // get the values needed if it is the case
    if(doc.hasItem("Customer_ID")){     // if:  level 2
      
      if(doc.getFirstItem("Customer_ID").getValueString().equals(cust_id)){  // if:  level 3
        Incident incident = new Incident(doc.getFirstItem("Incident_ID")getValueString());
							
        if(doc.hasItem("ODate")){
          it = doc.getFirstItem("ODate");			
          if(it!=null) { dt = it.getDateTimeValue();  }
          if(dt instanceof DateTime) { incident.setODate(dt.toString()); }
        }
        
        if(doc.hasItem("CDate")){
          it=doc.getFirstItem("CDate");
          if(it!=null){
             dt = it.getDateTimeValue();
             if(dt instanceof DateTime) incident.setCDate(dt.toString());
          }
        }
        
        incident.setDoc_id("");
        incidents.put(incident.getIncident_id(),incident);
    
      }  // end if level 3 
    }  // end if level 2
					
  }   // end if level 1
doc = v.getNextDocument(doc);
}  // end while loop
. . . 

        

Processing the Domino data gives us the incident ID of interest. With this information, we then access XML-based police reports to retrieve the detailed information we need (such as the name of the officer filing the report as well as a description of the incident.)

As mentioned earlier, a key feature of DB2 II is the ability to hide the actual storage of the data from the application developer. The name of the XML file is specified when you configure DB2 II to access XML. As such, the application developer does not need to know where the data is stored. This is the same with the access to the Lotus Domino database; the physical location of the database is not needed by the application developer since DB2 II manages that information. However, without DB2 II, the data's location is not transparent to the developer, as our code demonstrates.

The following excerpt from our XML access bean shows the logic for searching our file:


Listing 6c. JavaBean for XML access

. . .
// Walk the DOM tree to search for data of interest
for (int i=0;i<reportList.getLength();i++){

   Node report = reportList.item(i);
   reportElemList = report.getChildNodes();

  // Go through each report from the tree
  for (int j=0;j<reportElemList.getLength();j++){
     Node elem = reportElemList.item(j);

     // If we have an incidentID TAG
     if(elem.getNodeName().equals("IncidentID")){

        // Check the hashtable to see if we have to add it
        if(ht.containsKey(elem.getFirstChild().getNodeValue())){
           Incident inc = (Incident) ht.get(elem.getFirstChild().getNodeValue());

           // Copy the data
           for (int k=0;k<reportElemList.getLength();k++){

             Node elem2 = reportElemList.item(k);
             if(elem2.getNodeName().equals("OfficerName"))
                inc.setOfficername(elem2.getFirstChild().getNodeValue());
             if(elem2.getNodeName().equals("IncidentDesc"))								                
             inc.setIncidentdesc(elem2.getFirstChild().getNodeValue());

            }  // end for loop

        }   // end inner if
    }  // end outer if

  }  // end inner for loop
}  // end outer for loop




Back to top


Data consolidation issues

The final step in our data-related work involves providing a consolidated set of information for our presentation layer to display. With DB2 II, we didn't need to do anything else. Our single SQL query caused DB2 II to retrieve and return the consolidated result set to us.

Without DB2 II, we had a bit of work to do. Again, this work was quite modest, given the nature of our original query. We didn't have to implement complex join processing logic, perform additional sorting, eliminate duplicate data, or perform aggregations. Such activities would have increased the complexity of our work considerably. Instead, we wrote one helper class that simply took data values retrieved from different sources and placed them into a single, consolidated structure. We also provided accessor methods for attributes of this class, but we've omitted these methods from this article for brevity.


Listing 7. Helper class for consolidating our disparate data

. . .

**
 * @author IBM
 *
 * This bean contains the data stored as the same format
 * as Query1.class. This bean is used in the native case.
 * It gives the capacity to send this bean instead of
 * the other one to the jsp page
 */
public class JoinBean implements ShowCustBean{
	String CUSTOMER_ID="";
	String CUSTOMER_NAME="";
	String ADDRESS=""; 
	String CITY="";
	String STATE="";
	String POLICY_TYPE="";
	String POLICY_PREMIUM="";
	Iterator i;
	Incident inc = null;
	Collection c = null;

}

/* Method JoinBean.constructor to instantiate a bean instance, 
* in which all necessary customer information is stored.
* @param c: hashtable, which contains Incident instances. 
* For each incident of the customer, one incident instance is stored in the hashtable
*/
public JoinBean(Collection c,String CUSTOMER_ID,
  String CUSTOMER_NAME,String ADDRESS,
  String CITY,String STATE,String POLICY_TYPE,
  String POLICY_PREMIUM){
    this.CUSTOMER_ID = CUSTOMER_ID;
    this.CUSTOMER_NAME = CUSTOMER_NAME;
    this.ADDRESS = ADDRESS; 
    this.CITY = CITY;
    this.STATE = STATE;
    this.POLICY_TYPE = POLICY_TYPE;
    this.POLICY_PREMIUM = POLICY_PREMIUM;
    this.c = c;
    this.i = c.iterator();	
}
. . . 




Back to top


Summary

Developing a custom portlet that needs to integrate disparate data and present a single, consolidated result to users can be a challenging task. In this article, we compared and contrasted two approaches to this work and found that using EII technology in DB2 II significantly reduced our efforts. It reduced the amount of code we had to write by more than 50 percent and minimized the skills required to complete our work. And it did this even though our data integration work was extremely simple. If we needed to access more than our three data sources, or needed to implement complex queries involving joins, aggregations, and other sophisticated functions, the use of DB2 II would have yielded even greater development savings.

It's also worth noting that we compared and contrasted coding requirements for implementing one query only. In previous articles, we explored coding requirements involving multiple queries over disparate sources and found that an EII-based architecture enabled us to reuse much of our existing coding infrastructure as we expanded our application components to include additional queries. This is because DB2 II enabled us to code to a common API, to process a common data structure (a consolidated result set), and to avoid the need to hand-code logic to handle joins, aggregations, and other functions. Without DB2 II, different workloads required more extensive changes to the infrastructure of our code. If these workloads had to address varying data sources, the challenge would have been exacerbated by the introduction of yet more APIs, more data structures, more data type translations, and more semantic differences requiring our attention.



Resources



About the authors

Cynthia M. Saracco is a senior software engineer at IBM's Silicon Valley Laboratory. She has written two books on database management (one co-authored with Charles J. Bontempo) and taught courses on Java development and database management issues.


author photo

Jacques Labrie is a senior software engineer at the IBM Silicon Valley Lab in San Jose, CA. Jacques has been a manager, team lead, and developer on multiple IBM warehousing and metadata products since 1984. Currently, Jacques is responsible for prototype development of integration scenarios with the IBM WebSphere family of products.


Julien Muller is a software engineering intern at IBM's Silicon Valley Laboratory. He recently graduated from the University of Paris Dauphine in France with a Master's degree in computer science, specializing in Databases.




Rate this page


Please take a moment to complete this form to help us better serve you.



 


 


Not
useful
Extremely
useful
 


Share this....

digg Digg this story del.icio.us del.icio.us Slashdot Slashdot it!



Back to top