© 2003 International Business Machines Corporation. All rights reserved.
If you have any questions for Carol related to DB2 Information Integrator, please submit your question by June 30, 2003. We'll try to answer as many questions as possible in a future edition of the DB2 Developer Domain.
DB2DD: Carol, what do you do in Advanced Technical Support?
Carol: The members of the Advanced Technical Support organization provide technical support for the IBM sales organization in all of the Americas, North, Central and South. We deal with the sales force and customers on a daily basis. We're involved with the products we support from top to bottom, from working with development on the requirements process, to working with the sales force in understanding how products meet customers' needs, through the strategic planning and implementation phases for our customers. We get involved with critical customer situations as well, with deep skills in problem determination and resolution. We frequently provide assistance to defect support. We develop class materials, conduct workshops, work directly with customers involved in beta programs, and develop proofs-of-concept when customers are considering IBM's products.
DB2DD: Does this involvement with customers give you a unique view?
Carol: Yes it does. We get not just a broader but a deeper picture than most IBMers. We know what it's like in the trenches! Our point of view is practical rather than theoretical. Talk to us if you want to know how to make it work today.
DB2DD: Carol, I know your specialties are information integration, federation and replication. You've worked with customers for many years on DataJoiner and DataPropagatorTM, and now you're taking those skills to a new level with DB2 Information Integrator. What are customers telling you they're going to be doing with DB2 Information Integrator?
Carol: DB2 Information Integrator can be used to solve many, many customer problems. In the past there's been interest in heterogeneous replication - that is, replication from DB2 to non-DB2 databases. Where I'm getting questions today is on federation - that is, direct access to all types of data through a single interface. Using federation, customer applications can access data stored in many forms with a single interface. Almost every customer I talk to has data stored in at least two relational databases. Quite often, the diversity of data is just amazing. Each type of data has its own access method, its own security requirements, and a data model specific to the application that uses it. Applications that need information from each of these "islands" of data have to code their own bridges. DB2 Information Integrator provides that bridge.
DB2DD: We've mentioned the word federation. What is federation and what problem does it solve?
Carol: Federation is a piece of middleware that provides a way to access all your diverse data sources as though they were in one place. Instead of accessing each data source separately, then trying to bring it together and integrate it yourself or through your application, federation integrates the data at the database level.
What makes this powerful is that it gives you access to all the data in the enterprise. If you have data on multiple platforms, in multiple vendor databases, in XML documents , in flat files, in Excel spreadsheets, in ODBC data sources, you can access it all as if it were in a single database in DB2. What's more, you have all the power of DB2 to operate on that table. You can do distributed joins, create views over the distributed data, use query tools to analyze the distributed data, copy the data to other places. Applications are written using the DB2 interfaces, so programmers only need to know one data access method.
DB2DD: Does this mean that federation has a positive impact on application development?
Carol: Definitely. Our developers have done several exercises/tests to identify the benefits of using federation to shorten the application development cycle. The series of articles by Cindy Saracco published in the DB2 Developer Domain talks about their very positive findings.
DB2DD: OK, you told me you're the one to talk to if I want to make it work today. Can you explain in fairly simple terms how this works?
Carol: First you install DB2 Information Integrator. You have a DB2 relational engine. You have wrappers, which are DB2 II programs that issue the requests to the remote data source, which can be DB2 UDB, Informix, Microsoft SQL Server, Oracle, Sybase, Teradata, a structured flat file, an XML document, an Excel spreadsheet, an ODBC data source, a host of unstructured data from document repositories, email systems or the Web, or even data from Life Sciences applications. You have connectivity functions that also provide access to Web services, WebSphere MQ, and OLE DB sources.
If you want to access relational data sources, you install and configure the client for that data source. For instance, if you want to access Oracle databases, you install and configure Oracle Net8 on your DB2 II server.
For each data source you want to access, you define a server which includes the location and name of that data source, and then define user mappings to map the DB2 II userid to the remote userid. Then you define a nickname. Each nickname is an entry in the DB2 II catalog which points to a table or view at the remote data source. When you create a nickname for a relational data source, information about the remote table or view is extracted from the data source catalog and stored in the DB2 II catalog. This information includes column names and data type mapping, index information and statistics. All this work is done using the DB2 Control Center.
DB2DD: What if the data source is not a relational database?
Carol: The DB2 II nickname points to a structured flat file, an XML document, or some other non-relational object. If there is not a catalog or dictionary associated with the non-relational data source, then you must provide field/column information when you define a nickname.
DB2DD: What happens when I issue the DB2 SQL that references a nickname?
Carol: The DB2 engine optimizes the statement based on the information in the catalog and information it receives from the wrapper. This information includes statistics, indexes and data type mapping for the remote table. All or part of the statement is sent to the remote data source. We call that "pushdown". If the DB2 SQL includes a DB2 function that is not available at the remote data source, then DB2 II will retrieve the data and apply the function locally. We call that "compensation". If the DB2 SQL includes a comparison or requires a sort and the collating sequence of the data source is not the same as the DB2 II collating sequence, then DB2 II will retrieve the data and sort it locally. There are server options that can be set to affect the way that DB2 II processes SQL for the different types of data sources. Basically DB2 II attempts to decompose the query to minimize how much data flows across the network while insuring an accurate response.
DB2DD: What happens to the results?
Carol: The results from the DB2 SQL statement are sent back to the user in standard DB2 format,including the option of providing results as an XML document.
DB2DD: I know you've been participating in the beta program for DB2 Information Integrator. What kinds of customers are you seeing that are using these federation capabilities?
Carol: Quite a few customers in different industries are using federation. Some of the applications of federation I've seen are:
- Enhancing a business intelligence platform by federating access across a data warehouse and real-time production images
- Presenting a unified view to reporting tools across departmental or division boundaries
- Building a consolidated view from diverse data sources in order to do analysis of past business history and activity
- Managing inventory in real time when the inventory data is found in different places to lower costs
- Improving Customer Relationship Management by providing a customer view of all the customer assets and transactions, regardless of where they are stored
- Supporting coexistence of legacy databases with new databases during a migration period.
Our beta customers have come from many different industries including financial services, manufacturing, insurance, retail, life sciences health care, telecommunications, education, transportation, government.
DB2DD: Carol, I know you have a lot of experience with replication as well as federation. Why would a customer choose replication instead of federation?
Carol: Federation gives you real-time access and existing systems/applications do not need to be changed. This is definitely the best solution for applications that need current data.
However, remote users on slow networks may not be able to access data directly in a timely fashion, regardless of the tool used. It can be more efficient for those users to access data stored locally. Replication can keep the local data up-to-date. You can distribute parts of your enterprise data to multiple locations to provide local access for improved performance and availability. You can consolidate data from multiple locations to a central operational data store or data warehouse. But you should always keep in mind that replication is an asynchronus process, so there is latency. At any given point in time, there may be outstanding changes which have occurred at the source server, but have not yet been copied. The copied data is always consistent, but it may not be current.
DB2DD: How does replication work with DB2 Information Integrator?
Carol: Replication is really two processes, one process that captures changes to the database, and one that applies those changes to the target. If you're copying from a DB2 database, then the capture process is log based. If you want to capture changes from a non-DB2 relational database then the capture process is implemented as a set of triggers. Since DB2 II makes relational data sources look like DB2, the replication Apply program can apply changes to nicknames just as if they were DB2 tables. The Replication Center is used to define replication amongst DB2 and non-DB2 databases.
Actually, replication is a good example of a DB2 application that takes advantage of federation to extend its reach beyond DB2 data. You can copy from DB2, Oracle, Informix, Sybase, or Microsoft SQL Server tables. You can copy to DB2, Oracle, Informix, Sybase, Microsoft SQL Server, or Teradata tables.
DB2DD: You've mentioned that your contact with customers gives you a unique viewpoint. So how do you think this new federation capability is going to transform business?
Carol: When we say this is going to give you the ability to transparently access all the information in the enterprise, it's an exciting idea. We've never been able to do this before on such a broad scale. You can analyze and manage your data in a way you've never done before. Many times, application design and architecture have been restricted by availability of data in the proper place or format, without regard to the business needs. DB2 Information Integrator removes that restriction. IT organizations are looking for ways to reduce costs. Managing redundant data is viewed as a necessary evil to integrate information because it introduces significant hardware and administration costs. Federation gives them the ability to integrate data without the added costs for classes of applications and customers are very excited about that.
DB2DD: Thank you very much, Carol. We appreciate your insights.
Carol: You're welcome.
All statements regarding IBM's future direction or intent are subject to change without notice, and represent goals and objectives only.
- DB2 Information Integrator Web site: http://www.ibm.com/software/data/integration/iipreview.html
- IBM Systems Journal on Information Integration, at http://www.research.ibm.com/journal/sj41-4.html
- Haas, Laura and Eileen Lin. "IBM Federated Database Technology," at http://www-106.ibm.com/developerworks/db2/library/techarticle/0203haas/0203haas.html
- Lurie, Martin. "The Federation - Database Interoperability" at
http://www-106.ibm.com/developerworks/db2/library/techarticle/0304lurie/0304lurie.html
- Saracco, C. M. "Coping with Disparate Data in Web Applications," at http://www-106.ibm.com/developerworks/db2/library/techarticle/0208saracco/0208saracco.html
- Saracco, C. M. "Building Web Components that Access Federated Data," at http://www-106.ibm.com/developerworks/db2/library/techarticle/0209saracco/0209saracco.html
- Saracco, C. M. "Building Entity EJBs that Span Federated Data," at http://www-106.ibm.com/developerworks/db2/library/techarticle/0209saracco/0209saracco1.html
- Saracco, C. M., and T.J. Rieger. "Our Experience with Developing Entity EJBs over Disparate Data Sources," at http://www-106.ibm.com/developerworks/db2/library/techarticle/0305saracco/0305saracco.html
- Saracco, C. M., Susanne Englert and Ingmar Gebert. "J2EE Development Across Multiple Data Sources: Digging into the Details," at http://www-106.ibm.com/developerworks/db2/library/techarticle/0306saracco/0306saracco.html
Carol Rigdon is a Consulting IT Specialist with IBM Advanced Technical Support. She has supported cross-platform, cross-vendor database products in various IBM organizations since 1994 and currently supports DB2 ConnectTM, DB2 DataJoiner, DB2 replication, and DB2 Information Integrator. Carol is well-known expert on the topic of federated data.
Comments (Undergoing maintenance)
Table of contents
Local resources
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
