Skip to main content

By clicking Submit, you agree to the developerWorks terms of use.

The first time you sign into developerWorks, a profile is created for you. Select information in your profile (name, country/region, and company) is displayed to the public and will accompany any content you post. You may update your IBM account at any time.

All information submitted is secure.

  • Close [x]

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerworks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

By clicking Submit, you agree to the developerWorks terms of use.

All information submitted is secure.

  • Close [x]

developerWorks Community:

  • Close [x]

High-performance solution to feeding a data warehouse with real-time data, Part 2: Explore the integration options with staging tables and WebSphere MQ messages

Integrating InfoSphere Replication Server and InfoSphere DataStage to trickle feed the data warehouse

Anand Krishniyer (akrishni@us.ibm.com), Staff Software Engineer, InfoSphere Replication, IBM
Anand Krishniyer photo
Anand Krishniyer is a staff engineer with the InfoSphere Replication development organization. As a member of the admin team, his responsibilities include developing line items as well as providing technical assistance to customers with respect to installation, setup, and configuration of the Replication Server. Prior to his current role, Anand worked as a project lead for the Integration and Tools team for a process management company, Savvion (now part of Progress Software).
Tony Lee (tonylee@us.ibm.com ), Senior Certified IT Specialist, IBM
Tony Lee photo
Tony Lee is a certified senior IT specialist in the InfoSphere Replication Center of Competency (COC), a team within the InfoSphere Replication development organization. As a member of the COC, Tony has provided technical assistance to customers and business partners on InfoSphere replication technologies. Tony has many years of consulting experience with various IBM replication technologies, covering SQL replication, Q replication, and, most recently, InfoSphere Change Data Capture. Prior to his current role, Tony provided Information Management technical consultation to customers and partners for nearly a decade, covering a wide range of topics, from DB2 tuning, to Information Server and Master Data Management. Prior to becoming a consultant, Tony worked in many different roles in the Information Management area, ranging from management to development.
James Yau (jamesyau@us.ibm.com), Technical Solution Architect, InfoSphere Information Server, IBM
James Yau photo
James Yau is a senior solution architect certified on the InfoSphere Information Server DataStage product. Currently, he is part of the InfoSphere Technology Enablement organization responsible for Information Server Boot Camp content development and delivery. James has many years of consulting experience with the Information Server Suite of products, including InfoSphere DataStage, QualityStage, Information Analyzer, and FastTrack. Prior to his current role, James was part of a Business Partner Technical Enablement team, in which he was the technical program manager for InfoSphere Information Server. His role included course content development and delivery with various delivery vehicles, such as instructor-led, on-line, and self-paced learning. In the past, James worked in many different roles, ranging from software developer to marketing manager, both in IBM and outside of IBM.

Summary:  Feeding a data warehouse with changes from the source database can be very expensive. If the extraction is only done with SQL, there is no way to easily identify the rows that have been changed. IBM InfoSphere™ Replication Server can detect changed data by reading only the database log. This series shows how to use InfoSphere Replication Server to efficiently extract only the changed data and how to pass the changes to IBM InfoSphere DataStage® to feed the data warehouse. Part 1 of the 2-part series provided an overview of these products and how they can work together. In this Part 2, explore two integration options: using WebSphere® MQ messages with InfoSphere Event Publisher and using staging tables.

View more content in this series

Date:  02 Sep 2010
Level:  Intermediate PDF:  A4 and Letter (1877 KB | 52 pages)Get Adobe® Reader®

Activity:  24467 views
Comments:  


Download

DescriptionNameSizeDownload method
Scripts for sample configurationsscripts-paper-mastercopy.zip486KBHTTP

Information about download methods

4 of 6 | Previous | Next

Comments



static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management, WebSphere
ArticleID=515022
TutorialTitle=High-performance solution to feeding a data warehouse with real-time data, Part 2: Explore the integration options with staging tables and WebSphere MQ messages
publish-date=09022010
author1-email=akrishni@us.ibm.com
author1-email-cc=
author2-email=tonylee@us.ibm.com
author2-email-cc=
author3-email=jamesyau@us.ibm.com
author3-email-cc=