This post is the answer to one of the FAQs found in License Tips for IBM Data Replication.
You may have seen a recent announcement on ibm.com
that says IBM would no longer be marketing it's older data replication products in 2013. That includes InfoSphere CDC. Why?
And what happens to the CDC technology?
Over the years, IBM provided its data replication technologies
through a lot of different products. For example, IBM used to offer two major
data replication products at the same time -
InfoSphere CDC and InfoSphere Replication Server. That was a little
even to some IBM people. To simplify the situation, IBM consolidated all it's replication technologies into a single product called IBM InfoSphere Data Replication
(IIDR). Once IIDR was available, the older products no longer needed to be sold to new customers. That's why the end of marketing was announced. However, the replication technologies - CDC, Q Replication, and SQL Replication - are still alive and well. You can
continue to use them as you always
have. Of course, you may have two related questions:
- Are the older products still being supported?
- How do you move from your old InfoSphere CDC product to IIDR?
If you have any questions, feel free to post them in the comments section of this blog.
If you're looking for an excellent way to replicate changed data from a wide range of databases into a Netezza appliance, you can do so through InfoSphere Data Replication
. The latest release provides an Apply program that is both native to Netezza and optimized for Netezza targets. This Apply is built from Data Replication's CDC technology and is also compatible with the CDC technology found in InfoSphere Change Data Capture and InfoSphere Classic Change Data Capture for z/OS
. This means you can replicate data to Netezza from source databases ranging from Oracle, DB2, and others on UNIX or Windows to DB2* and IMS on the mainframe. Ordering information can be found in the Data Replication announcement letter on ibm.com
* Data Replication's CDC Apply program cannot be used to feed changed data to the IBM DB2 Analytics Accelerator (IDAA).
In response to: Best Practice - Target Considerations
Regarding Oracle triggers on the target tables, would these fire
during a standard refresh? We want them to fire off during normal
continuous replication from DB2 to Oracle..
Most are triggers on insert so I am concerned during the refresh
which we do whenever we change the table structure to match the
The following items need to be considered and taken into account when you are planning a replication architecture.
§Target table triggers
–Often if the target is a mirror image of the source, you may have triggers on target tables that if fired will have an affect on other tables that InfoSphere CDC is replicating into (CDC would have mirrored the source trigger effect and will get duplicate actions). To alleviate this, you should disable the trigger on the target table.
§Referential integrity constraints with DELETE CASCADE flag on target tables
–Similar to trigger, having cascade deletes set on the target will cause replication to try and delete a record (based on the delete that CDC would have replicated from the source log) that the database may have already deleted (or visa-versa). The following strategy can be deployed to deal with cascaded deletes:
Disable the RI constraints on target prior to starting replication
Please note that re-enabling these constraints may take some time during cut-over if you need to fail over to the target
Strategy: test how long re-enabling the RI constraints takes. If re-enabling all RI constraints takes too long and would impact your RTO (Recovery Time Objective), investigate whether it is possible to leave the RI constraints enabled and just change the CASCADE DELETE flag at cut-over time.
In 2011, IBM released three new data replication products:
One question that comes up is whether the two IMS replication products are compatible with either the new Data Replication product or the existing InfoSphere CDC products. The answer is yes - the IMS products are compatible with both new and existing products that contain the CDC technology. IMore specifically, they can provide IMS changed data to any data replication solution that you can build with IBM's CDC technology. For example, you can create unidirectional (one-way) subscriptions that feed IMS changed data to any database that can be targeted by CDC:
Two notes about this picture:
- IBM recommends you use the CDC technology in IIDR if you do not own InfoSphere CDC.
- The target DB2 can be DB2 for z/OS, DB2 LUW, or DB2 for System i.
You could also feed IMS changed data into other business software such as ETL, IBM's DataStage, and ESBs:
In other words, the new IMS data replication products extend the reach of IBM's CDC technology by adding IMS as a source for log-based capture of changed data. If you have technical questions, see the Classic CDC section of the Information Center
Modified on by MikeJory
IIDR 22.214.171.124 has been released with new functionality in addition to resolving some APARs.
IIDR 126.96.36.199-4 can be found on Fix Central.
* No IIDR downtime support for Microsoft SQL Server non-structural online index rebuild operations
Index and table reorganizations done to improve query performance no longer impact replication. Replication will continue with no outage and with no user actions required.
* Log based change capture for Oracle is now supported on Microsoft Windows
* No IIDR downtime support for Oracle Automatic Storage Management rebalancing activities
Previously in some situations rebalancing of ASM could could require replication be restarted
* An unplanned failure of Oracle to a DataGuard standby database no longer requires a resynchronization
Previously if an Oracle database being used as a source of changes failed over to an Oracle DataGuard standby, you would have been required to refresh the target data before restarting replication
* IIDR can now capture changes directly from ASM on Exadata
Previously replication required the ASM logs to be multiplexed to a non-ASM destination which typically could only be accomodated off the Exadata appliance
* IIDR can now target Cloudant
* IIDR can now use webHDFS to deliver change data to HDFS
This allows IIDR to be installed remotely from the cluster and enables targeting HDFS deployments that are secured via Kerberos
JR53019 POTENTIAL FOR DUPLICATE TRANSACTION REPLAY ON THE TARGET DUE TO INCORRECT BOOKMARKING ON IMMEDIATELY PREVIOUS SHUTDOWN.
JR53294 CDC TRIES TO READ AN UNCOMPLETED ORACLE LOG AND GETS ERROR IN LOG SHIPPING USING DATAGUARD SCENARIO.
JR53370 CDC DATA LOSS / CONSISTENCY UPOEN CDC RESTART AFTER ABORT
JR52825 CDC FOR DATASTAGE IN DIRECTCONNECT MODE NOT HONOURING THE DELAY PARAMETER FROM BATCH THRESHOLD SUBS PROPERTY
JR52973 DMSHUTDOWN COMMAND WITH -A OPTION DOESN'T TERMINATE SUBSCRIPTIONS
JR52797 ADDRESSING AN IDENTIFIED BUFFER OVERRUN CONDITION IN THE ORACLE SOLARIS JVM
JR53411 SETTING BLOB/CLOB TRUNCATION IN DATASTAGE PROPERTIES FOR A SUBSCRIPTION, REVERSES THE LIMITS
JR52923 TIMESTAMP IS NOT PROPERLY REPLICATING
JR53413 HAVING ROW SIZES GREATER THAN 341333 BYTES THROWS JAVA.NIO.BUFFEROVERFLOWEXCEPTION
JR52946 INSTANCE USING UPPER CASE ORACLE SID FOR STANDBY DATABASE CANNOT BE CREATED
JR53340 IIDR FOR ORACLE 12C FAILS TO START REPLICATION WHEN AN INSCOPE TABLE WAS ALTERED
For those who have been using the CDC Developer Works Community, you will notice a change to the look & feel of the entry page. This new menu driven mechanism will provide extra flexibility as significant new content gets added to this community. I have tested it out on various browsers, so if you experience an issue, please send me an email at firstname.lastname@example.org and I will investigate.
Note I will also be cleaning up some old blog entries where questions were asked. Questions should continue to be asked on the forum versus the blog.
Hope you continue to get benefit from this community.
The new IIDR 11.3.3 Release is available today!
Here is a link to a page that describes the new functionality available: https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W8d78486eafb9_4a06_a482_7e7962f5ac59/page/IIDR%20CDC%20New%20Features%20By%20Release?section=IIDR%2011.3.3
There are a large number of usability enhancements, new platform/DB support such as SQL Server 2014, along with technology previews/beta available for many new key technologies such as a Cloudant apply, WebHDFS support for Hadoop, and many others.
The following rules apply with respect to what Versions of Management Console (MC), Access Server, and CDC agents (engines) will inter-operate.
These rules apply to any CDC 6.x or higher release.
1) The MC and AS must be at the exact same release level
2) The CDC source and target agents (engines) can be at different release levels
3) The MC version must be >= the most recent CDC source or target agent (engine)
I have updated the look and feel of the wiki to make it easier to navigate and find what you need.
Samples are now organized in their own table and can be found by clicking the "Samples" icon on the home page.
All "how-to" documents can now be found by clicking on the "Documents" icon on the home page.
A new section has been added to the wiki which describes the features that have been added to all the recent IIDR CDC releases. In the near future I will be adding presentations with details on the features in this section.... so please check back.
I've added three new videos to my channel. They walk through configuring, operating and monitoring data replication using the CDC Management Console. This is basically the same thing you'd get if you came by the InfoSphere demo room at Information On Demand (now Insight) and agreed to let me show you a quick demo of CDC.
Here's the link to my channel "James talks about Data Replication":
I'm recording some videos where I provide technical background about data replication. I've created a YouTube channel to collect them all together. The channel is "James Talks about Data Replication". Here's a link:
I've uploaded two so far. The first one discusses the special considerations you should be aware of when using data replication with tables that have duplicate rows. The second discusses the role of data replication when moving to a real time operational analytics system from a traditional batch oriented data warehouse.
There are multiple deployment models available for InfoSphere CDC. The deployment model chosen for the source system will significantly affect the complexity of implementation.
Here are the CDC source deployment options from the least complex to the most complex:
1. InfoSphere CDC scraper runs on the source database server
2. InfoSphere CDC scraper runs on a remote tier reading logs from a shared disk (SAN)
This configuration is available for Oracle and Sybase. DB2 LUW has a similar capability, but utilizes a remote client instead of reading from a SAN.
3. InfoSphere CDC scraper runs on a remote tier using log shipping
This configuration is only available for Oracle.
Rule of Thumb
You should always use the least complex deployment option that will meet the business needs. The vast majority of CDC users install InfoSphere CDC on the source database server.
With a mere 4 weeks until IBM's 2013 Information on Demand, the data replication team thought it might be helpful to have a complete listing of all data replication sessions at IOD. From client presentations and our product roadmap to sneak peeks at new IBM Data Replication functionality, our sessions run the gamut!
Simply take a gander at the sessions below then go to the IOD agenda builder, click on Create Sign In, and then enter your confirmation number and the email address that you used to register for the conference. Create your agenda today!
§Using ‘Standard’ replication achieves much higher throughput performance than using ‘Consolidation’ or ‘Summarization’
Standard replication can do optimizations such as arraying, commit grouping, etc that can not be performed when using the other replication methods
Note some optimizations will also be disabled if using Adaptive apply or Conflict Detection & Resolution
§Be aware when you are parking tables/subscriptions
An inactive (not currently replicating) subscription that contains tables with a replication method of Mirror will continue to accumulate change data in the staging store from the current point back to the point where mirroring was stopped. For this reason, you should delete subscriptions or remove tables that are no longer required, or change the replication method of all tables in the subscription to Refresh to prevent the accumulation of change data in the staging store on your source system.
The same is true with a parked (idle) table. You need to insure that the replication method is set to Refresh