Contents


5 steps for real-time data replication with IBM i remote journaling

Configure IBM InfoSphere Data Replication for real-time replication with IBM i remote journaling

Comments

IBM InfoSphere® Change Data Capture (CDC) is included in IBM InfoSphere® Data Replication (IIDR). InfoSphere Change Data Capture replicates large volumes of heterogeneous data in near real time, helping you integrate data across your enterprise. This built-in software also supports cloud and big data databases.

IIDR CDC Replication Engine for DB2® for i (IIDR CDC for DB2 for i) is a journal-based product. Each subscription has its own log-reading (or, journal-scraping) process to capture changed data from the source tables within the replication subscription.

The architecture of remote journaling in IBM InfoSphere Data Replication

Remote journal management is a System i® feature that copies local journal entries from one system to another system. Remote journaling lets you maintain a data replica, which is a copy of your original data that resides on another system. The original data resides on a primary system, and applications make changes to the original data during normal operations. Once the remote journal function is activated, the source system continuously replicates journal entries to the new system.

Figure 1. The architecture of remote journal management in IBM InfoSphere Data Replication
The architecture of how remote journaling works in an IIDR                     solution
The architecture of how remote journaling works in an IIDR solution
  • The "data origin server" is the System i system where your source files, local journal objects, and receivers reside.
  • The "source server" refers to the system where your remote journals objects and receivers reside.

You must install IIDR CDC on the data origin server, the source server, and in the same product library. After installation, you must start the subsystem and the listener. The CDC on the data origin server only runs used service jobs that run occasionally and doesn't run replication jobs.

Advantages of using remote journal management

Remote journal management benefits the source system because it:

  • Lowers the source system's CPU consumption by shifting the processing requirements for receiving journal entries from the source system to the target system.
  • Eliminates the need to buffer journal entries to a temporary area before transmitting them to the target system, resulting in fewer disk writes and greater direct access storage device (DASD) efficiency on the source system.
  • Requires less code, which significantly improves the replication performance of journal entries and allows database images to be sent to the target system in real time.
  • Reduces the amount of resources used by the source system by moving the journal receiver's "Save and restore" operations to the target system.

Deploying remote journal management in IBM InfoSphere Data Replication solution

The following steps show you how to deploy remote journal managmement in IIDR using the following environments as an example:

Data origin server: i7cdc04.torolab.ibm.com

Source server: i7cdc06.torolab.ibm.com

Step 1. Install IBM InfoSphere Data Replication

  1. Install IIDR CDC on the data origin server and the source server separately.
  2. Install InfoSphere CDC on the target system to receive data from the source system and to apply that data elsewhere.

Note: IIDR CDC must be installed in the same product library on the data origin server and the source server.

Ensure that the System i user profile for both the data origin server and the source server have the same name. You will use this user profile to connect to the replication agent you created on the source server.

Step 2. Set up the remote journal from the data origin server (i7cdc04) to the source server (i7cdc06)

1. On the source server i7cdc06, add a relational database directory entry by running the ADDRDBDIRE command. The database entry should be the same as the name of the machine.

For example, to create the relational database directory on i7cdc06 named I7CDC06, enter the following command:

ADDRDBDIRE RDB(17CDC06) RMTLOCNAME(*LOCAL *IP)

2. On the data origin server i7cdc04, add a relational database entry for the database on the data origin server by running the ADDRDBDIRE command. For example, to create relational database directory entry on i7cdc04 named I7CDC06, enter the following command:

ADDRDBDIRE RDB(17CDC06) RMTLOCNAME(17CDC06.TOROLAB.IBM.COM *IP)

3. On the data origin server i7cdc04, prepare your system for the journal you want to use for remote journaling. For example, the table liuyan/table1 exists on i7cdc04. If the journal does not exist, you need to create the journal with the following commands.

To create the journal receiver liuyan/table1rcv, issue the command:

crtjrnrcv jrnrcv (liuyan/table1rcv).

To create the journal liuyan/table1jrn for the receiver liuyan/table1rcv, issue the following command:

crtjrn jrn (liuyan/table1jrn> jrnrcv (liuyan / table1rcv).

To start journaling for the liuyan/table1, issue the command:

strjrnpf file(liuyan/table1) jrn(liuyan/table1jrn).

Start journaling for both tables by using the command:

CHGJRNOBJ OBJ((LIUYAN/TABLE1 *FILE)) ATR(*IMAGES) IMAGES(*BOTH).

Make sure you are starting the journaling using *BOTH (before and after images). Otherwise, you will get an error when you create subscriptions.

4. On the Data Origin Server i7cdc04, add a remote journal by running the ADDRMTJRN command. To do this:

  • Set the relational database to the database directory entry created for i7cdc06.
  • Set the name of the local journal that was created for the table liuyan/table1.
  • Set the name of the remote journal as you want.

Create the remote journal liuyan/table1jrn, which is the same as the local journal's name, by issuing the command:

addrmtjrn rdb(i7cdc06) srcjrn(liuyan/table1jrn) tgtjrn(liuyan/table1jrn).

5. On the data origin server i7cdc04, activate the remote journal by issuing the command:

chgrmtjrn rdb(i7cdc06) srcjrn(liuyan/table1jrn) tgtjrn(liuyan/table1jrn) rnstate(*activate).

Step 3. Configure the source server to retrieve journal from the data origin server

1. On the source server i7cdc06, add a relational database directory entry that references the database where the source files reside on the data origin server.

To do this, issue the command:

addrdbdire rdb(i7cdc04) rmtlocname(i7cdc04.torolab.ibm.com *IP).

2. On the data origin server i7cdc04, add a relational database entry that references the local database, by issuing the command:

addrdbdire rdb(i7cdc04) rmtlocname(*local *IP).

So far, you've seen how to configure a remote journal setup. Next, see how to get your remote journal to work with Change Data Capture (CDC).

Step 4. Set up source datastore on management console to retrieve data from the source files that will live on the data origin server

1. Ensure that a source instance is created on both the source server and the data origin server. For example, create an IIDR CDC instance on i7cdc04 with port 11553. Create an IIDR CDC instance on i7cdc06 with port 11558.

2. Add the following three system parameters on the management console for the source datastore, as show in the following image:

  • Data Origin Relational DB Entry: The relational database directory entry, which should reference the Data Origin Server.
  • Data Origin TCP/IP Name: The IP address or hostname of Data Origin Server.
  • Data Origin TCP/IP Port: The TCP listener port number of the IIDR CDC instance created on Data Origin Server.

For example:

  • Data Origin Relational DB Entry: Set to I7CDC04.
  • Data Origin TCP/IP Name: Set to the IP address (9.26.104.160) of i7cdc04.
  • Data Origin TCP/IP Port: Set to the CDC instance port 11553
Figure 2. Add system parameters on MC
Figure 12: Add system parameters on MC
Figure 12: Add system parameters on MC

Step 5. Start mirroring from the management console

If you don't see any error in the log and the data is replicated to the target correctly, that means IIDR is correctly configured with IBM i remote journaling.

Troubleshooting problems

Ensure that a SQL package named 'DMS125SQL' is created in the QGPL library on the data origin server.

To create the SQL package, on the source server, change the current library to the IIDR product library and issue the following command:

CRTSQLPKG PGM (<product library>/DMS125) RDB (<RDB entry>)
                OBJTYPE (*SRVPGM)

The relational database (RBD) entry must be a valid RDB entry on the source server whose remote location is the data origin server. Verify that a *SQLPKG named 'DMS125SQL' was created in the QGPL folder.

You might get an authentication failure when you try to add a remote journal by running the ADDRMTJRN command on the data origin server.

Figure 3. Error when adding a remote journal
Error when adding a remote journal
Error when adding a remote journal

If you do, check the detailed error messages.

Figure 4. Detailed error messages when adding a remote journal
Figure 14: Detailed error messages when adding a remote                     journal
Figure 14: Detailed error messages when adding a remote journal

Change the authentication method by issuing the CHGDDMTCPA command:

CHGDDMTCPA AUTOSTART (*YES) PWDROD(*USERID) ENCALG(*DES)

After that, create the remote journal again.

Conclusion

This article illustrates how to configure IBM InfoSphere Data Replication (IIDR) with IBM i remote journal management for data synchronization. It also detail common problems you might experience during the set-up process.

Acknowledgements

I would like to thank Beata Pliszka and Jacob De Vos for their inputs and editing assistance with this article.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management
ArticleID=1021536
ArticleTitle=5 steps for real-time data replication with IBM i remote journaling
publish-date=11202015