DB2 pureScale disaster recovery using Storwize V7000 Metro Mirror

The DB2® pureScale® feature leverages the IBM® Storwize® V7000 storage system synchronous remote copy function like Metro Mirror to maintain an identical copy on primary and secondary volumes for disaster recovery purposes. The Metro Mirror features enable you to set up a relationship between two volumes, so updates made by an application to one volume are mirrored on the other volume. The volumes can be in the same system or on two different systems. Learn the steps required to deploy a Metro Mirror solution with the Storwize V7000 storage subsystem.

Share:

Aslam Nomani (aslam@ca.ibm.com), STSM, DB2 Quality Assurance, IBM Toronto Lab

Aslam NomaniAslam Nomani has been working with the Database Technology (DBT) team in the IBM Toronto Laboratory for five years. For the past four years, he has worked in the DB2 Universal Database (UDB) System Verification Test department. Aslam has worked extensively in testing DB2 Universal Database in high availability environments. He is currently a team lead within the DBT test organization.



Saroj Tripathy (stripat2@in.ibm.com), Advisory Software Engineer, IBM

Saroj TripathySaroj Kumar Tripathy has been working in system verification testing and functional verification testing for DB2 since 2010 at IBM. He is an IBM DB2 Certified Advanced DBA. His area of expertise is DB2 high availability, HADR and expert security solutions.



Sumair Kayani (skayani@ca.ibm.com), System Engineer, System Optimization and Competency Center, IBM

Sumair KayaniSumair Kayani is a systems engineer at IBM Toronto lab. As a member of Systems Optimization Competency Center, his responsibilities include optimization of software and hardware components that compromise the IBM integrated systems. His role as a systems administrator focused on System P, and Storage Area Networks.



Ian Boden (ianboden@uk.ibm.com), SAN Volume Controller and Storwize V7000 Software Development, IBM

Ian BodenIan Boden joined IBM in 2007 after completion of his master's in computer systems and software engineering. For the past five years, he has been working on IBM SAN Volume and Storwize V7000. His area of expertise is cache algorithms and performance.



31 January 2013

Introduction

In today's highly competitive marketplace, it is important to deploy a data processing architecture that not only meets your immediate tactical needs but that also provides the flexibility to grow and change to adapt to your future strategic requirements. In December 2009, IBM introduced the DB2 pureScale Feature for Enterprise Server Edition (DB2 pureScale Feature). The DB2 pureScale Feature leverages an active-active shared-disk database implementation based on the DB2 for z/OS® data-sharing architecture. It leverages proven technology from the DB2 database software on the mainframe to bring the active-active shared-disk technology to open systems. The DB2 pureScale Feature meets the needs of many customers by providing the following key benefits:

  • Virtually unlimited capacity— DB2 pureScale provides practically unlimited capacity by allowing for the addition and removal of members on demand. DB2 pureScale can scale to 128 members and has a highly efficient centralized management facility that allows for superior scale-out compared to peer-to-peer models. DB2 pureScale also leverages a technology called Remote Direct Memory Access (RDMA) that provides highly efficient inter-node communication mechanism that facilitates in the superior scaling ability.
  • Application transparency— An application that runs in a DB2 pureScale environment does not need to have any knowledge of the members in the cluster or of partitioning data. The DB2 pureScale Feature automatically routes applications to the members deemed most appropriate. The DB2 pureScale Feature also provides native support for syntax other database vendors use, which enables those applications to run in a DB2 pureScale environment with minimal or no changes.
  • Continuous availability— The DB2 pureScale Feature provides a fully active-active configuration such that if one member goes down, processing continues at the remaining active members. During a failure, only data being modified on the failing member is temporarily unavailable until database recovery completes for that set of data, which is very quick. This is an advantage over some competing solutions in which an entire system freeze occurs as part of the database recovery process.
  • Reduced total cost of ownership— The DB2 pureScale interfaces easily handle the deployment and maintenance of components integrated within the DB2 pureScale Feature. This helps reduce what might amount to steep learning curves associated with deploying and maintaining with some competing technologies.

The DB2 pureScale Feature provides a local high-availability solution, but many customers also require a disaster-recovery solution to meet business-continuity requirements. The DB2 pureScale Feature leverages remote disk-mirroring technology, and it is designed to work with database replication products. If the entire primary site running a DB2 pureScale instance fails, the remote site can be leveraged to enable business operations to continue. The DB2 pureScale Feature can also leverage the traditional database backup, restore, and roll-forward functionality to enable disaster-recovery solutions.

The Metro Mirror Copy Services features enable you to set up a relationship between two volumes, so updates made by an application to one volume are mirrored on the other. The volumes can be in the same system or on two systems. Although the application only writes to a single volume, the system maintains two copies of the data. If the copies are separated by a significant distance, the Metro Mirror copies can be used as a backup for disaster recovery. A prerequisite for Metro Mirror operations between systems is that the SAN fabric to which they are attached provides adequate bandwidth between the systems.

For Metro Mirror copy type, one volume is designated as the primary, and the other volume is designated as the secondary. Host applications write data to the primary volume, and updates to the primary volume are copied to the secondary volume. Normally, host applications do not perform I/O operations to the secondary volume. The Metro Mirror feature provides a synchronous-copy process. When a host writes to the primary volume, it does not receive confirmation of I/O completion until the write operation has completed for the copy on the primary volume and the secondary. This ensures that the secondary volume is always up to date with the primary volume in the event that a fail-over operation must be performed. However, the host is limited to the latency and bandwidth limitations of the communication link to the secondary volume.

The Metro Mirror operations support the following functions:

  • Intra-system copying of a volume, in which both volumes belong to the same clustered system.
  • Inter-system copying of a volume, in which one volume belongs to a system and the other volume belongs to a different system. The inter-system link is bidirectional. This means it can copy data from system A to system B for one pair of volumes while copying data from system B to system A for a different pair of volumes.
  • The copy direction can be reversed for a consistent relationship.

This article describes the steps required to deploy a Metro Mirror disk mirroring solution with the Storwize V7000 storage subsystem. Disk mirroring feature can also be deployed using disk mirroring technology from other storage vendors.


Understanding the solution configuration

In the example solution presented here, a DB2 pureScale instance is active and online at the primary site. Transactions are running against the database at the primary site, while Metro Mirror synchronously mirrors the data and logs to the target site. If a complete failure occurs on the primary site, the DB2 pureScale instance at the target site can be brought online to handle application requests while the primary site is unavailable. Figure 1 shows a basic topology of the configuration used in this example for Metro Mirror configuration.

Figure 1. Topology overview
Image shows topology overview

Figure 1 shows:

  • An identical DB2 instance called db2inst1 is created at the primary site and the target site.
  • The DB2 instance at each site consists of two members and these two CFs.
  • A Storwize V7000 storage subsystem is deployed at each site.
  • hdisk1 stores instance files in the home directory of the DB2 instance owner. This disk is not mirrored across sites.
  • hdisk2 is leveraged by DB2 pureScale cluster services. This disk is not mirrored across sites.
  • hdisk3 stores the data associated with a DB2 database. The file system created on this disk is called /db2fs/db2datafs. This disk is mirrored across sites.
  • hdisk4 stores the transaction logs associated with a DB2 database. The file system created on this disk is called /db2fs/db2logfs. This disk is mirrored across sites.
  • A storage-level consistency group is defined across both hdisk3 and hdisk4 to guarantee that a single-point-in-time copy of the data is seen at the target site upon any failure.

Figure 2 identifies more specifics of the two storage subsystems used in the example scenario.

Figure 2. Storage subsystem topology
Image shows storage subsystem topology

Deploying the DB2 pureScale Feature with Metro Mirror

The following steps explain how to deploy the Metro Mirror functionality with the DB2 pureScale Feature so that hdisk3 and hdisk4 are being mirrored from the primary site to the target site.

Step 1: Create the partnership on both V7000 servers

From the V7000 command-line interface, first check the path status between the two V7000 servers, then create the partnership between the two V7000 servers.

Listing 1. Syntax for creating the partnership
# Determine the Path status between the two V7000 servers (p-v7k-03 : Primary server,
p-v7k-01: Secondary server)
                
IBM_2076:p-v7k-03:superuser>svcinfo lspartnershipcandidate
id                 configured    system_name
00000200A0800E7E      no          p-v7k-01
                
# Create partnership on both V7000 servers.
# From primary site p-v7k-03:
IBM_2076:p-v7k-03:superuser>svctask mkpartnership -bandwidth 400 00000200A0800E7E
                
# From secondary site p-v7k-01:
IBM_2076:p-v7k-03:superuser>svctask mkpartnership -bandwidth 400 00000200A0000CB6
                
# Check partnership on primary V7000 server (p-v7k-03)
IBM_2076:p-v7k-03:superuser>svcinfo lspartnership
id             	 name      location partnership      bandwidth
00000200A0000CB6 p-v7k-03  local
00000200A0800E7E p-v7k-01  remote   fully_configured 400

Step 2: Create the Metro Mirror relationships

Create the Metro Mirror pairs between the two V7000 servers.

IBM_2076:p-v7k-03:superuser>svctask mkrcrelationship -aux secondary-d3 -cluster
00000200A0800E7E -master primary-d3
                
IBM_2076:p-v7k-03:superuser>svctask mkrcrelationship -aux secondary-d4 -cluster
00000200A0800E7E -master primary-d4

Step 3: Create consistency group and add both Metro Mirror relationships

Below commands will create the consistency group and will add both Metro Mirror relationships.

Listing 2. Syntax to create consistency group
IBM_2076:p-v7k-03:superuser>svctask mkrcconsistgrp -cluster 00000200A0800E7E -name GRP1
IBM_2076:p-v7k-03:superuser>svctask chrcrelationship -consistgrp GRP1 rcrel0
IBM_2076:p-v7k-03:superuser>svctask chrcrelationship -consistgrp GRP1 rcrel1

Step 4: Start the Metro Mirror relationship

This command will start the Metro Mirror relationship between the two V7000 servers: IBM_2076:p-v7k-03:superuser>svctask startrcconsistgrp GRP1.

Step 5: Check the status of the Metro Mirror relationship

The following command will check the status of the Metro Mirror relationship. Notice that master cluster is primary.

Listing 3. Syntax to check the status of Metro Mirror relationship
IBM_2076:p-v7k-03:superuser>svcinfo lsrcrelationship -delim :
id:name  :master_cluster_id:master_cluster_name:master_vdisk_id:master_vdisk_name:
2 :rcrel0:00000200A0000CB6 :p-v7k-03           :2	       :primary-d3:
3 :rcrel1:00000200A0000CB6 :p-v7k-03	       :3	       :primary-d4:
                
aux_cluster_id  :aux_cluster_name:aux_vdisk_id:aux_vdisk_name:
00000200A0800E7E:p-v7k-01        :6           :secondary-d3:
00000200A0800E7E:p-v7k-01        :7           :secondary-d4:
                
primary:consistency_group_id:consistency_group_name:state:            
master :0		    :GRP1		   :consistent_synchronized:
master :0		    :GRP1		   :consistent_synchronized:
                
bg_copy_priority:progress:copy_type:cycling_mode
50		:	 :metro    :none
50		:	 :metro    :none

Step 6: Creating the db2inst1 instance on the primary site

Create the db2inst1 instance on the primary site with two DB2 members and two CFs. Below steps show how to do this using the command line, but you can complete the same task from the DB2 graphical installation.

Listing 4. Syntax for Creating DB2 instance
# db2icrt -d instance_shared_dev /dev/hdisk1 -tbdev /dev/hdisk2 -m hosta:hosta- ib0 -cf
hostc:hostc-ib0 -u db2inst1 db2inst1
# db2iupdt -d -add -m hostb:hostb-ib0 db2inst1
# db2iupdt -d -add -cf hostd:hostd-ib0 db2inst1

Step 7: Creating the db2inst1 instance on the target site

Create the db2inst1 instance on the target site with two DB2 members and two CFs.

# db2icrt -d instance_shared_dev /dev/hdisk1 -tbdev /dev/hdisk2 -m hostw:hostw-ib0 -cf
hosty:hosty-ib0 -u db2inst1 db2inst1
# db2iupdt -d -add -m hostx:hostx-ib0 db2inst1
# db2iupdt -d -add -cf hostz:hostz-ib0 db2inst1

Note that any DB2 configuration parameter changes made at the primary site must be manually applied to the target site. Also, any required files that do not reside on the mirrored disks, including binary files from the creation of stored procedures, must be manually copied to the target site.

Step 8: Creating file system at the primary site

Create the file system for the DB2 data and transaction logs at the primary site and change the owner to the DB2 instance owner.

Listing 5. Syntax for creating file systems at primary site
# db2cluster -cfs -start –all
# db2cluster -cfs -create -filesystem db2datafs -disk /dev/hdisk3
# db2cluster -cfs -create -filesystem db2logfs -disk /dev/hdisk4
# chown db2inst1:db2adm /db2fs/db2datafs
# chown db2inst1:db2adm /db2fs/db2logfs

Step 9: Configure RSH and SSH

Configure passwordless rsh, ssh for root, and the user (db2inst1) for computers between the primary site and the target site.

Step 10: Synchronize the file system definition

From the primary site to the target site, synchronize the file system as shown below. Execute the command from target site (i.e., from hostw). This synchronization only needs to occur when the definition of the file system changes.

$ db2stop force
# cd /usr/lpp/mmfs/bin/mmshutdown –a

Step 10a: Creating remotenodefile in the primary site.

Create a file called remotenodefile with the entries as listed below under /usr/lpp/mmfs/bin.

hostw
hostx
hosty 
hostz

Step 10b: Exporting the file system definitions

Run the commands as shown below to export the file system definitions from the primary site to the target site.

# /usr/lpp/mmfs/bin/mmfsctl db2datafs syncFSconfig -n remotenodefile
# /usr/lpp/mmfs/bin/mmfsctl db2logfs syncFSconfig -n remotenodefile

Step 11: Validating the propagation of the file system definitions

Validate the propagation of the file system definitions to the target site by running the commands as shown below from hostw. The mmlsnsd command shows that the file system definitions for file systems db2datafs and db2logfs now exist on the target site. Once validation is complete, stop the file system.

# /usr/lpp/mmfs/bin/mmstartup -a
# /usr/lpp/mmfs/bin/mmlsnsd
# /usr/lpp/mmfs/bin/mmshutdown –a

On the primary site, an application workload can now run against the database.

Step 12: Performing the disk fail-over

Upon failure of the primary site, perform the disk fail-over such that the data on the target site can be used to continue database processing at the target site.

Do a fail-over to the target site.

IBM_2076:p-v7k-03:superuser>svctask switchrcconsistgrp -primary aux GRP1

Now check the status of the relationship. Notice, aux cluster has become the primary.

Listing 6. Syntax for checking the status of the metro mirror relationship
IBM_2076:p-v7k-03:superuser>svcinfo lsrcrelationship -delim :
id:name  :master_cluster_id:master_cluster_name:master_vdisk_id:master_vdisk_name:
2 :rcrel0:00000200A0000CB6 :p-v7k-03           :2              :primary-d3:
3 :rcrel1:00000200A0000CB6 :p-v7k-03           :3   	       :primary-d4:
                
aux_cluster_id  :aux_cluster_name:aux_vdisk_id:aux_vdisk_name:
00000200A0800E7E:p-v7k-01   	 :6	      :secondary-d3:
00000200A0800E7E:p-v7k-01	 :7	      :secondary-d4:
                
primary:consistency_group_id:consistency_group_name:state:
aux    :0		    :GRP1		   :consistent_synchronized:
aux    :0		    :GRP1		   :consistent_synchronized:
                
bg_copy_priority:progress:copy_type:cycling_mode
50  	        :	 :metro	   :none
50		        :	 :metro	   :none

Step 13: Start the file system at the target site

From the target site, start GPFS on all the hosts and mount both the file systems (db2datafs, db2logfs).

# /usr/lpp/mmfs/bin/mmstartup –a
# /usr/lpp/mmfs/bin/mmmount db2datafs –a
# /usr/lpp/mmfs/bin/mmmount db2logfs  –a

Step 14: Cataloging the database

The first time the database is accessed at the target site, catalog the database by user db2inst1 so the instance on the target site can see it: $ db2 catalog database database_alias on /db2fs/db2datafs.

Step 15: Start the database on the target site

Start DB2 on the target site with the db2inst1 user ID: $ db2start.

The database at the target site is now fully accessible and contains all committed updates that occurred on the primary site. When the primary site comes back online, the disk-mirroring direction must be reversed for the primary site to once again become the database used for processing.

Step 16: Reversing the disk-mirroring direction

Do a fail-back to primary: IBM_2076:p-v7k-03:superuser>svctask switchrcconsistgrp -primary master GRP1.

Now check the status of the relationship.

Listing 7. Syntax for checking the status of the metro mirror relationship
IBM_2076:p-v7k-03:superuser>svcinfo lsrcrelationship -delim :
id:name  :master_cluster_id:master_cluster_name:master_vdisk_id:master_vdisk_name:
2 :rcrel0:00000200A0000CB6 :p-v7k-03           :2              :primary-d3:
3 :rcrel1:00000200A0000CB6 :p-v7k-03           :3    	       :primary-d4:
                
aux_cluster_id  :aux_cluster_name:aux_vdisk_id:aux_vdisk_name:
00000200A0800E7E:p-v7k-01        :6  	      :secondary-d3:
00000200A0800E7E:p-v7k-01        :7	      :secondary-d4:
                
primary:consistency_group_id:consistency_group_name:state:
master :0		    :GRP1	 	   :consistent_synchronized:
master :0		    :GRP1		   :consistent_synchronized:
                
                
bg_copy_priority:progress:copy_type:cycling_mode
50		:	 :metro	   :none
50		:	 :metro	   :none

Step 17: Stopping the DB2 instance and the file system at the target site

To reintegrate the original primary site as the location for data processing, stop the DB2 instance and the file system from host hostw at the target site.

$ db2stop force
# cd /usr/lpp/mmfs/bin/mmshutdown –a

Step 18: Starting the instance and the file system at the primary site

From the primary site, start the GPFS on all the hosts and mount both the file systems (db2datafs, db2logfs). Start the DB2 instance from the host hosta.

# /usr/lpp/mmfs/bin/mmstartup –a
# /usr/lpp/mmfs/bin/mmumount db2datafs –a
# /usr/lpp/mmfs/bin/mmumount db2logfs  –a

Note: You can leverage the DB2 automatic client-reroute feature to automatically reconnect the application across sites once a site failure or reintegration has occurred.


Conclusion

The DB2 pureScale Feature for Enterprise Server Edition provides a database solution that meets the needs of the most demanding users. It is designed for high availability to allow for business continuity in the event of planned and unplanned outages. You can deploy a disaster recovery solution to allow customers to continue to run their businesses even in the unfortunate circumstance of a full site failure. The DB2 pureScale Feature combined with a solution like the Storwize V7000 Metro Mirror described here provides a resilient solution designed to meet the high-availability and disaster-recovery requirements of the most demanding customer environments.

Resources

Learn

Get products and technologies

  • Build your next development project with IBM trial software, available for download directly from developerWorks.
  • Now you can use DB2 for free. Download DB2 Express-C, a no-charge version of DB2 Express Edition for the community that offers the same core data features as DB2 Express Edition and provides a solid base to build and deploy applications.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Information management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management
ArticleID=856655
ArticleTitle=DB2 pureScale disaster recovery using Storwize V7000 Metro Mirror
publish-date=01312013