The DB2® pureScale Feature for Enterprise Server Edition builds on familiar and proven design features from the IBM DB2 for z/OS® database software. This article describes how to deploy the DB2 pureScale Feature. The example scenario is based on a configuration using two P6 550s servers connected to a DS5300 storage controller.

Aslam Nomani (aslam@ca.ibm.com), Quality Assurance Manager, IBM

Aslam Nomani has been with IBM for 14 years with most of that time spent with the Quality Assurance team. His main area of focus has been on high availability and disaster recovery solutions. Over the past several years Aslam has been the Quality Assurance Architect for the DB2 pureScale Feature.



Yashavanthi Renukaswamy (yrenukas@in.ibm.com), DB2 Software Developer, IBM

Yashavanthi Renukaswamy is an IBM certified Advanced Database Administrator. She has been working with IBM for 5 years on the DB2 database focusing on System Verification Testing. She has over 10 years of experience in QA on various databases. She is an active participant for IBM in many different forums, tech talks with customers, and University Relationship programmes.



Jimmy Stamkos (jstamkos@ca.ibm.com), DB2 Software Developer, IBM

Jimmy has been working at IBM for 4 years in the DB2 Quality Assurance team. Over the past few years, Jimmy has focused on the quality of the DB2 pureScale Feature and validating high availablility and AIX integration capabilities.



20 May 2010

Also available in Chinese Russian Spanish

Introduction

In today's highly competitive marketplace, it is important to deploy a data processing architecture that not only meets your immediate tactical needs but that also provides the flexibility to grow and change to adapt to your future strategic requirements. In December 2009, IBM introduced the DB2 pureScale Feature for Enterprise Server Edition (DB2 pureScale Feature). The DB2 pureScale Feature leverages an active-active shared-disk database implementation based on the DB2 for z/OS data-sharing architecture. It leverages proven technology from the DB2 database software on the mainframe to bring the active-active shared-disk technology to open systems.

The DB2 pureScale Feature meets the needs of many customers by providing the following key benefits:

Virtually unlimited capacity
The DB2 pureScale Feature provides practically unlimited capacity by enabling the addition and removal of members on demand. The DB2 pureScale Feature can scale to 128 members. It has a highly efficient, centralized management facility that allows for very efficient scale out capabilities. The DB2 pureScale Feature also leverages a technology called Remote Direct Memory Access (RDMA) that provides a highly efficient inter-node communication mechanism to augment its scaling capabilities
Application transparency
An application that runs in a DB2 pureScale environment does not need to have any knowledge of the different members in the cluster or of partitioning data. The DB2 pureScale Feature automatically routes applications to the members deemed most appropriate. The DB2 pureScale Feature also provides native support for syntax that other database vendors use, which enables those applications to run in a DB2 pureScale environment with minimal or no changes.
Continuous availability
The DB2 pureScale Feature provides a fully active-active configuration such that if one member goes down, processing continues at the remaining active members. During a failure, only data being modified on the failing member is temporarily unavailable until database recovery completes for that set of data, which is very quick. This is an advantage over some competing solutions in which an entire system freeze occurs as part of the database recovery process.
Reduced total cost of ownership
The DB2 pureScale interfaces easily handle the deployment and maintenance of components integrated within the DB2 pureScale Feature. This helps reduce what might amount to steep learning curves associated with deploying and maintaining with some competing technologies.

To understand these benefits, it helps to understand the basic architecture. Clients connect to a single database view comprised of four cluster members with shared storage. The clients connect anywhere. The DB2 member runs on several host computers with integrated cluster services. The members are connected over low-latency, high-speed connections to the PowerHA pureScale server. The logs and data-sharing architecture are centralized for all the clients. Figure 1 depicts the components that make up the DB2 pureScale configuration. Even though there are several advanced components, a significant portion of this architecture is transparent to the end user, because the DB2 pureScale Feature internally deploys and manages these components.

Figure 1. DB2 pureScale Feature topology overview
Clients connect to a single database view comprising 4 cluster members with shared storage

In Figure 1, clients can connect to any member. The DB2 pureScale Feature can automatically load balance the clients across the different members based on the utilization of each computer. If any host in the configuration fails, the DB2 pureScale Feature redirects clients among the active members on the remaining hosts.

Each DB2 member represents a DB2 processing engine. Up to 128 members can be deployed in a single DB2 pureScale configuration. The members cooperate with each other and with the PowerHA pureScale server to provide coherent access to the database from any member. Members can be added and removed as processing demands change. Modifications to the number of active members can be done without any impact to the clients.

Integrated with the DB2 pureScale Feature is a cluster services layer that provides failure detection, recovery automation, and a clustered file system. These technologies are integrated within the DB2 pureScale Feature, and they leverage IBM technologies optimized for DB2 software. The technologies include IBM Tivoli® Systems Automation for Multiplatforms (Tivoli SA MP), Reliable Scalable Cluster Technology (RSCT), and General Parallel File System (GPFS™). The DB2 pureScale Feature automatically deploys and configures these technologies according to a best-practice, pre-defined configuration. A customer does not need to determine how to configure the clustering technology that comes with the DB2 pureScale Feature, because the end user doesn't see it.

In the DB2 pureScale configuration, there is a need for communication between the members and the PowerHA pureScale servers. To make this communication as efficient as possible, the DB2 pureScale Feature leverages the RDMA technology. RDMA allows one computer to read or write to the memory of another computer without requiring any processor cycles on the target machine. This mechanism, in conjunction with extremely high-speed networks such as InfiniBand, allows for an extremely efficient transport layer, which enables the DB2 pureScale Feature to scale efficiently.

The PowerHA pureScale servers (also referred to as CF) provide a scalable and centralized locking mechanism to ensure data coherency. The servers also act as a fast cache for DB2 pages, leveraging RDMA technology to provide increased performance in situations where a physical disk operation might otherwise have been required. The PowerHA pureScale server and the efficient transport layer are two of the features that enable the DB2 pureScale Feature to scale so well, because each member does not have to negotiate with all other members when performing a task.

The DB2 pureScale Feature leverages a shared-disk technology. Any member can read or write to any portion of the database. If any member fails, the full set of data is still accessible from the other active members.


Deploying the DB2 pureScale Feature

Figure 1 shows four members for the sake of illustration, but the example deployment scenario described in this article deploys a two-member configuration.

Configuration overview

This scenario deploys the DB2 pureScale Feature on a pair of P6 550 machines with each physical computer having two Logical Partitions (LPARs). Each LPAR has the following characteristics:

  • It is on a public network that allows for client connectivity
  • It is on an InfiniBand network to allow for high-speed, low-latency communication between the members and the PowerHA pureScale servers
  • It has shared connectivity to a common set of disks

For the example scenario, ServerA and ServerB are connected to ServerC and ServerD by an InfiniBand network. The servers are connected to clients by the public network and by a storage area network.

Figure 2. Sample DB2 pureScale Feature hardware configuration
Clients connect through a public network to the server memebers that are connected to one another by InfiniBand Network. They connect to storage through Storage Area Network.

Table 1 shows the high-level configuration of each LPAR.

Table 1. Configuration overview
HostnameServerAServerBServerCServerD
OS level AIX® 6.1 TL3 SP3
ifix iz68803.epkg.z
AIX 6.1 TL3 SP3
ifix iz68803.epkg.z
AIX 6.1 TL3 SP3
ifix iz68803.epkg.z
AIX 6.1 TL3 SP3
ifix iz68803.epkg.z
Server typeMember 0 Primary CF Member 1 Secondary CF
Cores 6 2 6 2
RAM 20 GB 12 GB 20 GB 12 GB
Shared disks hdisk2 - 64MB Disk leveraged by DB2 Cluster Services layer
hdisk3 - 100 GB disk used to create a shared file system to hold shared instance files across hosts
hdisk4 - 200 GB disk used for DB2 data
hdisk5 – 20 GB disk used for DB2 transaction logs
Note: Disk sizes vary based on specific requirements
Disk device driver default MPIO (6.1.3.2) default MPIO (6.1.3.2) default MPIO (6.1.3.2) default MPIO (6.1.3.2)
Ethernet interface en0 en0 en0 en0
InfiniBand hostname servera-ib0 serverb-ib0 serverc-ib0 serverd-ib0
InfiniBand interface ib0 ib0 ib0 ib0
OpenSSH 4.5.0.5302 4.5.0.5302 4.5.0.5302 4.5.0.5302
FirmwareIBM,EL350_039 IBM,EL350_039

For information about how to configure InfiniBand, refer to Appendix A.

Prerequisites

Complete the following steps to prepare to install the DB2 pureScale Feature. Note that all commands listed during the installation steps should be executed as a user with root privileges unless specified otherwise.

  1. Ensure password-less ssh is setup within all the physical computers participating in a DB2 pureScale cluster. You can validate the ssh configuration by issuing the # ssh <target machine> hostname command from each computer to each computer in the cluster, and ensure it returns the valid hostname without any prompting.
  2. Make sure you have at least 5 GB of free space in the /tmp file system of each computer.
  3. Identify the disks that will be used for the DB2 pureScale Feature, and ensure each is tagged with a physical volume identifier (PVID) by completing the following steps:
  1. Enter the lspv command to list all physical volumes available on a computer, as shown in Listing 1.
Listing 1. List of physical volumes
# lspv
hdisk0 	00cc14e2ad7c570e 	rootvg 	active
hdisk1 	00cc14e2ba1a1dcd 	homevg 	active
hdisk2 	None		 	None
hdisk3 	None		 	None
hdisk4 	None	 		None
hdisk5 	None 			None

hdisk2 to hdisk5 do not have a PVID identified in the second column.

  1. To add a PVID, run the chdev command from one computer in the cluster. The results are shown in Listing 2.
Listing 2. List from the chdev command
# chdev -l hdisk2 -a pv=yes
# chdev -l hdisk3 -a pv=yes
# chdev -l hdisk4 -a pv=yes
# chdev -l hdisk5 -a pv=yes
  1. On each of the other computers in the cluster, run the commands in Listing 3 to remove the disk names.
Listing 3. Remove disk names
# rmdev -dl hdisk2
# rmdev -dl hdisk3
# rmdev -dl hdisk4
# rmdev -dl hdisk5
  1. Run the cfgmgr command from each computer in the cluster.
  2. Run the lspv command to display each hdisk's PVID that can uniquely identify the disk from any computer within the cluster, as shown in Listing 4.
Listing 4. Display PVIDs
# lspv
hdisk0 	00cc14e2ad7c570e	rootvg	active
hdisk1	00cc14e2ba1a1dcd 	homevg 	active
hdisk2 	00cc14e23bb31057 	None
hdisk3 	00cc14e23bb3b9ee 	None
hdisk4 	00cc14e23bb3e9dd 	None
hdisk5 	00cc14e23bb3e9df 	None

Tip: To validate the size of a disk, run the bootinfo -s hdisk2 command.

DB2 pureScale Feature installation steps

  1. Identify one of the LPARs as the installation-initiating host (IIH). This host runs the installation program for the DB2 pureScale Feature. The example scenario uses ServerA as the IIH.
  2. Ensure that the display is set to display graphical tools.
  3. Navigate to the ese_dsf folder on the downloaded product image or to the root directory of the product installation DVD, and run the /db2setup -t /tmp/db2setup.trc -l /tmp/db2setup.log command, which launches the DB2 graphical installer.
  4. The execution of db2setup takes you to a welcome screen, and an information panel is displayed as shown in Figure 3.
Figure 3. DB2 setup launchpad
Screen cap of DB2 Setup Launchpad
  1. Click the various topics on the left pane to see more information. The Architecture Overview topic provides a high-level overview of the DB2 pureScale instance environment.
  2. Click Install a Product > Install New to install the DB2 pureScale Feature, as shown in Figure 4.
Figure 4. DB2 setup launchpad, selecting to install
Screen cap: Install a Product as Root with Install New selected for DB2 Ent Server Edition with the pureScale Feature Version 9.8

A progress indicator bar appears at the bottom of the window. The setup wizard appears and provides a set of intuitive instructions to navigate through the remainder of the deployment.

  1. Click Next to continue the installation.
  2. Optionally, click View Features to see which features will be installed, as shown in Figure 5.
Figure 5. Setup wizard with View Features button
Screen cap: View Features button selected

The next window displays the software licensing associated with the DB2 pureScale Feature, as shown in Figure 6.

Figure 6. Licensing terms
Screen cap: Software License Agreement with Accept selected
  1. Read the license, click the Accept radio button if you accept the licensing terms, and click Next.
  2. Select an installation method. For the example scenario, install the DB2 pureScale Feature directly from the graphical interface by selecting the first option, Install DB2 Enterprise Server Edition with the pureScale Feature, as shown in Figure 7.
Figure 7. Selecting the installation
Screen cap: Select insatllation, response file creation, or both window with Install DB2 Enterprise Server Edition with the pureScale Feature selected
  1. Click Next.
  2. Provide information for the installation directory of the DB2 binaries, and click Next. In most cases, the default path, /opt/IBM/db2/V9.8, is appropriate, as shown in Figure 8.
Figure 8. Select path
Screen cap: Select the installation directory window with Directory of /opt/IBM/db2/V9.8
  1. Click Create a DB2 instance, and click Next, as shown in Figure 9.
Figure 9. Select Create a DB2 instance
Screen cap: Set up a DB2 instance window with Create a DB2 instance selected
  1. Enter the information of the instance owner, and click Next, as shown in Figure 10. Note that if an existing user is selected for the DB2 instance owner or for the DB2 fenced user, the user must exist on all the hosts with the same UID, GID, group name, and $HOME path. The hosts should not share the instance owner's $HOME directory, because that should be local to each host. If you want to create a new user, the defined new user must not exist in any of the hosts.
Figure 10. Set user information for the DB2 instance owner
Screen cap: User information window with db2sidm1 for user and group names, obscured passwords, and /home/db2sidm1 for home directory
  1. Enter the fenced user information, and click Next, as shown in Figure 11.
Figure 11. Set user information for the fenced user
Screen cap: Fenced user info with db2sidm1 for shows user information for fenced user with db2sidm1 for user and group names, obscured passwords, and /home/db2sidm1 for home directory

The next steps use one of the predefined disks to create a shared file system the DB2 pureScale instance environment uses for instance files that are shared across all computers. The file system will be mounted as /db2sd<timestamp>, and the directory /db2sd<timestamp>/<instance_name> will be the default database path (as defined by the DFTDBPATH configuration parameter). Another small disk will be leveraged for automatic internal cluster recovery purposes.

  1. Provide the full path to the disks, and click Next, as shown in Figure 12.
Figure 12. Set up a DB2 cluster file system
Screen cap: Set up a DB2 Cluster File System window with Shared disk path of /dev/hdisk3 and Device path of /dev/hdisk2
  1. Add the remaining hosts that should be part of the DB2 pureScale cluster. By default, the IIH will already be included. For each host that you need to add, click Add, as shown in Figure 13.
Figure 13. Host list
Screen cap: Host List window with ServerA selected
  1. Enter the host name, and click OK.
  2. Repeat Steps 15 and 16 for each host that you need to add, as shown in Figure 14.
Figure 14. Adding remote hosts
Screen cap: window to enter remote host name ServerB

Note that after each host is entered, the DB2 installation program does validation checking to ensure everything is configured properly for the new host, as shown in Figure 15.

Figure 15. Validation of installation settings
Screen cap: Validation of installation settings in progress

After adding all the hosts, the host list window appears with a check mark beside each host to confirm it has been validated, as shown in Figure 16.

Figure 16. Completed host list
Screen cap: Host List window with ServerA, ServerB, ServerC, and ServerD selected
  1. Check the configuration identified in the bottom portion of the window, and click Next. The DB2 installation program does one more validation of the passwordless ssh configuration. If there is more than one physical computer in the configuration, the installation wizard automatically assigns each PowerHA pureScale server to a different computer.
  2. Confirm the assignments in the Instance Settings section of the window.
  3. Optionally change which hosts serve which function in terms of members or PowerHA pureScale servers by clicking Advanced. It is strongly recommended that you have two PowerHA pureScale servers identified on two distinct physical computers to ensure that you do not have a single point of failure.
  4. Check and correct the summary of the values, and click Finish to start the installation, as shown in Figure 17.
Figure 17. Start copying files
Screen cap: Start copying files with DB2 pureScale Feature to install and a list of components already installed

During the actual installation process, a progress monitor bar enables you to see the progress of the installation, as shown in Figure 18. The installation takes several minutes as the DB2 pureScale Feature is being deployed to all the hosts in the configuration and as all the components are being deployed.

Figure 18. Installing task bar
Screen cap: Shows configuration progress monitor bar
  1. Upon successful installation of the DB2 pureScale Feature, click Finish. The installation of the DB2 pureScale Feature across all four hosts is complete with a DB2 instance ready for use, as shown in Figure 19.
Figure 19. Setup has completed successfully
Screen cap: Shows setup completed with post-install steps highlighted

DB2 pureScale Feature post-installation steps

With a DB2 pureScale Feature for Enterprise Server Edition instance ready for use, your next step is to create a file system to use for the data and the logs. You can create the file system using the db2cluster command. Complete the following steps from the root user.

  1. Create one file system for data and one file system for logs using the commands shown in Listing 5.
Listing 5. File system commands
#<DB2 Install Path>/bin/db2cluster -cfs -create -filesystem data -disk /dev/hdisk4
#<DB2 Install Path>/bin/db2cluster -cfs -create -filesystem log -disk /dev/hdisk5

The DB2 Installation Path in this example deployment is /opt/IBM/db2/V9.8. The data and log file systems are created by default under /db2fs. They are accessible on all hosts in the DB2 pureScale instance.

  1. Modify the owner of the filesystem to be the DB2 instance owner so the DB2 instance owner has full access to this file system. For the example, db2sdin1 is the instance owner's name, and db2iadm1 is the instance owner's group name, as shown in Listing 6.
Listing 6. Assign filesystem owner
#chown db2sdin1:db2iadm1 /db2fs/data
#chown db2sdin1:db2iadm1 /db2fs/log
  1. Start the DB2 instance by issuing the db2start command, as shown in Listing 7.
Listing 7. db2start command
> db2start
04/19/2010 11:02:08 0 0 SQL1063N DB2START processing was successful.
04/19/2010 11:02:08 1 0 SQL1063N DB2START processing was successful.
SQL1063N DB2START processing was successful.
  1. Check the state of the DB2 pureScale instance at any point using the db2instance command. Tip: You can view the state of a DB2 pureScale cluster using the db2instance -list command.
  2. Create the database and move the logs to the log file system using the commands in Listing 8. Note that you must run the commands from member hosts, not CF hosts.
Listing 8. Move logs to log file system
> db2 create db testdb on /db2fs/data
> db2 update db cfg for testdb using newlogpath /db2fs/log
  1. Catalog the client connections to any active pureScale members, and connect to the PowerHA pureScale server.

Using the DB2 pureScale Feature

The ease of installing and deploying the DB2 pureScale Feature reduces your total cost of ownership. This section offers more details about use-cases that demonstrate the added value of installing the DB2 pureScale Feature.

Adding and removing members

The DB2 pureScale Feature enables you to add members to the configuration quickly and without any data redistribution requirements. The DB2 installation binaries are automatically stored on the IIH, so they don't require access to the original installation media when you are adding members. Add a member by running the command in Listing 9 from the IIH.

Listing 9. Adding a member
db2iupdt -d -add -m ServerX:ServerX-ib0 db2sdin1

Members can be started or quiesced transparently to the application such that the application is unaware a change has occurred.

Automatic workload balancing

With the DB2 pureScale Feature, you can dynamically distribute a workload across all the active members based on the computer utilization characteristics of the different machines. Multi-threaded CLI applications by default have connection-level workload balancing without any changes. This workload balancing can be modified such that it applies at the transaction level instead of the connection level. For multi-threaded Java® applications, set enableSysplexWLB=true in the connection string to take advantage of transaction-level workload balancing.

As additional members are started, clients automatically route to the new member without any interruption of service. Also, members can be stopped without the application knowing.

You can also configure clients to have a preference to which members they should connect to. This feature is referred to as client affinity, and it can be beneficial if a partitioned workload already exists.

Note: To take advantage of DB2 pureScale Feature benefits such as transaction-level workload balancing or client affinity, the minimum client level should be Version 9.7 Fix Pack 1 or the correlating JCC level. To correlate the JCC levels included in the various fix pack levels, see Resources.

Stealth maintenance

At times, it is critical to apply maintenance to a system without negatively impacting the client applications. Stealth maintenance basically allows all transactions on a member to complete and then transparently routes that application to another member. To drain member 1 in the example scenario, you can run the db2stop member 1 quiesce command.

Sometimes a unit of work (UOW) is started for a user session, but the UOW is not committed or rolled back. The db2stop quiesce command waits for that UOW to be completed before stopping that member. You can specify a timeout value such as ten minutes that permits the application ten minutes to complete the UOW and, if after ten minutes the UOW is not completed, the DB2 software automatically forces off that application. Any applications that complete the UOW within the ten minutes are automatically rerouted to the active members. For example, to drain member 1 with a ten minute timeout, run the command db2stop member 1 quiesce 10.

High availability

One of the significant value propositions of the DB2 pureScale Feature is the high-availability characteristics integrated into the architecture. The DB2 pureScale cluster services automatically monitor all necessary resources and restart them as needed. Applications that are connected to a failing member are rerouted to an active member where the application can reissue any failed transactions. Applications connected to a non-failing component are not impacted.

One of the distinguishing factors of the DB2 pureScale Feature compared to most competing technologies is that upon a member failure, no cluster-wide freeze occurs. Only data in the process of being updated on the failing member is temporarily unavailable until recovery is complete. Applications on active members trying to access the locked data on the failing member are briefly in a lock-wait state, and by default they do not receive any errors. The recovery is completed quickly so that data availability through a member failure looks similar to the hypothetical representation in Figure 20.

Figure 20. Typical data availability pattern during member recovery
Graph of data availability over time, with only a slight loss of data availability during recovery (only in-flight updates are unavailable)

Disaster recovery

Many customers require a disaster recovery solution to meet their business continuity requirements. The DB2 pureScale Feature leverages remote disk-mirroring technology and is also designed to work with database replication products. If the entire primary site that is running a DB2 pureScale instance fails, you can leverage the remote site to allow business operations to continue, as shown in Figure 21. For the example scenario, the DB2 pureScale production instance is on site A, which feeds into the storage. With disk mirroring, the DB2 pureScale DR instance site B also feeds into storage. The DB2 pureScale Feature can also leverage the traditional database backup, restore, and roll-forward functions that also enable disaster recovery solutions.

Figure 21. Typical disaster recovery setup
Shows DB2 pureScale production instance sites A and B, with the storage for A mirrored to the storage for B

Conclusion

The DB2 pureScale Feature for Enterprise Server Edition provides a database solution that meets the needs of the most demanding customers. This article described some of the key benefits of using DB2 pureScale Feature. It leverages the PowerHA pureScale server and RDMA technologies, allowing it to scale effectively to meet the growing and dynamic needs of different organizations. You can add members to the DB2 pureScale environment without any impact to existing applications to meet the demands of peak processing times. The DB2 pureScale Feature automatically balances the workload across all DB2 members in the cluster without any changes on the application side, taking full advantage of the additional processing capacity. If a DB2 member fails, applications are automatically routed among the other active members. When the failed member host comes back online, applications are transparently routed to the restarted member.


Appendix A. Configuring and deploying InfiniBand

The DB2 pureScale Feature leverages an InfiniBand network to allow for optimal communication between members and PowerHA pureScale servers. This appendix describes the high level steps of deploying InfiniBand.

  1. Deploy the InfiniBand switch with an active subnet manager from the hardware management console, as shown in Listing 10. You need HMC Version 7, Release 3.5 or later. In the example scenario, a single InfiniBand card is shared across two LPARs. This step enables the InfiniBand resources to be shared by either 25%, 50%, or 75% of the InfiniBand card.
Listing 10. Deploying the InfiniBand switch
chsyscfg -m machine1-8204-E8A-SN0657742 -r -sys -i "hca_bandwidth_capabilities=25/50/75"
chsyscfg -m machine2-8204-E8A-SN0657742 -r -sys -i "hca_bandwidth_capabilities=25/50/75"

In this example, ServerA and ServerB reside on machine1 and ServerC and ServerD reside on machine2. You can instead specify values other than 25%, 50%, or 75%.

  1. Repeat Step 1 for each physical computer.
  2. Complete the following steps from the hardware management console's graphical interface to allocate the InfiniBand resources for each LPAR so that each LPAR is assigned 50%.
    1. Select the checkbox beside the LPAR
    2. Click Configuration > Manage Profiles.
    3. Select the default profile.
    4. Click HCA > Configure.
    5. Assign each LPAR a unique GUID index within each physical computer to distinguish the different LPARs using InfiniBand.
    6. Select Medium (50%) capability as the allocation of the InfiniBand resources for each LPAR, which is the middle setting from Step 1 .
    7. Repeat Steps a-f for each LPAR.
  3. Shut down and reactivate each of the LPARs from the HMC.
  4. Download the required uDAPL and InfiniBand packages as documented in the DB2 pureScale Feature documentation (see Resources), install these on all LPARs, and reboot each LPAR.
  5. Enable InfiniBand on each LPAR by running the smitty icm command and selecting Add an InfiniBand Communication Manager.
  6. Complete the following steps to set up the InfiniBand network interface on each LPAR.
    1. Run the smitty inet command.
    2. Click Change/Modify Characteristics.
    3. Select ib0.
    4. Reboot the LPAR. Note that you must change the Internet address and netmask. For the example scenario, assign 10.1.1.7 on serverA, 10.1.1.8 on serverB, 10.1.1.9 on serverC, and 10.1.1.10 on serverD for the Internet addresses, assign 255.255.255.0 for the netmask on each, and use the defaults for all other settings.
    5. Repeat Steps a-d for each LPAR.
  7. Ensure the /etc/dat.conf file has a format similar to Listing 11 for each LPAR. For the example scenario, make the host channel adapter /dev/iba0 , on port 1 using interface ib0.
Listing 11. Settings in /etc/dat.conf file
hca0 u1.2 nonthreadsafe default 
/usr/lib/libdapl/libdapl.a(shr_64.o) IBM.1.1 "/dev/iba0 
1 ib0" " "
  1. Edit the /etc/hosts file on each LPAR for routing purposes by following the example in Listing 12.
Listing 12. Editing the /etc/hosts file
10.1.1.1	servera-ib0.<domain_name> servera-ib0
10.1.1.2 	serverb-ib0.<domain_name> serverb-ib0
10.1.1.3 	serverc-ib0.<domain_name> serverc-ib0
10.1.1.4 	serverd-ib0.<domain_name> serverd-ib0
  1. Optionally validate the state of the InfiniBand on each LPAR by running the ibstat -v command from a user with root privileges. Verify that port 1 is active and that the link is up. You can also perform a ping test using the addresses and hostnames defined in the /etc/hosts file.

Appendix B. Supported AIX server stacks

The DB2 pureScale Feature currently supports the following POWER6® models:

  • 550
  • 595

The DB2 pureScale Feature currently supports the following POWER7™ models.

  • 750
  • 770
  • 780

Appendix C. Supported storage subsystems

The DB2 pureScale Feature supports a wide range of IBM storage, including the IBM DS3000, DS5000, DS8000®, and XIV® lines of storage, as well as storage provided by others vendors, including EMC, Hitachi, and others. To leverage the DB2 pureScale Feature with rapid fencing that is required for ultra-fast failover times, the disk storage should be validated with SCSI-3 persistent reserve with an underlying clustered file system. See Resources for the storage subsystems validated with SCSI-3 persistent reserve.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Information management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management
ArticleID=490758
ArticleTitle=Deploying the DB2 pureScale Feature
publish-date=05202010