Introduction

Starting with version 11.50.xC1, IBM Informix Dynamic Server (IDS) includes a group of high availability and clustering features known as Multi-node Active Cluster for High Availability (MACH 11). The support of Shared Disk (SD) secondary servers is one of these features. An SD secondary server shares disks with the primary server, but does not maintain a copy of the physical database. This article describes how you can build a sample configuration of true shared disks and SD secondary servers on Ubuntu Linux. By following this procedure, you should be able to customize your own shared disks and SD secondary servers on Ubuntu Linux.

The article gives you background information about the sample configuration, a description of the requirements for building the configuration, and instructions for how to build the configuration.

.

Background

The sample configuration that is described in this article uses an Internet SCSI (iSCSI) Storage Area Network (SAN) to provide shared disks for IDS. Following is a brief explanation as to why iSCSI was chosen over other methods.

IDS MACH 11 functionality requires concurrent access of multiple IDS instances to shared chunks. This allows you to run the IDS instances on different machines. For chunks stored in a file system (cooked devices), the solution is to use cluster file systems (for example, GFS and OCFS). For chunks stored on disk devices (raw devices), Storage Area Networks are offered (for example, Fibre Channel or iSCSI). Because cluster file systems cause unnecessary performance and administration overhead for operation of IDS, this article uses SAN as the means for storage of chunks.

Fibre Channel (FC) is a well known high performance technology for the implementation of a SAN. With FC, bandwidths between 100MB/s and 400MB/s per FC link are common, and bandwidths of of 800MB/s are already possible. However, the implementation of a FC SAN is relatively complex and expensive.

For lower bandwidth requirement applications, you can implement a SAN with Internet SCSI (iSCSI). With iSCSI, SCSI commands are transmitted on a standard 1000BASE-T (Gigabit Ethernet) network, which provides a theoretical bandwidth of 125MB/s per Gigabit Ethernet link. Greater bandwidth is also available with 10 Gigabit Ethernet. The implementation of a iSCSI SAN is relatively simple and cheap because it is based on common standard components. For test and development environments, this simplicity and low cost are important advantages.

Back to top

Requirements

The following figure shows the structure of the network for the sample configuration.

In the sample configuration, Ubuntu 8.04 LTS Server Edition is installed on three standard PCs (machines). The machine with multiple disks (labeled Piero) works as a dedicated disk array. The other two machines are both running IDS 11.50.UC2DE; one is the primary server (labeled mach1) and the other is the SD secondary server (labeled mach2).

The existing Gigabit Ethernet adapters are connected to a Gigabit Ethernet switch and have private IP addresses. This private network is reserved exclusively for iSCSI data (SAN). The network components must also support jumbo frames (MTU=9000). In Ubuntu you can configure this by adding the option mtu 9000 for the Gigabit Ethernet Adapter of the SAN in the /etc/network/interfaces file. You can use the ifconfig command to see the current setting for MTU.

Each machine has an additional Ethernet adapter that connects it to the LAN.

Back to top

Configuring iSCSI

This section describes how to configure iSCSI for the sample configuration.

Note: For Linux distributions other than Ubuntu 8.04 (such as, Red Hat or SUSE), you should be able to build a similar configuration. In particular, Red Hat Enterprise Linux 5 provides the scsi-target-utils package (based on the tgt project) for setting up a iSCSI target.

Install iscsitarget on target machine

Within a SAN, the machine that provides disks is called the target. Install the Ubuntu package iscsitarget (iSCSI Enterprise Target userland tools) on the target machine in your SAN.

Configure the iSCSI Enterprise Target Daemon

In the /etc/ietd.conf file (iSCSI Enterprise Target Daemon) of the target machine, specify which disk or partition can be shared. For example:

Target iqn.2008-07.com.ibm.munich.target:shared.disk1
	Lun 0 Path=/dev/sda5,Type=blockio
			

In the above example, iqn.2008-07.com.ibm.munich.target:shared.disk1 is the symbolic target name of the disk. The type blockio sets the disk to direct block I/O mode without page cache.

After making changes to /etc/ietd.conf, you must restart the iSCSI Enterprise Target Daemon to activate the changes:

/etc/init.d/iscsitarget restart
			

Alternatively, you could use the ietadm (iSCSI Enterprise Target Administration Utility) command to change the configuration dynamically without restarting.

You can see the active configuration in the /proc/net/iet/volume and /proc/net/iet/session files.

Install open-iscsi on initiator machines

A machine that accesses the disks in a SAN is called an initiator. Install the Ubuntu package open-iscsi (High performance, transport independent iSCSI implementation) on each of your initiator machines.

For the sample configuration, the default values for iscsid (Open-iSCSI daemon) are not changed. The iscsid configuration is defined in the /etc/iscsi/iscsid.conf file.

Define initiator access disks

Use the iscsiadm command to define which disks the initiator can log into.

Discover targets on the IP address of the target machine:

iscsiadm --mode discovery --type sendtargets --portal 192.168.0.100
			

List the discovered node records:

iscsiadm --mode node
			

Set the startup values for the node records to automatic. (The default, which is set in /etc/iscsi/iscsid.conf, is manual). You can set the startup value for individual node records or set the startup value for all node records. Remember to set the startup values after each discovery.

Here is an example of setting the startup value for an individual node:

iscsiadm -m node --targetname iqn.2008-07.com.ibm.munich.target:shared.disk1 \ 
    --op=update --name=node.startup --value=automatic --portal 192.168.0.100
			

Here is an example of setting the startup value for all node records:

iscsiadm -m node --op=update --name=node.startup --value=automatic \ 
    --portal 192.168.0.100
			

Login to all targets with a startup value of automatic (after a reboot, the login happens automatically):

iscsiadm --mode node --portal 192.168.0.100 --loginall automatic
			

Define a persistent device path

At this point, the new disks are visible for the kernel as usual, but the mapping of devices and target names is indistinguishable in /proc/partitions. Therefore, you should use a persistent device path. This path is generated automatically by udev (dynamic device management). For example:

/dev/disk/by-path/\
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0
			

Set up your partitions as usual. For example:

fdisk /dev/disk/by-path/\
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0
			

Partition 1 has the path:

/dev/disk/by-path/\
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0-part1
			

If logins to the same target name happen from multiple initiators, then a shared disk partition is provided as a block device with the same path.

Back to top

Configuring IDS

This section describes how to configure IDS for the sample configuration.

Install Ubuntu packages

Install the following packages on the Ubuntu 8.04 operating system of both the machine that will be your primary server (mach1) and the machine that will be your SD secondary server (mach2):

• bc
• gcc-3.3-base
• libaio1
• libstdc++5
• pdksh
• rpm

Create and activate permissions

IDS requires specific permissions for chunks. In this step, you use udev to set these permissions for shared disks and local disks.

Create a new rules file named /etc/udev/rules.d/99-informix-permissions.rules that defines permissions for your shared and local disks. For example:

###
# iSCSI shared disks by-path
ENV{DEVTYPE}=="disk", IMPORT{program}="path_id %p"
ENV{DEVTYPE}=="partition", 
ENV{ID_PATH}=="ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk*", OWNER="informix", GROUP="informix", MODE="0660"

# local disks
KERNEL=="sda7",         OWNER="informix", GROUP="informix", MODE="0660"
###
			

You must activate the rules either by using the udevtrigger command or by rebooting.

It is also recommended that you create simpler paths for the devices by using symbolic links. For example:

/home/informix/chunks/shared.disk1 -> /dev/disk/by-path/\ 
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0-part1
			

Create and configure IDS instances

Create an instance of IDS on the machine that will be your primary server (mach1) and the machine that will be your SD secondary server (mach2).

For both machines, set the ROOTPATH parameter of the ONCONFIG file to the same shared disk:

ROOTPATH /home/informix/chunks/shared.disk1
			

Also, update the INFORMIXSQLHOSTS file for both machines to include entries for communications between the two:

demo_on         onsoctcp    mach1    9088
demo_on_sds1    onsoctcp    mach2    9088
			

For the SD secondary server machine only, add the following entries to the ONCONFIG file:

SDS_ENABLE 1
SDS_TEMPDBS sds1temp,/home/informix/chunks/sds1temp,2,0,2048
SDS_PAGING /home/informix/chunks/sds1page1,/home/informix/chunks/sds1page2
			

Start IDS instances

Initialize the IDS instance on your primary server (mach1) with the command:

oninit -iv
                                    

After initialization of IDS on the mach1 machine, define it as the primary server:

onmode -d set SDS primary demo_on
			  

Start the IDS instance on your SD secondary server (mach2) with the command:

oninit -v
                                    

You can check the status of each instance with the command:

onmode -g sds
			

Back to top

Conclusion

The configuration described in this article showed remarkably good performance with IDS. Important differences between a local disk and a shared disk could not be measured while running a simplified TPC-B benchmark.

This article described the setup and configuration of iSCSI shared disks on a specific version of Ubuntu Linux, to help you get a IDS SDS up and running. You should be able to use this procedure as a guide and adapt it for other distributions of the Linux operating system.

Resources

Learn

• Find out more about The iSCSI Enterprise Target Project.
  
  
• Find out more about the Open-iSCSI project.
  
  
• The IBM Informix Dynamic Server V11.50 Information Center contains documentation for IDS.
  
  
• developerWorks Information Management zone: Learn more about Information Management. Find technical documentation, how-to articles, education, downloads, product information, and more.
  
  
• Stay current with developerWorks technical events and webcasts.
  
  

Get products and technologies

• Download Informix Dynamic Server Developers Edition.
  
  

Discuss

• Participate in the discussion forum.
  
  

About the author

Andreas Breitfeld is a software developer who has been working on IBM Informix Database Server for several years. He has been involved at various stages in porting and development of IDS.

Comments

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction
• Background
• Requirements
• Configuring iSCSI
• Configuring IDS
• Conclusion
• Resources
• About the author
• Comments

Next steps from IBM

• Try: Informix Dynamic Server Enterprise Edition
• Demo: Use DB2 First Steps and the DB2 Control Center
• Webcast: A smarter way to successful SOA - Information as a trusted strategic asset
• Briefing: Information on Demand
• Buy: Informix Dynamic Server Enterprise Edition

My developerWorks community

Dig deeper into Information Management on developerWorks

DB2 pureScale

See how true transparent scaling architecture can be achieved on distributed platforms

Special offers