Configure Linux shared disks for Informix Dynamic Server MACH 11

Explained by example

Find out how to configure true shared disks for IBM® Informix® Dynamic Server (IDS) Version 11.50. This article contains instructions on how to get an IDS Shared Disk (SD) secondary server up and running on Ubuntu Linux®. You can also easily adapt the instructions for other Linux distributions.

Andreas Breitfeld (abreitfe@de.ibm.com), IBM Informix Development, IBM Deutschland GmbH

Andreas Breitfeld photoAndreas Breitfeld is a software developer who has been working on IBM Informix Database Server for several years. He has been involved at various stages in porting and development of IDS.



05 March 2009

Also available in Chinese Russian Japanese Portuguese

Introduction

Starting with version 11.50.xC1, IBM Informix Dynamic Server (IDS) includes a group of high availability and clustering features known as Multi-node Active Cluster for High Availability (MACH 11). The support of Shared Disk (SD) secondary servers is one of these features. An SD secondary server shares disks with the primary server, but does not maintain a copy of the physical database. This article describes how you can build a sample configuration of true shared disks and SD secondary servers on Ubuntu Linux. By following this procedure, you should be able to customize your own shared disks and SD secondary servers on Ubuntu Linux.

The article gives you background information about the sample configuration, a description of the requirements for building the configuration, and instructions for how to build the configuration.

.

Background

The sample configuration that is described in this article uses an Internet SCSI (iSCSI) Storage Area Network (SAN) to provide shared disks for IDS. Following is a brief explanation as to why iSCSI was chosen over other methods.

IDS MACH 11 functionality requires concurrent access of multiple IDS instances to shared chunks. This allows you to run the IDS instances on different machines. For chunks stored in a file system (cooked devices), the solution is to use cluster file systems (for example, GFS and OCFS). For chunks stored on disk devices (raw devices), Storage Area Networks are offered (for example, Fibre Channel or iSCSI). Because cluster file systems cause unnecessary performance and administration overhead for operation of IDS, this article uses SAN as the means for storage of chunks.

Fibre Channel (FC) is a well known high performance technology for the implementation of a SAN. With FC, bandwidths between 100MB/s and 400MB/s per FC link are common, and bandwidths of of 800MB/s are already possible. However, the implementation of a FC SAN is relatively complex and expensive.

For lower bandwidth requirement applications, you can implement a SAN with Internet SCSI (iSCSI). With iSCSI, SCSI commands are transmitted on a standard 1000BASE-T (Gigabit Ethernet) network, which provides a theoretical bandwidth of 125MB/s per Gigabit Ethernet link. Greater bandwidth is also available with 10 Gigabit Ethernet. The implementation of a iSCSI SAN is relatively simple and cheap because it is based on common standard components. For test and development environments, this simplicity and low cost are important advantages.


Requirements

The following figure shows the structure of the network for the sample configuration.

Figure 1. Sample configuration network structure
sample configuration network structure

In the sample configuration, Ubuntu 8.04 LTS Server Edition is installed on three standard PCs (machines). The machine with multiple disks (labeled Piero) works as a dedicated disk array. The other two machines are both running IDS 11.50.UC2DE; one is the primary server (labeled mach1) and the other is the SD secondary server (labeled mach2).

The existing Gigabit Ethernet adapters are connected to a Gigabit Ethernet switch and have private IP addresses. This private network is reserved exclusively for iSCSI data (SAN). The network components must also support jumbo frames (MTU=9000). In Ubuntu you can configure this by adding the option mtu 9000 for the Gigabit Ethernet Adapter of the SAN in the /etc/network/interfaces file. You can use the ifconfig command to see the current setting for MTU.

Each machine has an additional Ethernet adapter that connects it to the LAN.


Configuring iSCSI

This section describes how to configure iSCSI for the sample configuration.

Note: For Linux distributions other than Ubuntu 8.04 (such as, Red Hat or SUSE), you should be able to build a similar configuration. In particular, Red Hat Enterprise Linux 5 provides the scsi-target-utils package (based on the tgt project) for setting up a iSCSI target.

Install iscsitarget on target machine

Within a SAN, the machine that provides disks is called the target. Install the Ubuntu package iscsitarget (iSCSI Enterprise Target userland tools) on the target machine in your SAN.

Configure the iSCSI Enterprise Target Daemon

In the /etc/ietd.conf file (iSCSI Enterprise Target Daemon) of the target machine, specify which disk or partition can be shared. For example:

Target iqn.2008-07.com.ibm.munich.target:shared.disk1
	Lun 0 Path=/dev/sda5,Type=blockio

In the above example, iqn.2008-07.com.ibm.munich.target:shared.disk1 is the symbolic target name of the disk. The type blockio sets the disk to direct block I/O mode without page cache.

After making changes to /etc/ietd.conf, you must restart the iSCSI Enterprise Target Daemon to activate the changes:

/etc/init.d/iscsitarget restart

Alternatively, you could use the ietadm (iSCSI Enterprise Target Administration Utility) command to change the configuration dynamically without restarting.

You can see the active configuration in the /proc/net/iet/volume and /proc/net/iet/session files.

Install open-iscsi on initiator machines

A machine that accesses the disks in a SAN is called an initiator. Install the Ubuntu package open-iscsi (High performance, transport independent iSCSI implementation) on each of your initiator machines.

For the sample configuration, the default values for iscsid (Open-iSCSI daemon) are not changed. The iscsid configuration is defined in the /etc/iscsi/iscsid.conf file.

Define initiator access disks

Use the iscsiadm command to define which disks the initiator can log into.

Discover targets on the IP address of the target machine:

iscsiadm --mode discovery --type sendtargets --portal 192.168.0.100

List the discovered node records:

iscsiadm --mode node

Set the startup values for the node records to automatic. (The default, which is set in /etc/iscsi/iscsid.conf, is manual). You can set the startup value for individual node records or set the startup value for all node records. Remember to set the startup values after each discovery.

Here is an example of setting the startup value for an individual node:

iscsiadm -m node --targetname iqn.2008-07.com.ibm.munich.target:shared.disk1 \ 
    --op=update --name=node.startup --value=automatic --portal 192.168.0.100

Here is an example of setting the startup value for all node records:

iscsiadm -m node --op=update --name=node.startup --value=automatic \ 
    --portal 192.168.0.100

Login to all targets with a startup value of automatic (after a reboot, the login happens automatically):

iscsiadm --mode node --portal 192.168.0.100 --loginall automatic

Define a persistent device path

At this point, the new disks are visible for the kernel as usual, but the mapping of devices and target names is indistinguishable in /proc/partitions. Therefore, you should use a persistent device path. This path is generated automatically by udev (dynamic device management). For example:

/dev/disk/by-path/\
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0

Set up your partitions as usual. For example:

fdisk /dev/disk/by-path/\
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0

Partition 1 has the path:

/dev/disk/by-path/\
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0-part1

If logins to the same target name happen from multiple initiators, then a shared disk partition is provided as a block device with the same path.


Configuring IDS

This section describes how to configure IDS for the sample configuration.

Install Ubuntu packages

Install the following packages on the Ubuntu 8.04 operating system of both the machine that will be your primary server (mach1) and the machine that will be your SD secondary server (mach2):

  • bc
  • gcc-3.3-base
  • libaio1
  • libstdc++5
  • pdksh
  • rpm

Create and activate permissions

IDS requires specific permissions for chunks. In this step, you use udev to set these permissions for shared disks and local disks.

Create a new rules file named /etc/udev/rules.d/99-informix-permissions.rules that defines permissions for your shared and local disks. For example:

###
# iSCSI shared disks by-path
ENV{DEVTYPE}=="disk", IMPORT{program}="path_id %p"
ENV{DEVTYPE}=="partition", 
ENV{ID_PATH}=="ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk*", OWNER="informix", GROUP="informix", MODE="0660"

# local disks
KERNEL=="sda7",         OWNER="informix", GROUP="informix", MODE="0660"
###

You must activate the rules either by using the udevtrigger command or by rebooting.

It is also recommended that you create simpler paths for the devices by using symbolic links. For example:

/home/informix/chunks/shared.disk1 -> /dev/disk/by-path/\ 
ip-192.168.0.100:3260-iscsi-iqn.2008-07.com.ibm.munich.target:\ 
shared.disk1-lun-0-part1

Create and configure IDS instances

Create an instance of IDS on the machine that will be your primary server (mach1) and the machine that will be your SD secondary server (mach2).

For both machines, set the ROOTPATH parameter of the ONCONFIG file to the same shared disk:

ROOTPATH /home/informix/chunks/shared.disk1

Also, update the INFORMIXSQLHOSTS file for both machines to include entries for communications between the two:

demo_on         onsoctcp    mach1    9088
demo_on_sds1    onsoctcp    mach2    9088

For the SD secondary server machine only, add the following entries to the ONCONFIG file:

SDS_ENABLE 1
SDS_TEMPDBS sds1temp,/home/informix/chunks/sds1temp,2,0,2048
SDS_PAGING /home/informix/chunks/sds1page1,/home/informix/chunks/sds1page2

Start IDS instances

Initialize the IDS instance on your primary server (mach1) with the command:

oninit -iv

After initialization of IDS on the mach1 machine, define it as the primary server:

onmode -d set SDS primary demo_on

Start the IDS instance on your SD secondary server (mach2) with the command:

oninit -v

You can check the status of each instance with the command:

onmode -g sds

Conclusion

The configuration described in this article showed remarkably good performance with IDS. Important differences between a local disk and a shared disk could not be measured while running a simplified TPC-B benchmark.

This article described the setup and configuration of iSCSI shared disks on a specific version of Ubuntu Linux, to help you get a IDS SDS up and running. You should be able to use this procedure as a guide and adapt it for other distributions of the Linux operating system.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Information management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management, Linux
ArticleID=374189
ArticleTitle=Configure Linux shared disks for Informix Dynamic Server MACH 11
publish-date=03052009