Enable DB2 in OpenStack

Set up the OpenStack environment to support DB2

OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a data center. All resources are managed through a dashboard that gives administrators control while empowering users to provision resources through a web interface. OpenStack supports MySQL, SQLite and PostgreSQL as its default databases — in this article, the author shows you how to quickly enable OpenStack to support DB2®.

Share:

Sheng Bo Hou (sbhou@cn.ibm.com), Software Engineer, IBM

Sheng Bo Hou is a software engineer on the IBM Open Standards and Open Source Team. Since he joined IBM in 2010, his focus has been on cloud computing standards efforts. To align with IBM's support of standards and integrating open source software and development, he is currently contributing to the OpenStack community with development ideas on bug reporting and fixing, blueprint engagements, and more.



05 November 2012

Also available in Chinese Russian Japanese

OpenStack is a great cloud operating system; a developer, administrator, and operator would be smart to have skills pertaining to it in their virtual toolkit. An efficient cloud needs an efficient database and many databases are acceptable. But what if you want to use IBM® DB2 with OpenStack ... it doesn't support it by default.

This short exercise shows how to enable support for DB2 in an OpenStack environment. The six steps include:

  1. Gather the materials to get started.
  2. Install the database access packages.
  3. Add additional supports for DB2.
  4. Use OpenStack scripts.
  5. Configure OpenStack for DB2.
  6. Run OpenStack and DB2.

Let's get started.

Step 1: Gather the materials to get started

OpenStack is built on Python 2.7 and some additional packages within Ubuntu 11.10, the 64-bit environment version. IBM DB2 is the target database you want to be able to support. You need the following to get started:

  • A physical machine.
  • An operating system such as Ubuntu 11.10 or higher; it should be a 64-bit OS.
  • A programming language like Python 2.7+. For Python, run the following commands when the system is ready:
    sudo apt-get install python-software-properties
    sudo apt-get update, sudo apt-get install python-dev libaio-dev
    Install pip: sudo apt-get install python-pip
    Install ez_setup: sudo pip install ez_setup
  • DB2 Version 9.7 or higher is recommended.

Step 2: Install the database access packages

SQLAlchemy is the Python SQL toolkit and object relational mapper that enables SQL for Python application developers through a suite of enterprise-level persistence patterns that are designed for efficient and high-performing database access. SQLAlchemy tries to accommodate the two principles of size and performance and abstraction by treating the database more like a relational algebra engine than just a collection of tables.

The object relational mapper in SQLAlchemy (that lets classes be mapped to the database in open ended, multiple ways) allows the object model and database schema to develop in a cleanly decoupled way.

SQLAlchemy-migrate are schema migration tools. It provides a way to deal with database schema changes in SQLAlchemy projects and a database change repository mechanism which can be used from the command line as well as from inside Python code.

  1. Download SQLAlchemy. Install with sudo pip install SQLAlchemy==0.7.2 (or whichever version you choose).
  2. Download SQLAlchemy-migrate. Since DB2 is not supported, install the patch for DB2, called ibmdb2.patch (see the Download section. Save the patch in the root folder of SQLAlchemy-migrate and run the command -p1 < ibmdb2.patch. Next, install SQLAlchemy-migrate.

Step 3: Add additional supports for DB2

To connect DB2 with Python install the IBM Data Server Driver, ibm_db, and ibm_db_sa. Also, to install OpenStack, you need the authority of the super user, so the necessary variables must be set for the sudoers and global. You need the following drivers and adapters to make OpenStack support DB2.

Download the IBM Data Server Driver for ODBC and CLI from Fix Central; choose the proper version with a file name like ibm_data_server_driver_for_odbc_cli_linuxxXX_vXX.tar.gz. Unzip it.

Make three variables global; in OpenStack, many commands run through sudo, which needs to reserve these three variables:

  • Defaults env_keep+="IBM_DB_DIR"
  • Defaults env_keep+="IBM_DB_LIB"s
  • Defaults env_keep+="LD_LIBRARY_PATH"

Add the code after Defaults env_reset.

Three variables need to be set in the environment. They can be put into the ~/.bashrc file. Add the following lines into ~/.bashrc:

export IBM_DB_DIR=<ODBC and CLI DIR>/clidriver
export IBM_DB_LIB=<ODBC and CLI DIR>/clidriver/lib
export LD_LIBRARY_PATH=<ODBC and CLI DIR>/clidriver/lib
alias sudo='sudo LD_LIBRARY_PATH=/<ODBC and CLI DIR>/clidriver/lib'

Download ibm_db, a Python driver and DBI wrapper for IBM databases, that provides a Python interface for IBM databases.

  1. Go to the directory:
    cd <ibm_db directory>

    .
  2. Run the commands: sudo python setup.py build and sudo python setup.py install.

Download ibm_db_sa: This adapter provides the Python/SQLAlchemy interface to IBM data servers. According to the requirements of SQLAlchemy-migrate and OpenStack code, some modifications have been done on the code. Use the ibm_db_sa.zip in the attachment.

  1. Go to the directory:
    cd <ibm_db_sa directory>

    .
  2. Run the commands sudo python setup.py build and sudo python setup.py install.

Step 4: Use OpenStack scripts

OpenStack writes the code by default supporting MySQL. IBM DB2 has more stringent requirements for fields with nullable (allows datatypes to be set) and index types to be configured, acceptance of UNIQUE constraint (uniquely identifies each record in a database table), etc. So the Python code for SQLAlchemy-migrate needs to be replaced with code supporting IBM DB2's requirements (includes OpenStack's Project Nova/Compute, the cloud computing fabric controller, the main part of an IaaS system).

Note: Since Nova is not properly supporting the requirements according to my testing, I prepared specific Python scripts, 082_essex.py and 096_recreate_dns_domains. You can get them from the Download section. Install the Python script to enable Nova for DB2.

There are two alternatives:

  1. If you install from the source code, back up the one under the directory <nova dir>/nova/db/sqlalchemy/migrate_repo/versions and then copy the two script files into that directory.
  2. If you install by running the command, back up the file under <nova dir*gt;/nova/db/sqlalchemy/migrate_repo/versions and then copy the two script files into that directory.

I have already made a bug report for this issue that you can follow:

Some columns should be defined nullable=False,since they are unique in the table. 
For example, address in virtual_interfaces, host in aggregate_hosts, name in 
aggregates, etc. MySQL is OK, but databases with more strict column creation can 
yield errors, if it is not strictly defined.

Due to the same reason, method register_models in nova/db/sqlalchemy/models.py 
does not work for MYSQL. uuid in instance must be define unique and not null.

Step 5: Configure OpenStack for DB2

The database connection has to be set to the proper address for IBM DB2; then all the database tables should be initialized. This step shows you how.

For the Nova component (the cloud fabric controller):

  1. Create a table named nova; this table is particular. You need to make the page size 8K, PAGESIZE=8K, instead of the default 4K. In SQL, the command looks like this: CREATE DATABASE nova AUTOMATIC STORAGE YES ON '/home/db2inst1' DBPATH ON '/home/db2inst1' USING CODESET UTF-8 TERRITORY CN COLLATE USING SYSTEM PAGESIZE 8192.
  2. Modify the sql_connection or connection in the /etc/nova/nova.conf file, setting it to ibmdb://name:password@address:50000/nova.

For the Glance component (the Image Service component that provides discovery, registration, and delivery services for virtual disk images):

  1. Create a table named glance.
  2. Modify the sql_connection in the /etc/glance/glance-registry.conf file, setting it to ibmdb://name:password@address:50000/glance.

For the Keystone component (the component that provides identity, token, catalog, and policy services):

  1. Create a table named keystone.
  2. Modify the sql_connection or connection in the /etc/keystone/keystone.conf file, setting it to ibmdb://name:password@address:50000/keystone.

For the Cinder component (the component that provides block storage as a service):

  1. Create a table named cinder.
  2. Modify the sql_connection or connection in the /etc/cinder/cinder.conf file, setting it to ibm_db_sa://name:password@address:50000/cinder.

Step 6: Run OpenStack with DB2

After you have completed these steps, you can install your OpenStack (for example, by using DevStack, a shell script that makes it relatively easy to build a complete OpenStack development environment). After that, you can run your new OpenStack dev environment with DB2.

Note: If you build with DevStack, change stack.sh file by modifying this line:

screen_it n-cpu "cd $NOVA_DIR && sg libvirtd $NOVA_DIR/bin/nova-compute"

To this:

screen_it n-cpu "cd $NOVA_DIR && $NOVA_DIR/bin/nova-compute"

Or you will receive an error similar to this:

ImportError: libdb2.so.1: cannot open shared object file: No such file or directory

However, make sure your current user is in the group libvirtd. You can add it by editing /etc/group.

One last note: OpenStack is an evolving platform which means that the scripts for each of the project components are likely to need updating from time to time. Check with the OpenStack site for any changes to the components on a regular basis.


Download

DescriptionNameSize
Script filesscript_and_patches.zip42KB

Resources

Learn

Get products and technologies

Discuss

  • Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • developerWorks Labs

    Experiment with new directions in software development.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing, Information Management, Open source
ArticleID=844205
ArticleTitle=Enable DB2 in OpenStack
publish-date=11052012