IBM Support

Tutorial: Adding a database instance to an AWS ROSA cluster using the Db2 Operator

General Page

This tutorial walks you through a database deployment, using the Db2 Operator, on the Red Hat OpenShift Service for the Amazon Web Service (ROSA). There are many configuration options available to you, based on your size and performance requirements. In this tutorial, you'll be provided with options to create single-partition instances of either Db2 or Db2 Warehouse, or a multi-partition Db2 Warehouse instance.

Objective

Completing the tutorial gives you a working Db2 or Db2 Warehouse instance on the AWS cloud. Steps include:
  • Setting up an AWS account.
  • Configuring ROSA with the needed prerequisites.
  • Creating a ROSA cluster.
  • Adding nodes to your cluster.
  • Setting up block storage for your cluster.
  • Setting up shared file storage for your cluster.
  • Deploying the database to your ROSA cluster.

Overview of a Db2/Db2 Warehouse on AWS configuration

Deploying a Db2 or Db2 Warehouse database on AWS can be summarized as follows:

  1. Your system administrator creates an AWS account and chooses a managed service platform for your database instance. You can run your database on the following AWS managed services:
    • Db2 on the Red Hat OpenShift Service on AWS (ROSA).
    • Amazon Elastic Kubernetes Service (EKS).
  2. Using a series of command line tools, your administrator creates an AWS cluster based on your specific requirements. An EC2 instance is selected and a file system and storage class are created.
  3. Your administrator then runs the Db2 Operator and deploys your database instance to your AWS cluster.

Once your database instance is deployed, users can connect to a database in the instance in much the same way as they connect to an on-premise data source.

The flowchart below shows how users interact with the database instance when it is hosted on AWS:

Db2 and Db2 Warehouse on AWs

Choosing an Amazon EC2 instance type

Use the information in the following tables to determine the size requirements of your Db2 or Db2 Warehouse instance. Based on your selection, go to the list of Amazon EC2 instance types to find the AWS instance type that's right for your database deployment. In addition to the sizing values shown in the tables, the EC2 instance type list shows other factors to consider, such as cost and region.

For OTLP workloads on single-partition Db2 environments, choose an instance from the General Purpose or Storage Optimized categories.

Table 1 provides sizing guidelines for small, medium, and large Db2 OLTP systems.
 

Table 1. Sizing guidelines for Db2 on AWS systems

Size CPUs Memory (GB) AWS Instance Type
(Suggested)
Description
Small 8 24
  • m5.xlarge
  • m5a.xlarge
  • m5n.xlarge
  • m5zn.xlarge
  • m4.xlarge
  • Entry level, departmental OLTP workloads.
  • Five concurrent connections.
  • 500 GB of data and logs.
Medium 16 128
  • r5.8xlarge
  • r5a.8xlarge
  • r5b.8xlarge
  • r5n.8xlarge
  • r4.8xlarge
  • Mid-range, line of business OLTP workloads.
  • 1.4 TB of data and logs.
Large 49 1001
  • r6g.16xlarge
  • r6i.16xlarge
  • r5.16xlarge
  • r5a.16xlarge
  • r5b.16xlarge
  • r5n.16xlarge
  • High end Enterprise OLTP workloads.
  • 11 TB of data and logs.

For Db2 Warehouse workloads on single-partition or multi-partition environments, choose an instance from the Memory Optimized  category.

Table 2 provides sizing guidelines for small, medium, and large single partition Db2 Warehouse on AWS systems.

Table 2. Sizing guidelines for a single partition Db2 Warehouse on AWS system

Size CPUs Memory (GB) AWS Instance Type
(Suggested)
Description
Small 7 98
  • r6g.4xlarge
  • r6i.4xlarge,
  • r5.4xlarge
  • r5a.4xlarge
  • r5b.4xlarge
  • r5n.4xlarge
  • 2 TB uncompressed data.
  • 500 GB storage.
Medium 15 226
  • r6g.8xlarge
  • r6i.8xlarge
  • r5.8xlarge
  • r5a.8xlarge
  • r5b.8xlarge
  • r5n.8xlarge
  • r4.8xlarge
  • 4 TB uncompressed data.
  • 1 TB storage.
Large 31 482
  • r6g.16xlarge
  • r6i.16xlarge
  • r5.16xlarge
  • r5a.16xlarge
  • r5b.16xlarge
  • r5n.16xlarge
  • 8 TB uncompressed data.
  • 2 TB storage.
For more information on Db2 single-partition environments, see Single database partition with multiple processors .
 

Table 3 provides sizing guidelines for small, medium, and large multi-partition Db2 Warehouse on AWS systems.

Table 3. Sizing guidelines for a multi-partition Db2 Warehouse on AWS system

Size CPUs Memory (GB) AWS Instance Type
(Suggested)
Description
Small 39 610
  • r6g.4xlarge
  • r6i.4xlarge
  • r5.4xlarge
  • r5a.4xlarge
  • r5b.4xlarge
  • r5n.4xlarge
  • 20 TB of uncompressed data. This sizing estimate is for the entire Db2 deployment (all Db2 Warehouse database partitions).
  • 4 to 32 vCPUs per Db2 Warehouse database partition.
  • Use a memory (in GB) to vCPU ratio of 8:1 to 32:1, with 16:1 or higher being the optimal range.
  • Note: The optimal number of Db2 Warehouse database partitions per OpenShift worker depends on the memory-to-core ratio as well as the ability to scale-out or provide fail-over.
Medium 77 1201
  • r6g.8xlarge
  • r6i.8xlarge
  • r5.8xlarge
  • r5a.8xlarge
  • r5b.8xlarge
  • r5n.8xlarge
  • r4.8xlarge
40 TB of uncompressed data.
Large 153 2406
  • r6g.16xlarge
  • r6i.16xlarge
  • r5.16xlarge
  • r5a.16xlarge
  • r5b.16xlarge
  • r5n.16xlarge
80 TB of uncompressed data.
For more information on Db2 multi-partition environments, see Database partitions with one processor.
 

Choosing cloud storage

When choosing cloud storage options for your Db2 on AWS configuration, consider the following points:
  • For database storage, log storage, and use of temporary table spaces, use a block storage solution.
  • For metadata storage and backup storage, use a shared file storage solution.

Amazon provides block storage (EBS) and shared file storage (EKS) options for your Db2 deployment. The following diagram shows how storage is distributed in a Db2 OLTP formation:

Db2 OLTP cluster formation detail
 The following diagram show how storage is distributed in a Db2 Warehouse formation:
Db2 Warehouse formation

Environment

Before you start the configuration, you need to set some environment variables locally that are used for this tutorial. These variables are:
  • ACCOUNT_ID: Your 12-digit AWS account ID (for example, 001234567890).
  • CLUSTER: The name you use for your EKS Cluster. Use rosa-db2-demo.
  • REGION: The region where your AWS instances are being deployed. For example, us-east-2.
  • INSTANCE_TYPE: The kind of instance to select for deploying workloads:
    • For For single-partition Db2 instances, use m5.xlarge.
    • For single-partition Db2 Warehouse instances, use r5.8xlarge.
    • For multi-partition Db2 Warehouse instances, use r5.16xlarge.
  • MIN_NODES: The minimum number of instances for the EKS cluster:
    • For single-partition Db2 instances, use 1.
    • For single-partition Db2 Warehouse instances, use 1.
    • For multi-partition Db2 Warehouse instances, use 6.
  • MAX_NODES: The maximum number of instances for the EKS cluster.
    • For single-partition Db2 instances, use 1.
    • For single-partition Db2 Warehouse instances, use 1.
    • For multi-partition Db2 Warehouse instances, use 6.
  • NAMESPACE: The namespace where your database instance will be deployed. Use db2u.
You also need to install three command-line tools locally. These are required to complete the tutorial:
  • AWS CLI: An open source tool for communicating with an AWS service, directly from your OS command line, or from a remote terminal program.  This tool requires some post-install configuration.
  • ROSA CLI: A Red Hat tool used to create, update, manage, and remove your ROSA cluster and resources.
  • OpenShift CLI: A Red Hat tool used to create and manage RHOS projects container platform projects.

Create an AWS account

Before you begin to create your ROSA cluster on AWS, you need to have an AWS account.
Procedure
  1. From a web browser, go to https://portal.aws.amazon.com/billing/signup .
  2. Follow the online instructions.
    NOTE: You will be contacted by phone and be required to enter a verification code on your phone keypad.

Configure ROSA with the needed AWS prerequisites

There are a number of access, support, and security prerequisites that you must meet before proceeding with the creation of your ROSA cluster on AWS.
Procedure
  1. From the AWS CLI, run the command aws configure. Use your IAM user account and access key.  ROSA does not support configuration using the root user account.

  2. Verify the AWS configuration: 

    aws sts get-caller-identity

    A successful configuration returns a table, listing the account, userid, and ARN values.

  3. Confirm that the Elastic Load Balancing role exists:

    aws iam get-role --role-name "AWSServiceRoleForElasticLoadBalancing" || aws iam create-service-linked-role 
    --aws-service-name "elasticloadbalancing.amazonaws.com

    NOTE: Running the command creates the role if it does not already exist.

  4. Log in using the ROSA CLI:

    rosa login --token=<token>

    where token can be retrieved from the OpenShift console.

  5. Confirm that the configuration is done successfully:

    rosa whoami
  6. Confirm that you have enough quota on AWS for the region to which ROSA was configured:

    rosa verify quota 
  7. Verify that your RHOS client is using ROSA:

    rosa verify openshift-client
  8. From the OpenShift console, attach the ServiceQuotaFullAccess for the IAM user - This can be done from the console - https://aws.amazon.com/iam/ or command line if you're familiar with it.

For more information, see AWS prerequisites for ROSA with STS.

NOTE: The Amazon Security Token Service (STS) provides enhanced security and is the recommended access management method for installing and interacting with clusters on ROSA.

Create a ROSA cluster

Once the above prerequisites are completed, you can proceed with creating a ROSA cluster.
Procedure
  1. If this is your first time deploying ROSA in this account, create the account-wide roles:  

    rosa create account-roles --mode auto –yes
  2. Run the following command to begin the ROSA cluster creation in interactive mode:

    rosa create cluster --cluster-name ${CLUSTER} --sts --mode auto --yes --interactive
    To configure interactive mode options, follow the Interactive cluster creation mode reference.
  3. Check the status of your cluster: 

    rosa describe cluster --cluster ${CLUSTER}
    It takes around 30 - 40 minutes to complete this setup. The cluster is ready when you see the cluster listed as Ready in the description. Any errors will also appear in the description of the cluster. 
  4. Create an admin user for your cluster: 

    rosa create admin --cluster=${CLUSTER}
  5. To obtain the console URL, run:

    rosa describe cluster --cluster=${CLUSTER} | grep -i Console
​​​​​​

Configure your ROSA cluster for shared file storage

Shared file storage is the best option for your Db2 metadata/backup storage, log storage, and uses of temporary table spaces. To make use of shared file storage, you create an Amazon Elastic File System (EFS) storage class for your ROSA cluster. Creating shared file storage involves the following steps:

  • Installing and setting up AWS EFS CSI Driver Operator.
  • Creating and configuring an EFS volume.
  • Creating a storage class for your EFS volume.

Install and set up AWS EFS CSI Driver Operator

Db2U supports version 4.10+ of the AWS EFS CSI Driver Operator, which is not installed in OpenShift Container Platform by default. Use the following steps to install the AWS EFS CSI Driver Operator in your cluster:

Procedure

  1. Log in to the OpenShift Container Web Console.
  2. Click Operators → OperatorHub.
  3. Search for "AWS EFS CSI Driver Operator" in the filter box.
  4. Click Install on the AWS EFS CSI Driver Operator page. 
    NOTE: Ensure that:
    • AWS EFS CSI Driver Operator Version is 4.10+.
    • All namespaces on the cluster (default) is selected.
    • Installed namespace is set to openshift-cluster-csi-drivers.
  5. Click Install administration → CustomResourceDefinitions.
  6. Search for ClusterCSIDriver in the filter box.
  7. On the Instances tab, click Create ClusterCSIDriver and enter the following YAML:
    apiVersion: operator.openshift.io/v1
    kind: ClusterCSIDriver
    metadata:
        name: efs.csi.aws.com
    spec:
      managementState: Managed
  8. Click Create.
  9. Wait for the EFS pods in the openshift-cluster-csi-drivers to appear.
    EXAMPLE: 
    users-MacBook-Pro:db2u user$ oc get pods -n openshift-cluster-csi-drivers | grep -i efs
    aws-efs-csi-driver-controller-84bf85d9df-86ms5   4/4     Running   0          3d14h
    aws-efs-csi-driver-controller-84bf85d9df-c7k89   4/4     Running   0          3d14h
    aws-efs-csi-driver-node-cc4gc                    3/3     Running   0          3d14h
    aws-efs-csi-driver-node-hnh97                    3/3     Running   0          3d14h
    aws-efs-csi-driver-node-sc8ln                    3/3     Running   0          3d14h
    aws-efs-csi-driver-node-sq5qg                    3/3     Running   0          3d14h
    aws-efs-csi-driver-node-w6j5c                    3/3     Running   0          3d14h
    aws-efs-csi-driver-node-wz5kp                    3/3     Running   0          3d14h
    aws-efs-csi-driver-node-xcqxx                    3/3     Running   0          3d14h
    aws-efs-csi-driver-node-zb28n                    3/3     Running   0          3d14h
    aws-efs-csi-driver-operator-76db8f8d97-jh795     1/1     Running   0          3d18h
    NOTE: If the aws-efs-csi-driver-controller pod is stuck in container-creating because aws-efs-cloud-credentials is not getting created, enter the following to create a secret: 
    apiVersion: v1
    kind: Secret
    metadata:
      namespace: openshift-cluster-csi-drivers
      name: aws-efs-cloud-credentials
    stringData:
      aws_access_key_id: <access key id>
      aws_secret_access_key: <secret access key>
    Replace <access key id> and <secret key id> with the user ID of the IAM user who created the ROSA cluster.

Create and configure an EFS volume

The steps in the following procedure explain how to create EFS volumes and access for the IAM user that created the ROSA cluster. 

Procedure

  1. On the AWS console, open https://console.aws.amazon.com/efs.
  2. Navigate to File systems and click Create file system
    NOTE: Ensure that you are in the correct AWS region before proceeding.
    • Enter a name for the file system.
    • For Virtual Private Cloud (VPC), select the VPC that was used for the OpenShift cluster built by using ROSA CLI.
    • Accept default settings for all other selections.
  3. Wait for the volume and mount targets to be created fully.
  4. Select your new EFS volume and navigate to the Network tab.
  5. Copy the Security Group ID to your clipboard (for example, sg-012115e5809162746)
  6. Go to Security Groups and find the Security Group used by the EFS volume copied from the previous step.
  7. On the Inbound rules tab, click Edit inbound rules, and then add a new rule with the following settings:
    Type: NFS
    Protocol: TCP
    Port range: 2049
    Source: Custom/IP address range of your nodes (for example: “10.0.0.0/16”)
  8. Save the rule.

Create a storage class for your EFS volume

    Storage classes are used to differentiate and delineate storage levels and usages. By defining a storage class, users can obtain dynamically provisioned persistent volumes. Installing the AWS EFS CSI Driver Operator does not create a storage class by default. Use the following steps to manually create the EFS storage class validated for Db2u.
    Ensure that you have the following information before running the command to create your EFS storage class:
    • provisioningMode: The EFS access point mode. Use efs-ap
    • fileSystemId: The EFS file system ID. For example, - fs-08a5b4467d198bf3e.
    • uid: The access point user ID. Use zero (0).
    • gid: The access point group ID. Use zero (0).
    • directoryPerms: The directory permissions for the access point root directory. Use 777.

    Procedure

    1. In the OpenShift Container Platform console, click Storage → Storage Classes
    2. On the StorageClasses overview page, click Create Storage Class.
    3. On the StorageClasses create page, do the following:
      • Enter a name to reference the storage class.
      • Select the reclaim policy.
      • Select efs.csi.aws.com from the Provisioner drop-down list.
      • Enter additional parameters for the storage class as follows:
        apiVersion: storage.k8s.io/v1
        kind: StorageClass
        metadata:
          name: efs-test-sc
        parameters:
          directoryPerms: "777"
          fileSystemId: <efs file system ID>
          uid: "0"
          gid: "0"
          provisioningMode: efs-ap
        provisioner: efs.csi.aws.com
        reclaimPolicy: Delete
        volumeBindingMode: Immediate
        NOTE: The above parameters are validated for Db2u. If you are deploying any other service, validate the parameters that work for your service.
    4.  Replace <efs file system ID> with the Elastic File System (EFS) ID created in the previous "Create and configure an EFS volume" procedure.

    Create an EBS storage class

    Before you deploy your database instance, ensure that you create an EBS storage class. The EBS CSI Driver Operator is installed by default on the ROSA cluster.
    Ensure that you have the following information:

    • The EBS volume type (type).
    • The number of I/O operations per second, per GB (iopsPerGB).
    • The storage class YAML code:
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
        name: ebs-sc
      parameters:
        type: io2 
        iopsPerGB: "500" #this field is required for io1 and io2 type storage classes. This can be calulated based on your volume size and maximum allowable iops. Details: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html
      provisioner: ebs.csi.aws.com
      reclaimPolicy: Delete
      volumeBindingMode: WaitForFirstConsumer
    Deploy a database instance on your ROSA cluster

      When you have finished creating your ROSA cluster, and configured your cloud storage option, you can deploy one of the following instances to the cluster, using the Db2 Operator:

      • A single-partition instance of Db2.
      • A single-partition instance of Db2 Warehouse.
      • A multi-partition instance of Db2 Warehouse.

      When you log in to your ROSA cluster you will need to complete the following tasks:

      • Create the namespace for the Db2 operator.
      • Create a CatalogSource object in the olm namespace to install the Db2 operator.
      • Deploy the Db2 operator in the namespace.
      Procedure
      1. Create an IBM Operator Catalog using the following YAML: 
        cat << EOF | kubectl create -f -
        apiVersion: operators.coreos.com/v1alpha1
        kind: CatalogSource
        metadata:
          name: ibm-db2uoperator-catalog
          namespace: openshift-marketplace
        spec:
          sourceType: grpc
          image: icr.io/cpopen/ibm-db2uoperator-catalog@sha256:8a5ab72a72f7bd42b7874b687f20ac01cb82b0f05ca24e56d5a21b1413c8ef09
          displayName: IBM Db2 Catalog
          publisher: IBM
          updateStrategy:
            registryPoll:
              interval: 45m
        EOF
      2. Confirm the catalogsource and the ibm-operator-catalog pod is created by using the following command: 
        oc get catalogsource,pods -n openshift-marketplace
        
      3. Go to Red Hat OpenShift's OperatorHub and search for "IBM Db2" in the filter box.
      4. Select db2uOperator from the search results and then select the namespace in which you want the operator to be installed.
      5. Confirm the creation of the operator pod:
        oc get pods -n db2u | grep db2u-operator
        db2u-operator-manager-88884fb5f-r59j2                             1/1     Running     0          0d1h
      6. To create a database instance, you can either use a Custom Resource Definition (CRD) to create it from the UI or in command line. Go to Installed Operators → IBM Db2 → Db2uCluster Tab → Create Db2uCluster.
        NOTE: See Deploying Db2 using the Db2uCluster custom resource for more information on CRDs.
      7. After the db2u-operator pod is up and running, run the YAML code to deploy your database instance.
        Choose from the following:
        Parameters common to this code include:
        Field Values
        (with examples)
        Description
        metadata.name db2oltp-test The name of the Db2uCluster CR.
        metadata.namespace db2u The namespace where the database instance will be deployed.
        .spec.size 1 The number of Db2 nodes.
        For single-partition Db2 and Db2 Warehouse instances, the value is 1; For multi-partition Db2 Warehouse instances, the value can be 2 or greater.
        .spec.environment.database.name BLUDB The database name for the instance.
        .spec.environment.dbType db2oltp Accepted values: db2wh, db2oltp.
        .spec.environment.ldap.enabled false To enable LDAP, set this to true
        .spec.license.accept true This is a required value that must be set to true
        .spec.podConfig.resource.db2u.limits.cpu "2" The CPU limits for the db2u engine pods. Limits and request are set to the same value (which is the expected value).
        .spec.podConfig.resource.db2u.limits.memory 8Gi The memory limits for the db2u engine pods. Limits and request are set to same value (which is the expected value).
        .spec.version 11.5.7.0-cn5 The Db2u version that the operator supports. The value in the left column is the latest release.
        .spec.storage [] An array of storage configurations. This is the required storage configuration for meta and data (or shared).
      8. Check the status of your Db2uCluster:
        oc get db2ucluster -n ${NAMESPACE} ${DB2U_CLUSTER_NAME}
        where DB2U_CLUSTER_NAME is the name value set in the metadata section of the YAML code. For example,
        name: db2oltp-test
        NOTE: You can define an alternate value for name by using the db2ucluster custom resource.
      9. When the STATE value returned is Ready, the instance is deployed successfully.
        ${DB2U_CLUSTER_NAME}
        NAME                        STATE   MAINTENANCESTATE          AGE
        db2-001234567890          Ready         None               6h8m
      10. Log on to the database engine pod as db2inst1:
        oc -n ${NAMESPACE} exec -it $(oc get pods -n ${NAMESPACE} | grep ${DB2U_CLUSTER_NAME}-db2u-0 | awk '{print $1}') -- su - db2inst1
      11. Connect to the database bludb on your AWS cluster:
        db2 connect to bludb
        NOTE: While we are using the database name bludb for this tutorial, you can change this name by using the db2ucluster custom resource.

      [{"Type":"MASTER","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"ARM Category":[{"code":"a8m500000008PknAAE","label":"Install\/Migrate\/Upgrade"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

      Document Information

      Modified date:
      05 August 2022

      UID

      ibm16600093