Backing up Db2 Big SQL metadata

It is critical to back up your Db2® Big SQL metadata before you do an upgrade.

Before you begin

The following list describes prerequisites and related information that are required before you back up Db2 Big SQL metadata.

  • The Ambari administrator username and password.
  • A user with the following attributes:
    • Passwordless sudo access on all nodes of the cluster, including the Ambari server itself.
    • The ability to connect passwordlessly through ssh from the Ambari server to all Db2 Big SQL nodes.

    This user can be root. If the user is not root, the username must be passed to the upgrade script with the -a option. The upgrade script must be run with root user privilege, which you can do by using the sudo command.

  • The backup metadata process requires about 5 GB of free disk space in the /tmp directory. This space is temporarily used by the installation process and released when the installation is complete. In addition, the backup metadata process requires about 2 GB of free disk space on /usr for Db2 installation. If these disk space requirements are not available, the upgrade prerequisite checker warns you and stops the installation process.

    On each Db2 Big SQL host, the backup metadata process requires sufficient space to back up the contents of the sqllib directory, the Db2 Big SQL database path, and the Db2 Big SQL database directories. This backup is written in compressed format to the /var/ibm/bigsql/upgrade-backup directory.

    In particular, on the head node, more space is needed for the backup if Db2 Big SQL contains native Db2 tables, as these tables are stored in the Db2 Big SQL database directories local to the head node. You can run the following command while connected as the bigsql user on the existing head node to check the Db2 Big SQL database size:
    call get_dbsize_info(?,?,?,-1)

    This command returns the size of the database in bytes. As the backup is compressed inline as it is being written to disk, the actual space that is needed by the backup is less than size of the database itself, depending on the natural redundancy of the data in the database. The size that is returned by the get_dbsize_info function can be considered an upper limit.

  • Ambari configuration groups are supported by the upgrade process for HDP services. Configuration groups for the Db2 Big SQL service are not supported. Remove any configuration groups for the Db2 Big SQL service before you initiate the backup metadata process.
  • The HDFS, Hive, and HBase (if installed) services, and the services that they depend on, must be up and running in the Ambari server. HDFS cannot be running in safe mode.

    If these services are not running, the upgrade prerequisite checker warns you and stops the patch process.

  • YARN integration must be disabled for Db2 Big SQL.
  • If the following section exists in the $BIGSQL_HOME/conf/bigsql-conf.xml file, remove it.
    <property>
    <name>scheduler.force.cppwriter</name>
    <value>true</value>
</property>
  • Meet Db2 package requirements, as they also apply to Db2 Big SQL. For more information, see the Package requirements section in the Db2 topic Additional installation considerations (Linux).
  • Make sure that the cluster and service are healthy. In Ambari, run the service actions Check Cluster Health and Run Service Check for the Db2 Big SQL service.
    Recommendation: Restart the Db2 Big SQL service before you run service actions.

About this task

You back up Db2 Big SQL metadata by using the Db2 Big SQL python script bigsql_upgrade.py.

Procedure

  1. Install the Db2 Big SQL 7.1.0.0 binary file.
    1. Obtain the IBM® Db2 Big SQL package. See Obtaining Db2 Big SQL for details.
    2. Run the downloaded file. See Extracting the Db2 Big SQL installation files for details.
  2. Run the Db2 Big SQL configuration utility to set the Ambari server connection information and other configuration parameters, if necessary.
    1. To specify the Ambari server connection information, set the following configuration parameters:
      • CM_HOST
      • CM_PORT
      • CM_ADMIN_USER
      • HDFS_USER
      • If https is used to access Ambari, CM_PROTOCOL and CM_SSL_CA_CERTIFICATE_PATH
    2. If you are using a non-default Db2 Big SQL user ID or system admin group, set the following parameters:
      • BIGSQL_USER
      • BIGSQL_USER_ID
      • SYSADMGRP_ID
      • SYSADM_GRP
  3. If High Availability (HA) is enabled, disable it.
  4. In Ambari, disable the Db2 Big SQL Data Directory check alerts by doing the following substeps:
    1. Open the Ambari UI and select the Alerts tab.
    2. In the Service box, filter on IBM Db2 Big SQL, and in the Alert Definition Name box, type Data.
    3. Under the State column, disable Big SQL Head Node Data Directories Check by clicking Enabled.
    4. Disable Big SQL Worker Data Directory Check by clicking Enabled.
  5. If the Db2 Big SQL plug-in for Ranger is enabled, disable it.
    Note: It is also recommended that you export existing policies. For more information about disabling Ranger and exporting policies, see Ranger requirements when upgrading Db2 Big SQL.
  6. Run the backup option of the Db2 Big SQL python script on the Db2 Big SQL head node. For details on the script, see bigsql_upgrade.py utility.
    1. The binary files of the new version of service Db2 Big SQL are installed from a yum repository. If you want to check the URL, open /usr/ibmpacks/IBM-Big_SQL/7.1.0.0/bigsql-cli/repos/IBM-Big_SQL-7_1_0_0.repo and locate the following code:
      [IBM-Big_SQL-7_1_0_0]
name=IBM-Big_SQL-7_1_0_0
baseurl=<IBM external repo URL>
enabled=1
gpgcheck=0

      If needed, change the repository URL.

    2. Back up your Db2 Big SQL environment by running the Backup option of the bigsql_upgrade.py python command. The Backup option does the following actions on all nodes of the cluster:
      • Backs up the Db2 Big SQL catalog, metadata, and configuration information
      • Installs the binary files for the new version
      • Disconnects the Db2 Big SQL service definition from Ambari
      To do the backup phase of the upgrade, run the bigsql_upgrade.py script with the -m option and the value Backup. Include any additional options as documented in the bigsql_upgrade.py utility. For example, if you configured Ambari for non-root access, use the -a option.
      /usr/ibmpacks/IBM-Big_SQL/7.1.0.0/upgrade/bigsql_upgrade.py -m Backup
  7. Optional: In case the backup phase fails, consult the master log output or the upgrade log that is at /var/ibm/bigsql/logs/upgrade.log to identify and resolve the problem. After it is resolved, rerun the backup phase.
  8. When the backup is successful, log in to Ambari and do the following steps:
    1. Stop the Db2 Big SQL service.
    2. Delete the service.
      Important: Do not decommission the service.
    3. Make sure that the Db2 Big SQL listener property com.ibm.biginsights.bigsql.sync.BIEventListener is removed from the Hive service.
      Note: This property is present in two places.
    4. In hive-env configurations, remove the following section. 
      # Allow Hive to read Big SQL HBase tables
if [ -d "/usr/ibmpacks/current/bigsql/bigsql/lib/java" ]; then
export HIVE_AUX_JARS_PATH=\
/usr/ibmpacks/current/bigsql/bigsql/lib/java/biga-io.jar,\
/usr/ibmpacks/current/bigsql/bigsql/lib/java/biga-hbase.jar,\
/usr/ibmpacks/current/bigsql/bigsql/lib/java/commoncatalog.jar,\
/usr/ibmpacks/current/bigsql/hive/lib/hive-hbase-handler.jar,\
${HIVE_AUX_JARS_PATH}
fi
    5. Restart Hive.

Results

Db2 Big SQL is removed from Ambari. However, it is still operational, but not running. If needed, you can start the service from the command line and use it. In this case, the version that is run is the initial Db2 Big SQL version.