Upgrading Query Optimizer

You can upgrade an active Query Optimizer service in an existing IBM® watsonx.data instance.

watsonx.data on IBM Software Hub

About this task

To upgrade Query optimizer manager in watsonx.data, complete the following steps.

Procedure

  1. Optional: Save DDL statements for constraints before you deactivate Query optimizer manager.
    Important: Complete this step if you want to save constraint definitions for reuse after the upgrade. If constraint metadata is lost during the upgrade, you can use the saved DDL statements to recreate the constraints after you reactivate Query optimizer manager.

    Because schema names that are registered in Query optimizer manager can be difficult to identify, first list the schema names explicitly:

    ExecuteWxdQueryOptimizer '
    SELECT DISTINCT TABSCHEMA
    FROM SYSCAT.KEYCOLUSE
    WHERE CONSTNAME IN (
    SELECT CONSTNAME
    FROM SYSCAT.TABCONST
    WHERE TYPE = 'P'
    )';

    From the output, identify the schema that you want to use in the following queries.

    Generate DDL statements for NOT NULL constraints:

    -- ========================================
    -- DDL Generation Query for NOT NULL 
    -- Schema: iceberg_data.IMIO_SF1_NO_PART
    -- ========================================
    
    ExecuteWxdQueryOptimizer '
      select 'ALTER TABLE ' || tabschema || '.' || tabname  || ' ALTER COLUMN ' || COLNAME  || ' SET NOT NULL '  AS DDL_STATEMENT
      from syscat.columns where  tabschema = 'iceberg_data.IMIO_SF1_NO_PART' AND NULLS= 'N' ';

    Generate DDL statements for primary keys:

    -- ========================================
    -- DDL Generation Query for Primary Keys (INFORMATIONAL)
    -- Schema: iceberg_data.IMIO_SF1_NO_PART
    -- ========================================
    ExecuteWxdQueryOptimizer '
    SELECT
    'ALTER TABLE ' || RTRIM(TABSCHEMA) || '.' || RTRIM(TABNAME) ||
    ' ADD CONSTRAINT ' || RTRIM(CONSTNAME) ||
    ' PRIMARY KEY (' ||
    LISTAGG(RTRIM(COLNAME), ', ') WITHIN GROUP (ORDER BY COLSEQ) ||
    ') NOT ENFORCED' AS DDL_STATEMENT
    FROM SYSCAT.KEYCOLUSE
    WHERE TABSCHEMA = 'iceberg_data.IMIO_SF1_NO_PART'
    AND CONSTNAME IN (
    SELECT CONSTNAME
    FROM SYSCAT.TABCONST
    WHERE TABSCHEMA = 'iceberg_data.IMIO_SF1_NO_PART'
    AND TYPE = 'P'
    )
    GROUP BY TABSCHEMA, TABNAME, CONSTNAME
    ORDER BY TABSCHEMA, TABNAME';

    Generate DDL statements for foreign keys:

    -- ========================================
    -- DDL Generation Query for Foreign Keys (INFORMATIONAL)
    -- Schema: iceberg_data.IMIO_SF1_NO_PART
    -- ========================================
    ExecuteWxdQueryOptimizer '
    SELECT
    'ALTER TABLE ' || RTRIM(r.TABSCHEMA) || '.' || RTRIM(r.TABNAME) ||
    ' ADD CONSTRAINT ' || RTRIM(r.CONSTNAME) ||
    ' FOREIGN KEY (' ||
    LISTAGG(RTRIM(k.COLNAME), ', ') WITHIN GROUP (ORDER BY k.COLSEQ) ||
    ') REFERENCES ' || RTRIM(r.REFTABSCHEMA) || '.' || RTRIM(r.REFTABNAME) ||
    ' ON DELETE ' ||
    CASE r.DELETERULE
    WHEN 'A' THEN 'NO ACTION'
    WHEN 'C' THEN 'CASCADE'
    WHEN 'N' THEN 'SET NULL'
    WHEN 'R' THEN 'RESTRICT'
    ELSE 'NO ACTION'
    END ||
    ' ON UPDATE ' ||
    CASE r.UPDATERULE
    WHEN 'A' THEN 'NO ACTION'
    WHEN 'R' THEN 'RESTRICT'
    ELSE 'NO ACTION'
    END || ' NOT ENFORCED' AS DDL_STATEMENT
    FROM SYSCAT.REFERENCES r
    JOIN SYSCAT.KEYCOLUSE k
    ON r.CONSTNAME = k.CONSTNAME
    AND r.TABSCHEMA = k.TABSCHEMA
    AND r.TABNAME = k.TABNAME
    WHERE r.TABSCHEMA = 'iceberg_data.IMIO_SF1_NO_PART'
    GROUP BY r.CONSTNAME, r.TABSCHEMA, r.TABNAME, r.REFTABSCHEMA, r.REFTABNAME, r.DELETERULE, r.UPDATERULE
    ORDER BY r.CONSTNAME';

    Save the generated DDL statements so that you can reuse them after the upgrade.

    Note: When you run queries by using ExecuteWxdQueryOptimizer in the Presto utility, do not include semicolons in the query text. If necessary, add semicolons manually to the generated DDL statements before you run them later.
  2. Deactivate Query optimizer manager. See Deactivating Query optimizer manager.
    Important: For upgrading to version 2.3.1 and above, you must complete this step before upgrading watsonx.data. If you upgrade watsonx.data before deactivating Query optimizer manager, the optimizer can enter an error state that prevents proper deactivation.
  3. Upgrade the existing version of watsonx.data to the latest version. See Upgrading watsonx.data.
  4. To upgrade Query optimizer manager, you must upgrade the Db2 version by running the following command.
    Note: Starting with IBM Software Hub 5.3.0, the apply-olm command is deprecated. You must now use the install-components workflow, which requires multiple commands instead of the single command used in earlier releases.

    Use the cpd-cli manage case-download command to generate cluster-scoped resources for the Query optimizer manager upgrade:

    cpd-cli manage case-download \
      --release=${VERSION} \
      --components=wxd_query_optimizer \
      --cluster_resources=true \
      --operator_ns=${PROJECT_CPD_INST_OPERATORS}
  5. Change to the directory that contains the generated YAML files. The default location of the work directory is:
    cpd-cli-workspace/olm-utils-workspace/work
  6. Apply the cluster-scoped resources from the cluster_scoped_resources.yaml file:
    oc apply -f cluster_scoped_resources.yaml \
    --server-side \
    --force-conflicts
  7. Install the operators for Query optimizer manager:
    cpd-cli manage install-components \
    --license_acceptance=true \
    --components=wxd_query_optimizer \
    --release=${VERSION} \
    --operator_ns=${PROJECT_CPD_INST_OPERATORS} \
    --instance_ns=${PROJECT_CPD_INST_OPERANDS} \
    --block_storage_class=${STG_CLASS_BLOCK} \
    --file_storage_class=${STG_CLASS_FILE} \
    --image_pull_prefix=${IMAGE_PULL_PREFIX} \
    --image_pull_secret=${IMAGE_PULL_SECRET} \
    --param-file='/tmp/work/override.yaml' \
    --upgrade=true
  8. Activate Query optimizer manager to upgrade to the latest version of Query optimizer manager. See Activating Query optimizer in IBM watsonx.data.

    This step reactivates Query optimizer manager after the watsonx.data upgrade is complete.

    Note: If you saved DDL statements for constraints before the upgrade, use them after reactivation to recreate any constraints that were lost during the upgrade.
  9. Run the following pass-through command in the Query workspace of watsonx.data with the Presto (C++) engine to verify that the Query optimizer manager version (Db2 build level) is updated to the latest version as in PTF version.
    ExecuteWxdQueryOptimizer 'select service_level,
    release_num,
    ptf
    from sysibmadm.ENV_INST_INFO'
    For each watsonx.data version, the result must be as follows:
    Release version SERVICE_LEVEL RELEASE_NUM PTF
    2.1.0 DB2 v11.5.9.0 060A010F DYN2411080940AMD64_50545
    2.2.0 DB2 v11.5.9.0 060A010F DYN2505081325AMD64_59605
    2.2.1 DB2 v11.5.9.0 060A010F DYN2507310822AMD64_63280
    2.2.2 DB2 v11.5.9.0 060A010F DYN2509251021AMD64_68051
    2.3.0 DB2 v12.1.3.0 02040110 DYN2511061342AMD64_70372