Installing IBM Knowledge Catalog

An instance administrator can install IBM Knowledge Catalog on IBM® Software Hub Version 5.2.

Who needs to complete this task?

Instance administrator To install IBM Knowledge Catalog, you must be an instance administrator. An instance administrator has permission to install software in the following projects:

The operators project for the instance

The operators for this instance of IBM Knowledge Catalog are installed in the operators project.

In the installation commands, the ${PROJECT_CPD_INST_OPERATORS} environment variable refers to the operators project.

The operands project for the instance

The custom resources for the control plane and IBM Knowledge Catalog are installed in the operands project.

In the installation commands, the ${PROJECT_CPD_INST_OPERANDS} environment variable refers to the operands project.

When do you need to complete this task?

Review the following options to determine whether you need to complete this task:

  • If you want to install multiple services at the same time, follow the process in Running a batch installation of solutions and services instead.
  • If you didn't install IBM Knowledge Catalog as part of a batch installation, complete this task to add IBM Knowledge Catalog to your environment.

    Repeat as needed If you are responsible for multiple instances of IBM Software Hub, you can repeat this task to install more instances of IBM Knowledge Catalog on the cluster.

Information you need to complete this task

Review the following information before you install IBM Knowledge Catalog:

Version requirements

All of the components that are associated with an instance of IBM Software Hub must be installed at the same release. For example, if the IBM Software Hub control plane is installed at Version 5.2.2, you must install IBM Knowledge Catalog at Version 5.2.2.

Environment variables

The commands in this task use environment variables so that you can run the commands exactly as written.

  • If you don't have the script that defines the environment variables, see Setting up installation environment variables.
  • To use the environment variables from the script, you must source the environment variables before you run the commands in this task. For example, run:
    source ./cpd_vars.sh
Security context constraint

IBM Knowledge Catalog works with the default Red Hat® OpenShift® Container Platform security context constraint, restricted-v2.

Common core services

IBM Knowledge Catalog requires the IBM Software Hub common core services.

If the common core services are not installed in the operands project for the instance, the common core services are automatically installed when you install IBM Knowledge Catalog. The common core services installation increases the amount of time the installation takes to complete.

Storage requirements
You must specify storage classes when you install IBM Knowledge Catalog. The following storage classes are recommended. However, if you don't use these storage classes on your cluster, ensure that you specify a storage class with an equivalent definition.
Storage Notes Storage classes
OpenShift Data Foundation When you install the service, specify file storage and block storage.
  • File storage: ocs-storagecluster-cephfs
  • Block storage: ocs-storagecluster-ceph-rbd
IBM Fusion Data Foundation When you install the service, specify file storage and block storage.
  • File storage: ocs-storagecluster-cephfs
  • Block storage: ocs-storagecluster-ceph-rbd
IBM Fusion Global Data Platform When you install the service, specify the same storage class for both file storage and block storage.
  • File storage:

    Either of the following storage classes:

    • ibm-spectrum-scale-sc
    • ibm-storage-fusion-cp-sc
  • Block storage:

    Either of the following storage classes:

    • ibm-spectrum-scale-sc
    • ibm-storage-fusion-cp-sc
IBM Storage Scale Container Native When you install the service, specify the same storage class for both file storage and block storage.
  • File storage: ibm-spectrum-scale-sc
  • Block storage: ibm-spectrum-scale-sc
Portworx When you install the service, the --storage_vendor=portworx option ensures that the service uses the correct storage classes.
  • File storage: portworx-shared-gp3
  • Block storage:
    • portworx-cassandra-sc
    • portworx-couchdb-sc
    • portworx-db2-rwo-sc
    • portworx-elastic-sc
    • portworx-gp3-sc
    • portworx-kafka-sc
    • portworx-metastoredb-sc
    • portworx-solr-sc
NFS When you install the service, specify the same storage class for both file storage and block storage.
  • File storage: managed-nfs-storage
  • Block storage: managed-nfs-storage
Amazon Elastic storage
When you install the service, you can specify:
  • File storage only
  • File storage and block storage (recommended)

File storage is provided by Amazon Elastic File System. Block storage is provided by Amazon Elastic Block Store.
  • File storage: efs-nfs-client
  • Block storage:

    Either of the following storage classes:

    • gp2-csi
    • gp3-csi
NetApp Trident When you install the service, specify the same storage class for both file storage and block storage.
  • File storage: ontap-nas
  • Block storage: ontap-nas
Nutanix When you install the service, specify file storage and block storage.
  • File storage: nutanix-file
  • Block storage: nutanix-volume

Before you begin

This task assumes that the following prerequisites are met:

System requirements
This task assumes that the cluster meets the minimum requirements for IBM Knowledge Catalog.
Where to find more information
If this task is not complete, see System requirements.
Workstation
This task assumes that the workstation from which you will run the installation is set up as a client workstation and has the following command-line interfaces:
  • IBM Software Hub CLI: cpd-cli
  • OpenShift CLI: oc
Where to find more information
If this task is not complete, see Setting up a client workstation.
Control plane
This task assumes that the IBM Software Hub control plane is installed.
Where to find more information
If this task is not complete, see Installing an instance of IBM Software Hub.
Private container registry
If your environment uses a private container registry (for example, your cluster is air-gapped), this task assumes that the following tasks are complete:
  1. The IBM Knowledge Catalog software images are mirrored to the private container registry.
    Where to find more information
    If this task is not complete, see Mirroring images to a private container registry.
  2. The cpd-cli is configured to pull the olm-utils-v3 image from the private container registry.
    Where to find more information
    If this task is not complete, see Pulling the olm-utils-v3 image from the private container registry.
Load balancer
The load balancer timeout settings are adjusted for IBM Knowledge Catalog.
Where to find more information
If this task is not complete, see Changing load balancer settings.
GPU operators
If you plan to use features that require GPUs, this task assumes that the operators required to use GPUs are installed.
Where to find more information
If this task is not complete, see Installing operators for services that require GPUs.

Procedure

Complete the following tasks to install IBM Knowledge Catalog:

  1. Specifying your IBM Knowledge Catalog edition
  2. Specifying installation options
  3. Installing the service
  4. Validating the installation
  5. What to do next

Specifying your IBM Knowledge Catalog edition

IBM Knowledge Catalog is available in three different editions. You must specify which edition to install.

Set the IKC_TYPE environment variable to the edition of IBM Knowledge Catalog that you want to install:

IBM Knowledge Catalog
export IKC_TYPE=wkc
IBM Knowledge Catalog Premium
export IKC_TYPE=ikc_premium
IBM Knowledge Catalog Standard
export IKC_TYPE=ikc_standard

Specifying installation options

If you plan to install IBM Knowledge Catalog, IBM Knowledge Catalog Premium, or IBM Knowledge Catalog Standard, you can specify installation options in a file named install-options.yml in the cpd-cli work directory (For example: cpd-cli-workspace/olm-utils-workspace/work).

The parameters are optional. If you do not set these installation parameters, the default values are used. Uncomment the parameters that you want to override and update the values appropriately.

The applicable parameters and default values depend on which service you install:

IBM Knowledge Catalog

The sample YAML content uses the default values.

################################################################################
# IBM Knowledge Catalog parameters
################################################################################
custom_spec:
  wkc:
#    enableDataQuality: False
#    enableKnowledgeGraph: False
#    useFDB: False
IBM Knowledge Catalog Premium

The sample YAML content uses the default values.

################################################################################
# IBM Knowledge Catalog parameters
################################################################################
custom_spec:
  wkc:
#    enableDataQuality: False
#    enableKnowledgeGraph: False
#    useFDB: False
#    enableAISearch: False
#    enableSemanticAutomation: False
#    enableSemanticEnrichment: True
#    enableSemanticEmbedding: False
#    enableTextToSql: False
#    enableModelsOn: 'cpu'
#    customModelTextToSQL: granite-3-3-8b-instruct
IBM Knowledge Catalog Standard

The sample YAML content uses the default values.

################################################################################
# IBM Knowledge Catalog parameters
################################################################################
custom_spec:
  wkc:
#    enableKnowledgeGraph: False
#    useFDB: False
#    enableAISearch: False
#    enableSemanticAutomation: False
#    enableSemanticEnrichment: True
#    enableSemanticEmbedding: False
#    enableTextToSql: False
#    enableModelsOn: 'cpu'
#    customModelTextToSQL: granite-3-3-8b-instruct
Property Description
enableDataQuality Specify whether to enable data quality features in projects.
Important: If you enable this feature, DataStage, specifically DataStage Enterprise, is automatically installed.

If you did not purchase a DataStage license, use of DataStage Enterprise is limited to creating, managing, and running data quality rules. For examples of accepted use, see Enabling optional features after installation or upgrade for IBM Knowledge Catalog.

Editions the setting applies to
  • IBM Knowledge Catalog
  • IBM Knowledge Catalog Premium
Default value
False
Valid values
False
Do not enable the data quality feature.
True
Enable the data quality feature.
enableKnowledgeGraph Specify whether to enable the knowledge graph feature. The knowledge graph provides the following capabilities:
  • Relationship explorer and business term relationship search
  • Lineage
    Important: Lineage requires IBM Manta Data Lineage or MANTA Automated Data Lineage.
Editions the setting applies to
  • IBM Knowledge Catalog
  • IBM Knowledge Catalog Premium
  • IBM Knowledge Catalog Standard
Default value
False
Valid values
False
Do not enable the knowledge graph feature.
True
Enable the knowledge graph feature.

If you set enableKnowledgeGraph: True, review useFDB.
useFDB Specify which database to use to store the data generated by knowledge graph.
The database depends on which service you use for lineage:
  • For IBM Manta Data Lineage, use Neo4j:
    useFDB: false
  • For MANTA Automated Data Lineage, use FoundationDB:
    useFDB: true
Default value
False
Valid values
False
Do not use FoundationDB. Use Neo4j.

Required if you use IBM Manta Data Lineage.

True
Use FoundationDB.

Required if you use MANTA Automated Data Lineage.
enableAISearch Specify whether to enable LLM-based semantic search for assets and artifacts across all workspaces.
Default value
False
Valid values
False
Do not enable LLM-based semantic search.
True
Enable LLM-based semantic search.
enableSemanticAutomation Specify whether to enable gen AI features.
Editions the setting applies to
  • IBM Knowledge Catalog Premium
  • IBM Knowledge Catalog Standard
Default value
False
Valid values
False
Do not enable gen AI based features.
True
Enable gen AI based features.
enableSemanticEnrichment Specify whether to enable gen AI metadata expansion. Metadata expansion includes:
  • Table name expansion
  • Column name expansion
  • Description generation
Editions the setting applies to
  • IBM Knowledge Catalog Premium
  • IBM Knowledge Catalog Standard
Prerequisite
This feature requires semantic automation. You must set enableSemanticAutomation:true.
Default value
False
Valid values
False
Do not enable gen AI metadata expansion.
True
Enable gen AI metadata expansion.
enableSemanticEmbedding

5.2.1 and later This parameter is available starting in IBM Software Hub Version 5.2.1.

Specify whether to enable semantic embedding.

You must enable semantic embedding if you plan to use the following features:
  • Text to SQL
Editions the setting applies to
  • IBM Knowledge Catalog Premium
  • IBM Knowledge Catalog Standard
Prerequisite

This feature requires GPU. You cannot run the required model on CPU.

In addition, this feature requires gen AI capabilities. You must set enableGenerativeAICapabilities: true.

Default value
false
Valid values
false
Do not enable semantic embedding.
true
Enable semantic embedding.
enableTextToSql

5.2.1 and later This parameter is available starting in IBM Software Hub Version 5.2.1.

Specify whether to generate SQL queries from natural language input. Text-to-SQL capabilities can be used to create query-based data assets, which can be use for data products or in searches.

Editions the setting applies to
  • IBM Knowledge Catalog Premium
  • IBM Knowledge Catalog Standard
Prerequisite

This feature requires GPU. You can choose where to run the required models:

  • To run the required models locally, set enableModelsOn: gpu
  • To run the required models on a remote instance of watsonx.ai™, set enableModelsOn: remote

In addition, this feature requires the following settings:

  • Semantic embedding.

    You must set enableSemanticEmbedding: true.

Default value
false
Valid values
false
Do not convert natural language queries to SQL queries.
true
Convert natural language queries to SQL queries.
enableModelsOn Specify where you want the models that are used with the gen AI capabilities to run.
Editions the setting applies to
  • IBM Knowledge Catalog Premium
  • IBM Knowledge Catalog Standard
Prerequisite
This feature requires semantic automation. You must set enableSemanticAutomation:true.
Default value
'cpu'
Valid values
'cpu'
Run the foundation model on CPU.
Restriction: This option can be used only for expanding metadata and term assignment when enriching metadata (enableSemanticEnrichment: true).

This option is not supported for converting natural language queries to SQL queries ( enableTextToSql: true).

'gpu'
Run the foundation model on GPU.

If you are upgrading the service and you want to continue to run the model on GPU, you must specify enableModelsOn: 'gpu'.

Important: If you use this setting, the inference foundation models component (watsonx_ai_ifm) is automatically installed.

This option requires at least one GPU. For information about supported GPUs, see GPU requirements for models.

'remote'
Run the foundation model on a remote instance of watsonx.ai. The instance can be running on:
  • Another on-premises instance of IBM Software Hub
  • IBM watsonx™ as a Service
Important: If you use this setting, you must:
  1. Ensure that the foundation model is available and running on the remote instance.
  2. Create a connection to the remote instance.

    For more information, see Enabling users to connect to an external IBM watsonx.ai foundation model in the Cloud Pak for Data documentation.

If the preceding requirements are not met, any tasks that rely on the model will fail.

customModelTextToSql Specify a custom model for Text-To-SQL conversions.
Default model

By default, the Text-To-SQL feature uses the granite-3-8b-instruct model (ID: granite-3-8b-instruct).

Recommended model for better accuracy

You can improve the accuracy of results when converting plain text queries to SQL queries if you use the llama-3-3-70b-instruct model (ID: llama-3-3-70b-instruct).

However, this model requires significantly more resources than the granite-3-8b-instruct model. For more information about the resources required for each model, see GPU requirements for models.

Using other models

If you chose to use a different model, the accuracy of the results might vary.

Prerequisite

This option applies only to environments with local GPUs (enableModelsOn: gpu).

If you want to use a custom model on a remote instance of watsonx.ai (enableModelsOn: remote), see Enabling users to connect to an external IBM watsonx.ai foundation model in the Data Fabric documentation.

In addition, this feature requires the following settings:

  • Text-To-SQL conversions.

    You must set enableTextToSql: true.

Default value
granite-3-8b-instruct
Valid values
Specify the ID of the model that you want to use. The IDs of the recommended models are:
  • granite-3-8b-instruct
  • llama-3-3-70b-instruct

Installing the service

To install IBM Knowledge Catalog:

  1. Log the cpd-cli in to the Red Hat OpenShift Container Platform cluster:
    ${CPDM_OC_LOGIN}
    Remember: CPDM_OC_LOGIN is an alias for the cpd-cli manage login-to-ocp command.
  2. Run the following command to create the required OLM objects for IBM Knowledge Catalog in the operators project for the instance:
    cpd-cli manage apply-olm \
--release=${VERSION} \
--cpd_operator_ns=${PROJECT_CPD_INST_OPERATORS} \
--components=${IKC_TYPE}
    Wait for the cpd-cli to return the following message before you proceed to the next step:
    [SUCCESS]... The apply-olm command ran successfully

    If the apply-olm fails, see Troubleshooting the apply-olm command during installation or upgrade.

  3. Create the custom resource for IBM Knowledge Catalog.

    The command that you run depends on the storage on your cluster.

    Red Hat OpenShift Data Foundation storage

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    IBM Fusion Data Foundation storage

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    IBM Fusion Global Data Platform storage
    Remember: When you use IBM Fusion Global Data Platform storage, both ${STG_CLASS_BLOCK} and ${STG_CLASS_FILE} point to the same storage class, typically ibm-spectrum-scale-sc or ibm-storage-fusion-cp-sc.

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    IBM Storage Scale Container Native storage
    Remember: When you use IBM Storage Scale Container Native storage, both ${STG_CLASS_BLOCK} and ${STG_CLASS_FILE} point to the same storage class, typically ibm-spectrum-scale-sc.

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    Portworx storage

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--storage_vendor=portworx \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--storage_vendor=portworx \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    NFS storage
    Remember: When you use NFS storage, both ${STG_CLASS_BLOCK} and ${STG_CLASS_FILE} point to the same storage class, typically managed-nfs-storage.

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    AWS with EFS storage only
    Remember: When you use EFS storage, both ${STG_CLASS_BLOCK} and ${STG_CLASS_FILE} point to the same storage class, typically efs-nfs-client.

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    AWS with EFS and EBS storage

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true
    NetApp Trident
    Remember: When you use NetApp Trident storage, both ${STG_CLASS_BLOCK} and ${STG_CLASS_FILE} point to the same storage class, typically ontap-nas.

    Run the appropriate command to create the custom resource.

    Default installation (without installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--license_acceptance=true
    Custom installation (with installation options)
    cpd-cli manage apply-cr \
--components=${IKC_TYPE} \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--block_storage_class=${STG_CLASS_BLOCK} \
--file_storage_class=${STG_CLASS_FILE} \
--param-file=/tmp/work/install-options.yml \
--license_acceptance=true

Validating the installation

IBM Knowledge Catalog is installed when the apply-cr command returns:
[SUCCESS]... The apply-cr command ran successfully

If you want to confirm that the custom resource status is Completed, you can run the cpd-cli manage get-cr-status command:

cpd-cli manage get-cr-status \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--components=${IKC_TYPE}

What to do next

Your next steps depend on the version of IBM Software Hub that you installed:

Version 5.2.1 or later

  1. Complete Post-installation tasks for IBM Knowledge Catalog.

After you complete the preceding steps, IBM Knowledge Catalog is ready to use.

Version 5.2.0
  1. Install the services that you need on this instance of IBM Software Hub
  2. Apply the IBM Software Hub Version 5.2.0 - Day 0 patch

    You must apply the patch to each instance of IBM Software Hub Version 5.2.0 that you install.

  3. Complete Post-installation tasks for IBM Knowledge Catalog.

After you complete the preceding steps, IBM Knowledge Catalog is ready to use.