SSQNUZ_5.0.x - Documentation Index
Table of Contents
Welcome
Overview
Version 5.0 release notes
What's new?
Cloud Pak for Data platform
Cloud Pak for Data APIs
Cloud Pak for Data command-line interface (cpd-cli)
Common core services
Scheduling service
AI Factsheets
Analytics Engine powered by Apache Spark
Cognos Analytics
Cognos Dashboards
Data Gate
Data Privacy
Data Product Hub
Data Refinery
Data Replication
DataStage
Data Virtualization
Db2
Db2 Big SQL
Db2 Data Management Console
Db2 Warehouse
Decision Optimization
EDB Postgres
Execution Engine for Apache Hadoop
IBM Knowledge Catalog
IBM Match 360
Informix
MANTA Automated Data Lineage
MongoDB
OpenPages
Orchestration Pipelines
Planning Analytics
Product Master
RStudio Server Runtimes
SPSS Modeler
Synthetic Data Generator
Voice Gateway
Watson Discovery
Watson Machine Learning
Watson Machine Learning Accelerator
Watson OpenScale
Watson Speech services
Watson Studio
Watson Studio Runtimes
watsonx.ai
watsonx Assistant
watsonx Code Assistant for Red Hat Ansible Lightspeed
watsonx Code Assistant for Z
watsonx Code Assistant for Z Code Explanation
watsonx.data
watsonx.governance
watsonx Orchestrate
Known issues and limitations
Flight service issues
FIPS-enabled clusters
Scheduling service
Service issues
AI Factsheets
Analytics Engine powered by Apache Spark
Cognos Analytics
Cognos Dashboards
Common core services
Data Gate
Data Privacy
Data Product Hub
Data Refinery
Data Replication
DataStage
Data Virtualization
Db2 and Db2 Warehouse
Db2 Big SQL
Db2 Data Management Console
EDB Postgres
IBM Knowledge Catalog
IBM Match 360 with Watson
Informix
MongoDB
OpenPages
Orchestration Pipelines
Product Master
SPSS Modeler
Watson Discovery
Watson Machine Learning
Watson Machine Learning Accelerator
Watson OpenScale
Watson Speech services
Watson Studio
watsonx.ai
watsonx Assistant
watsonx Code Assistant for Red Hat Ansible Lightspeed
watsonx Code Assistant for Z
watsonx.data
watsonx.governance
watsonx Orchestrate
Use cases
Data fabric use cases
Data governance use case
Data integration use case
Master Data Management use case
Build and govern AI use cases
Data science and MLOps use case
AI governance use case
watsonx.ai use case
watsonx.governance use case
Business analytics use case
Customer care use case
PDF documentation for previous 5.0.x refreshes
Regulatory compliance
Accessibility
Considerations for GDPR readiness
Considerations for HIPAA readiness
Language support
Getting started and tutorials
Roadmaps for getting started
Setting up your console and first steps
Setting your preferences
Finding services
Checking your permissions
Identifying tasks that need to be completed
Generating API keys
Switching between experiences
Asset types and properties
Previews
Profiles
Data quality
Relationships
Lineage
Visualizations
Activities
Quick start tutorials
Refine data
Generate synthetic tabular data
Analyze data in a Jupyter notebook
Tell a story with a dashboard
Build and deploy a machine learning model with AutoAI
Build and deploy a machine learning model in a Jupyter notebook
Build and deploy a machine learning model with SPSS Modeler
Build, run, and deploy a Decision Optimization model
Evaluate a machine learning model
Prompt a foundation model with Prompt Lab
Prompt a foundation model by using RAG
Tune a foundation model
Try the watsonx.ai end-to-end use case
Evaluate and track a prompt template
Evaluate a deployment in spaces
Evaluate a machine learning model
Use case tutorials
Build and deploy a model
Test and validate the model
Orchestrate an AI pipeline with model monitoring
Orchestrate an AI pipeline with data integration
Integrate data
Virtualize external data
Configure a 360 degree view
Curate high quality data
Protect your data
Consume your data
Govern virtualized data
Video library
Planning
Architecture
Storage architecture
Operators
Supported project (namespace) configurations
Multitenancy support
Remote physical locations
Installation roles and personas
Licenses and entitlements
Licensing guidance: Enterprise Edition
Planning for multiple solutions
Planning for multiple IBM Cloud Paks
Planning for a deployment with stand-alone software
Cloud deployment environments
Storage considerations
Shared components
System requirements
Hardware requirements
Downloading the component scaling guidance PDF
Planning for Cloud Pak for Data for Data on IBM Z and LinuxONE
Storage requirements
Software requirements
Private container registry requirements
Client workstation requirements
Remote physical location requirements
Operator and operand versions
5.0.3 (September 2024)
5.0.2 (August 2024)
5.0.1 (July 2024)
5.0.0 (June 2024)
Security considerations
Basic security features
CIS Benchmark for Red Hat OpenShift Container Platform v4
Services that support FIPS
Enabling FIPS
Auditing Cloud Pak for Data
Services that support audit logging
Network security
Kerberos authentication
Planning to access data
Installing
Setting up a client workstation
Installing the Cloud Pak for Data CLI
Installing the OpenShift CLI
Setting up a cluster
Installing Red Hat OpenShift
Installing persistent storage
Setting up a private container registry
Collecting required information
Obtaining your IBM entitlement API key
Determining which components to install
Setting up installation environment variables
Preparing to run installs in a restricted network
Obtaining the olm-utils-v3 image
Downloading CASE packages
Preparing to run installs from a private container registry
Mirroring images to a private container registry
Mirroring images directly to the private container registry
Mirroring images using an intermediary container registry
Configuring an image content source policy
Pulling the olm-utils-v3 image from the private container registry
Preparing your cluster
Updating the global image pull secret
Manually creating projects (namespaces) for the shared cluster components
Installing shared cluster components
Configuring persistent storage
Setting up IBM Storage Fusion Global Data Platform or IBM Storage Scale Container Native storage
Creating Portworx storage classes
Setting up NFS storage
Configuring your NFS server
Creating an SELinux policy module
Setting up dynamic provisioning
Setting up Amazon Elastic File System
Setting up NetApp Trident
Creating custom SCCs for services
Creating the Db2 SCC
Manually creating the custom SCC
Specifying a custom service account, SCC, role, and role binding
Creating the Db2 Warehouse SCC
Manually creating the custom SCC
Specifying a custom service account, SCC, role, and role binding
Creating the Informix SCC
Creating the SCC for embedded Db2 databases
Changing required node settings
Changing load balancer settings
Changing the process IDs limit
Changing kernel parameter settings
Changing Power settings
Installing prerequisite software
Installing operators for services that require GPUs
Configuring MIG
Installing Multicloud Object Gateway
Installing Red Hat OpenShift Serverless Knative Eventing
Installing App Connect
Preparing to install an instance of Cloud Pak for Data
Manually creating projects (namespaces)
Applying the required permissions to projects (namespaces)
Applying the required permissions by running the authorize-instance-topology command
Applying the required permissions by giving the NamespaceScope operator the minimum RBAC to manage an instance Cloud Pak for Data
Authorizing an instance administrator
Giving a user the admin role
Giving a user the minimum RBAC
Creating secrets for services that use Multicloud Object Gateway
Installing an instance of Cloud Pak for Data
Installing the IBM Cloud Pak foundational services
Installing Cloud Pak for Data
Tethering projects to the control plane
Setting up the control plane
Installing privileged monitors
Installing the configuration admission controller webhook
Installing the RSI webhook
Applying your entitlements
Applying your entitlements without node pinning
Applying your entitlements and pinning pods to nodes
Installing solutions and services
Specifying installation options for services
Specifying the privileges that Db2U runs with
Running a batch installation of solutions and services
Setting up a remote physical location
Setting up environment variables
Updating the global image pull secret
Manually creating projects (namespaces) for the shared cluster components
Installing the shared cluster components
Creating the projects for a remote physical location
Creating a remote physical location
Installing service-specific software
Registering a remote physical location
Creating data planes to organize remote physical locations
Uninstalling the platform and services
Untethering projects
Uninstalling an instance of Cloud Pak for Data
Uninstalling the scheduling service
Upgrading
Planning an upgrade
Upgrading from IBM Cloud Pak for Data Version 4.7
Updating your cluster
Upgrading Red Hat OpenShift
Upgrading your persistent storage
Updating client workstations
Updating the Cloud Pak for Data CLI
Updating the OpenShift CLI
Collecting required information
Determining which components to upgrade
Updating your environment variables script
Preparing to run upgrades in a restricted network
Obtaining the olm-utils-v3 image
Downloading CASE packages
Preparing to run upgrades from a private container registry
Mirroring images to a private container registry
Mirroring images directly to the private container registry
Mirroring images using an intermediary container registry
Pulling the olm-utils-v3 image from the private container registry
Installing prerequisite software
Upgrading Multicloud Object Gateway
Installing Red Hat OpenShift Serverless Knative Eventing
Upgrading shared cluster components
Preparing to upgrade an instance of Cloud Pak for Data
Reauthorizing the instance administrator
Reauthorizing a user with the admin role
Reauthorizing a user with the minimum RBAC
Applying your entitlements
Applying your entitlements without node pinning
Applying your entitlements and pinning pods to nodes
Upgrading an instance of Cloud Pak for Data
Upgrading the IBM Cloud Pak foundational services
Upgrading IBM Cloud Pak for Data
Updating the cpdbr service
Installing privileged monitors
Installing the configuration admission controller webhook
Installing or upgrading the RSI webhook
Upgrading from IBM Cloud Pak for Data Version 4.8
Upgrading your cluster
Upgrading Red Hat OpenShift
Migrating to an image digest mirror set
Upgrading your persistent storage
Updating client workstations
Updating the Cloud Pak for Data CLI
Updating the OpenShift CLI
Collecting required information
Determining which components to upgrade
Updating your environment variables script
Preparing to run upgrades in a restricted network
Obtaining the olm-utils-v3 image
Downloading CASE packages
Preparing to run upgrades from a private container registry
Mirroring images to a private container registry
Mirroring images directly to the private container registry
Mirroring images using an intermediary container registry
Pulling the olm-utils-v3 image from the private container registry
Installing or upgrading prerequisite software
Installing operators for services that require GPUs
Upgrading Multicloud Object Gateway
Upgrading Red Hat OpenShift Serverless Knative Eventing
Upgrading App Connect
Upgrading shared cluster components
Preparing to upgrade an instance of Cloud Pak for Data
Reauthorizing the instance administrator
Reauthorizing a user with the admin role
Reauthorizing a user with the minimum RBAC
Applying your entitlements
Applying your entitlements without node pinning
Applying your entitlements and pinning pods to nodes
Upgrading an instance of Cloud Pak for Data
Upgrading the IBM Cloud Pak foundational services
Upgrading IBM Cloud Pak for Data
Updating the cpdbr service
Upgrading privileged monitors
Installing the configuration admission controller webhook
Installing or upgrading the RSI webhook
Upgrading from IBM Cloud Pak for Data Version 5.0
Updating client workstations
Collecting required information
Determining which components to upgrade
Updating your environment variables script
Preparing to run upgrades in a restricted network
Obtaining the olm-utils-v3 image
Downloading CASE packages
Preparing to run upgrades from a private container registry
Mirroring images to a private container registry
Mirroring images directly to the private container registry
Mirroring images using an intermediary container registry
Pulling the olm-utils-v3 image from the private container registry
Upgrading prerequisite software
Upgrading operators for services that require GPUs
Upgrading Multicloud Object Gateway
Upgrading Red Hat OpenShift Serverless Knative Eventing
Upgrading App Connect
Upgrading shared cluster components
Preparing to upgrade an instance of Cloud Pak for Data
Reauthorizing a user with the minimum RBAC
Applying your entitlements
Applying your entitlements without node pinning
Applying your entitlements and pinning pods to nodes
Upgrading an instance of Cloud Pak for Data
Upgrading the IBM Cloud Pak foundational services
Upgrading IBM Cloud Pak for Data
Updating the cpdbr service
Upgrading privileged monitors
Upgrading the configuration admission controller webhook
Installing or upgrading the RSI webhook
Upgrading remote physical locations
Updating the shared cluster components
Updating the Cloud Pak for Data agents
Updating service-specific software
Services and integrations
Services
AI services
AI Factsheets
Installing
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Uninstalling
IBM Match 360 with Watson
Preparing to install
Installing
Post-installation setup
Modifying the custom resource
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Giving users access to IBM Match 360
Configuring event streaming
Event streaming message reconciliation
Tracing and debugging event streaming
Uninstalling
Orchestration Pipelines
Installing
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Uninstalling
Synthetic Data Generator
Installing
Upgrading
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Uninstalling
Voice Gateway
Installing
Setting installation options for Voice Gateway
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Post installation
Uninstalling
Watson Assistant for Voice Interaction
Setting up
Watson Discovery
Installing
Post-installation setup
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Upgrading
Preparing to upgrade
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Giving users access
Managing security for your data stores
Monitoring the PostgreSQL datastore
Scaling
Uninstalling
Watson Machine Learning
Installing
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Customizing a runtime to use a MIG-enabled profile
Uninstalling
Watson Machine Learning Accelerator
Installing
Post-installation setup
Connecting to Watson Machine Learning
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Uninstalling
Watson OpenScale
Installing
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Configuring container timeouts
Managing users for the Watson OpenScale service
Monitoring Watson OpenScale
Changing environmental variables for the Watson OpenScale service
Retrieving log files for Watson OpenScale
Completing post-restore tasks for an online backup
Resetting status of ongoing model evaluations
Migrating subscriptions
Managing persistent volumes
Changing the data encryption key
Reconciling Watson OpenScale custom resources
Uninstalling
Watson Speech services
Preparing to install
Installing
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Advanced configuration
Upgrading
Preparing to upgrade
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Updating models and voices
Managing security for your datastores
Monitoring the PostgreSQL datastore
Scaling up your installation
Scaling up your datastores
Importing and exporting data
Troubleshooting your installation
Uninstalling
Watson Studio
Installing
Post-installation setup
Setting up your default Watson Studio catalog
Administering
Migrating project and spaces assets
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
watsonx.ai
Preparing to install
Choosing an installation mode
Foundation models
Partitioning GPU processors
Installing
Post-installation setup
Adding foundation models
Adding custom foundation models to lightweight engine
Getting started with lightweight engine
Upgrading
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Creating network policies
Controlling prompt tuning availability
Managing model sharding configuration
Removing foundation models
Uninstalling
watsonx Assistant
Installing
Post-installation setup
Enabling watsonx Assistant to process API requests on multiple clusters
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Testing the installation
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Giving users access
Managing security for your data stores
Enabling or disabling Analytics
Enabling or disabling Postgres Sync Replication
Enabling or disabling GPU features
Monitoring
Uninstalling
watsonx Code Assistant for Red Hat Ansible Lightspeed
Installing
Upgrading
Upgrading from Version 5.0
Administering
Metering
Uninstalling
watsonx Code Assistant for Z
Installing
Upgrading
Upgrading from Version 5.0
Administering
Metering
Uninstalling
watsonx Code Assistant for Z Code Explanation
Installing
Post-installation setup
Administering
Metering
Uninstalling
watsonx.governance
Installing
Post-installation setup
Upgrading
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Uninstalling
watsonx Orchestrate
Installing
Post-installation setup
Applying hot fix for Version 5.0.3
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Upgrading
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Giving users access
Monitoring
Uninstalling
Analytics services
Analytics Engine powered by Apache Spark
Installing
Post-installation setup
Enabling installation options after installation or upgrade
Specifying additional configurations
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Administering
Using advanced features
Managing instances
Managing access to instances
Managing service and instance level configurations
Changing instance resource quota
Changing the Spark default runtime
Changing cron job configurations
Implementing runtime retention policy
Upgrading
Preparing to upgrade Analytics Engine powered by Apache Spark
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Cognos Analytics
Installing
Post-installation setup
Optional: Applying role bindings to the tethered project
Optional: Adding a network policy
Configuring the content store
Creating administration scripts
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Administering
Creating a logging database and setting up audit reporting
Reviewing the security settings
Managing users
Copying SSL certificates
Managing deployments
Configuring Framework Manager for Cognos Analytics
Updating the OpenShift secret for a password change
Updating the secret when the external content store password changes
Updating the secret when the audit password changes
Updating the secret when the notification (SMTP) password changes
Data catalogs
Backing up and restoring
Backing up the service
Restoring the service
Changing the plan size
Changing the time zone
Enabling long data names in prompts
Managing artifacts with APIs
Setting up Cognos PowerCubes
Creating a Cognos PowerCubes PVC
Adding the Cognos PowerCubes PVC
Copying Cognos PowerCubes files and folders
Removing Cognos PowerCubes
Deleting Cognos PowerCubes with the project
Migrating data between clusters
Managing images
Adding an image
Downloading an image
Listing existing images
Deleting an image
Creating a soft link
Listing existing soft links
Deleting a soft link
Managing styles in reports
Data Refinery
Installing
DataStage
Installing
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Administering
Creating a service instance
Deploying on a remote data plane
Creating a service instance from the web client
Creating a service instance programmatically
Creating a service instance with cpd-cli service-instance create
Runtime storage instance isolation
Customizing resources
Customizing hardware configurations
Enabling multiple conductor pods
Configuring memory and heap size in DataStage
Modifying the ephemeral storage limit
Changing the default umask
Runtime storage layout
Storage-level isolation
Setting environment definitions
Setting up an NFS mount
Dynamic workload management
Enabling auto-scaling
Configuring workload management policies
Changing values in the dynamic configuration file
Creating and setting the APT_CONFIG_FILE
Job log retention
Storing and persisting metrics
Installing third-party libraries and creating custom images
Building custom images
Accessing Azure storage with the AzCopy CLI
Adding local scratch devices
Enabling the Data service connector for DataStage Enterprise Plus
Db2 Big SQL
Db2 Big SQL architecture
Security
Installing
Post-installation setup
Optional: Configuring dedicated OpenShift worker nodes
Optional: Preparing to connect Db2 Big SQL to an IBM Spectrum Scale object store
Preparing to provision a Db2 Big SQL instance
Preparing to connect Db2 Big SQL to a remote Hadoop cluster
Creating a service instance
Creating a service instance from the web client
Specifying resources for the head and worker nodes
Specifying resources for persistent storage
Setting up a connection to a remote data source
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Post-provisioning tasks
Configuring Db2 Big SQL to use datanode hostnames when connecting to the datanodes on the Hadoop cluster
Configuring Db2 Big SQL to connect to an IBM Spectrum Scale object store
Connecting to a Hadoop cluster with a manually managed Kerberos configuration
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Running the health check command
Synchronize the Db2 Big SQL catalog and the Hive metastore
Manually synchronizing the Db2 Big SQL catalog and the Hive metastore
Automatically synchronizing the Db2 Big SQL catalog and the Hive metastore
Setting up a connection
Managing access to Db2 Big SQL instances
Configuring a Db2 Big SQL instance
Scaling Db2 Big SQL
Updating the Db2 Big SQL and Hadoop configuration
Enabling or disabling impersonation
Enabling database auditing
Monitoring the service
Uninstalling
Decision Optimization
Installing
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Uninstalling
Execution Engine for Apache Hadoop
Installing
Post-installation setup
Installing on Apache Hadoop clusters
Template parameters
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Post-upgrade tasks
Administering
Establishing connection to Apache Hadoop clusters
Managing connection to Apache Hadoop clusters
Establishing connection to Cloud Pak for Data clusters
Managing connection to Cloud Pak for Data clusters
High availability
Uninstalling
Planning Analytics
Installing
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Modifying the default route
Increasing the DNS buffer size
Downloading and installing Planning Analytics for Microsoft Excel
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Managing users
Adding users in Cloud Pak for Data
Adding a user to the TM1 database
Setting an idle web session timeout
Migrating data between instances
Selecting regions and zones
Backing up and restoring
Backing up an instance
Restoring the instance
Uninstalling
SPSS Modeler
Installing
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Cleaning configuration files
Uninstalling
Dashboard services
Cognos Dashboards
Installing
Post-installation setup
Disabling mapbox visualization for air-gapped client workstations
Optional: Adding a custom TLS certificate
Optional: Disabling session affinity
Setting up permissions
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Administering
Tracking license usage
Data governance services
Data Privacy (Masking flow)
Installing
Administering
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Data Product Hub
Installing
Post-installation setup
Upgrading
Upgrading from Version 5.0
Administering
Managing storage
Managing the Data Product Hub community
Managing data source connections for Data Product Hub
Roles and permissions for Data Product Hub
Uninstalling
IBM Knowledge Catalog
Preparing to install
Installing
Post-installation setup
Enabling optional features after installation or upgrade
Setting up your default catalog
Installing the patch for version 5.0.3
Administering
Migrating governance artifacts
Migrating catalog assets
Configuring permissions for cross-cluster import operations
Migrating data protection rules
Custom properties, relationships, and asset types
Creating custom asset types
Creating custom properties
Creating custom relationships
Importing custom properties or relationships from a file
Managing custom properties, relationships, and asset types
Adding custom abbreviation files for use in metadata enrichment
Enabling validation of the business start date on CSV import
Enabling lineage import
Uploading JDBC drivers for lineage import
Configuring Hive with Kerberos for lineage imports
Importing a certificate for metadata imports from Cognos Analytics
Managing rule settings
Renewing the Db2 SSL certificate after the Cloud Pak for Data self-signed certificate is updated (IBM Knowledge Catalog)
Scaling up Db2 for IBM Knowledge Catalog
Enable queries for total record counts for profiling
Migrating profiling results
Bulk sync assets for global search
Bulk sync relationships for global search
Resync of lineage metadata
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Upgrading to IBM Knowledge Catalog Premium
Migrating
Migrating data from IBM InfoSphere Information Server to IBM Cloud Pak for Data
Known issues for migration from Information Server
What can be migrated to Cloud Pak for Data
Where you can find the migrated data
Prerequisites to migrating data
Preparing for migration in IBM InfoSphere Information Server (Linux, AIX)
Upgrading your IBM Db2 client instance
Preparing for migration in IBM InfoSphere Information Server (Windows)
Preparing for migration in IBM Cloud Pak® for Data
Running the export in inspect mode
Migrating users and groups
Exporting data from the Information Server system
Importing the data to the target system
Verifying migrated content
Post-migration tasks
Troubleshooting migration
Uninstalling
MANTA Automated Data Lineage
Installing
Administering
Enabling lineage import
Uploading JDBC drivers for lineage import
Configuring Hive with Kerberos for lineage imports
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Product Master
Preparing to install
Preparing for fresh installation
Enabling single sign-on
Preparing for upgrade
Installing OpenSearch
Installing
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli manage apply-cr
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Getting application URL
Enabling additional features
Creating database schema
Customizing Product Master
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Administering
Creating a company
Assigning roles
Scaling instances
Upgrading specific images
Enabling external MQ service
Accessing Machine learning APIs
Installing MongoDB
Data source services
Data Gate
Preparing to install
Data Gate components
Data Gate system requirements
High-availability
Configuring network access between Data Gate and Z
Running AQTSSLDG to complete network setup steps
Installing existing certificates
Configuring inbound access to Db2 for z/OS
Defining a secure network port for connections to Data Gate
AT-TLS configuration
Modifying the RACF profile
Creating certificates to encrypt the connection between Db2 for z/OS and Data Gate
Creating a RACF key ring
Defining TTLS rules for connections to Data Gate
Defining a TTLS rule for a data sharing group
Encrypting outbound network traffic
Creating a RACF® profile with appropriate permissions
Generating and exporting a key pair and a certificate for Data Gate
Configuring the Policy Agent on IBM Z
Network configuration example
Installing Data Gate on Z
Installing the Data Gate backend on z/OS
Installing Db2 maintenance to enable Data Gate
Setting Db2 for z/OS subsystem parameters
Creating the Data Gate database and tables in Db2 for z/OS
Creating a WLM environment
Verifying the setup of Db2-supplied stored procedures for Data Gate
Defining WLM performance goals
Creating Data Gate stored procedures
Support for schema changes
Creating users and granting privileges on z/OS
Installing a Db2 instance for Data Gate on Cloud Pak for Data
Installing
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Changes to the target database
Upgrading
Steps before upgrading the instance
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Fixing incorrect Db2 configuration settings
Migrating the SSL certificate
Administering
Managing user access
Viewing instance details
Updating a source definition
Monitoring a Data Gate instance
Adding tables
Importing tables
Enabling or disabling table synchronization
Loading tables
Avoiding table state ERROR
Creating custom indexes
Preserving custom indexes
Changing the target database configuration
Scaling up the target database
Scaling the Data Gate instance
Removing tables
Removing an instance
Updating certificates
Updating Db2 for z/OS certificate
Replacing certificates for Data Gate and the target database
Updating passwords
Updating Db2 for z/OS user passwords
Refreshing the target database secret
Making queries wait for synchronization updates
Query acceleration
Configuring query acceleration
Submitting queries for acceleration
Storage saver
Archiving data
Querying archived data
Restoring data
Integrating IBM Knowledge Catalog
Configuring publishing to IBM Knowledge Catalog
Publishing table metadata
Stored procedures
Collecting stored procedure traces
Daisy chain synchronization
Uninstalling
Data Gate APIs
Authentication
Retrieving the status
Synchronization events
Data Replication
Preparing to install
Non-containerized data stores
Containerized data stores
Installing
Administering
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Data Virtualization
Preparing to install
Installing
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Optional: Configuring dedicated OpenShift worker nodes
Configuring network requirements
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Administering instances
Connecting and authenticating to the service
Enabling the Db2 REST interface
Connecting applications to Data Virtualization
Managing roles for users and groups
Assigning roles to users
Assigning roles to groups
Modifying user or group roles
Revoking data access authority from Admin role
Scaling Data Virtualization
Securing Data Virtualization
Fetch phase warnings and errors
Enabling and disabling fetch phase warnings and errors
Enabling and disabling pass-through authentication
Enabling Kerberos authentication
Enabling autocaching to automate the caching lifecycle
Copying secrets and configmaps to tethered projects
Setting the Federation server options
Setting the query mode
Setting the storage class for caching
Auditing in Data Virtualization
Monitoring user activity in Data Virtualization
Enabling and disabling auditing in Data Virtualization
Customizing an audit policy in Db2 audit facility
Db2 audit events for Data Virtualization
Uninstalling
Db2
Preparing to install
Db2 compatible versions for Cloud Pak for Data versions
Setting up dedicated nodes
Configuring database storage
Creating separate storage for archive logs in Db2
Creating separate storage for database transaction logs
Creating separate storage for temporary table spaces
Installing
Post-installation setup
Deploying with limited privileges
Adding privileges to deploy Db2 with the console
Upgrading the license before you deploy Db2
Creating a database deployment
Configuring tethered namespaces
Advanced configuration for demanding workloads
Disabling database connection validation after installation or upgrade
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Upgrading deployments with an HADR configuration
Upgrading the external etcd store created with the etcd image
Upgrading Db2 11.5.7-cn4 and earlier HADR deployments
Administering
Exec into the pod
Cataloging the database
Connecting applications to the database server
Connecting with a Cloud Pak for Data user ID
User credentials and supported authentication methods
Giving users access to the database
Connecting to Db2 ports and services
Connecting clients to the HADR configuration
Configuring TLS client connections
Changing Db2 configuration settings
Setting a timezone for new or existing deployments
Upgrading the license of a deployed service
Enabling the Db2 Spatial Extender
Managing multiple databases on a deployment
Q Replication
Deploying Q Replication
Use cases
Enabling Q Replication
Planning for data Q Replication
Choosing the replication key
Considerations for RI constraints
Replicating column-organized tables
Workload and database considerations
Data type restrictions
Supported DDL operations
Creating tables at the target database
Initial loading of replication targets
Disabling Q Replication
Deploying Q Replication by using Helm
Upgrading Q Replication by using Helm
Uninstalling Q Replication by using Helm
Troubleshooting Q Replication
Administering Q Replication
Accessing the Q Replication REST API
Activating Q Replication
Configure targets and replication sets
Trusting targets and exchanging Db2 TLS certificates
Initializing the replication target on the replication source
Creating a replication set
Adding a table to a replication set
Deleting tables
Deleting a replication set
Deleting replication targets
Managing replication
Managing replication using shell scripts
Deploying replication
Configuring replication
Operating replication
Monitoring replication
Reverse Q replication configuration and operation
Listing replication sets
Listing replication set details
Starting replication on a replication set
Stopping replication on a replication set
Starting replication for a list of tables
Stopping replication for a list of tables
Setting the REPL_SITE_ID
Schema-level restore of replicated column-organized tables
Checking the overall health of Q Replication API servers
Checking the health metrics of Q Replication API
Replication in Db2 for containerized deployments
Db2 Warehouse Common Container to Db2 Warehouse RHOS
Db2 Warehouse RHOS to Db2 Warehouse Common Container
Extending the MQ security certificate
Updating external source ports
Updating external target ports
Upgrading the trial license
Scaling up Q Replication
Db2 high availability disaster recovery (HADR)
Prerequisites for configuring HADR
Configuring and starting HADR
Setting up an etcd store for HADR
Deploy etcd packaged with the installation
Deploy etcd packaged by Bitnami
Using the built-in etcd store
HADR with multiple standbys
Configuring shared log archiving
Configuring the HADR API
Using the HADR API
Using a shared volume for backup storage
Configuring HADR with scripts
Creating services to expose the HADR endpoints
Creating HADR services for a single cluster topology
Creating HADR services for a multiple cluster topology
Determining the NodePorts for a multiple cluster configuration
Setting up Ingress HA proxy for different OpenShift clusters
Creating a network policy for built-in etcd
Setting up the HADR configuration
setup_config_hadr script
Example: Setting up HADR in a single OpenShift project
Example: Setting up HADR in a single OpenShift cluster across multiple OpenShift projects
Example: Setting up HADR in multiple OpenShift clusters
Example: Setting up HADR in a single IBM Cloud cluster
Modifying HADR configuration parameters
Enabling reads on standby
Starting an HADR configuration
Reinitializing an HADR configuration to resolve errors
Managing HADR with CLI commands
Showing the HADR status
Manually initiating an HADR takeover
Restarting HADR after a failover
Restarting HADR on the designated auxiliary standby database
Restarting HADR on the primary or principal standby databases after failover to an auxiliary standby database
Stopping HADR
Enabling HADR Automatic Client Reroute
Updating the password secrets
Locating the keystore
Db2 SCC capabilities
Updating the SSL certificate after the Cloud Pak for Data self-signed certificate is updated
Audit logging
Log streaming
Backing up and restoring
Enabling automatic backup
Backing up
Using Db2 container commands
Using cpd-cli backup-restore
Backing up with a custom resource
Online backup
Offline backup
Copying a backup to a different instance or remote cluster
Restoring
Snapshot restore
Restoring from a snapshot backup with Db2 container commands
Restoring Db2 with a custom resource
Online backup
Offline backup
Using the restore script
Using Db2 commands
Deleting a custom resource for backing up or restoring
Backing up and restoring with IBM Spectrum Protect (TSM)
Scaling up
Changing your Db2 deployment to use dedicated resources
Stopping and starting a Db2 instance
Shutting down and restarting an instance
Support
Using the db2support tool
Viewing log files
Using the OpenShift must-gather utility
Node failure behavior and expectations
Managing transaction logs
Managing diagnostic directories
Running admin tasks from the command-line interface with ATS
Monitoring
Loading data
Uninstalling
Db2 REST service
Enabling the Db2 REST interface
Creating the route for the Db2 REST service
Authenticating REST commands
Setting up Db2 REST server metadata
Using the Db2 REST interface
Updating ephemeral storage
Db2 Data Management Console
Installing
Db2 Data Management Console pods
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Managing user access
Updating the database password secrets
Deleting a Db2 Data Management Console instance
Uninstalling
Db2 Warehouse
Preparing to install
Db2 Warehouse compatible versions for Cloud Pak for Data versions
Network requirements for MPP configurations
Setting up dedicated nodes
Configuring database storage
SELinux requirements
Requirements for OpenShift Data Foundation
Requirements for Portworx
Creating separate storage for archive logs in Db2 Warehouse
Creating separate storage for database transaction logs
Creating separate storage for temporary table spaces
Installing
Post-installation setup
Deploying with limited privileges
Creating an inter-pod communication network
Creating a database deployment
Configuring tethered namespaces
Advanced configuration for demanding workloads
Disabling database connection validation after installation or upgrade
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Upgrading deployments with an HADR configuration
Upgrading the external etcd store created with the etcd image
Upgrading Db2 Warehouse 11.5.7-cn4 and earlier HADR deployments
Administering
Exec into the pod with catalog partition
Cataloging the database
Connecting applications to the database server
Connecting with a Cloud Pak for Data user ID
User credentials and supported authentication methods
Giving users access to the database
Connecting to Db2 Warehouse ports and services
Connecting clients to the HADR configuration
Configuring TLS client connections
Changing Db2 Warehouse configuration settings
Setting a timezone for new or existing deployments
Q Replication
Deploying Q Replication
Use cases
Enabling Q Replication
Planning for data Q Replication
Choosing the replication key
Considerations for RI constraints
Replicating column-organized tables
Workload and database considerations
Data type restrictions
Supported DDL operations
Creating tables at the target database
Initial loading of replication targets
Disabling Q Replication
Deploying Q Replication by using Helm
Upgrading Q Replication by using Helm
Uninstalling Q Replication by using Helm
Troubleshooting Q Replication
Administering Q Replication
Accessing the Q Replication REST API
Activating Q Replication
Configure targets and replication sets
Trusting targets and exchanging Db2 TLS certificates
Initializing the replication target on the replication source
Creating a replication set
Adding a table to a replication set
Deleting tables
Deleting a replication set
Deleting replication targets
Managing replication
Managing replication using shell scripts
Deploying replication
Configuring replication
Operating replication
Monitoring replication
Reverse Q replication configuration and operation
Listing replication sets
Listing replication set details
Starting replication on a replication set
Stopping replication on a replication set
Starting replication for a list of tables
Stopping replication for a list of tables
Setting the REPL_SITE_ID
Schema-level restore of replicated column-organized tables
Checking the overall health of Q Replication API servers
Checking the health metrics of Q Replication API
Replication in Db2 Warehouse for containerized deployments
Db2 Warehouse Common Container to Db2 Warehouse RHOS
Db2 Warehouse RHOS to Db2 Warehouse Common Container
Extending the MQ security certificate
Updating external source ports
Updating external target ports
Upgrading the trial license
Scaling up Q Replication
Db2 Warehouse high availability disaster recovery (HADR)
Prerequisites for configuring HADR
Configuring and starting HADR
Setting up an etcd store for HADR
Deploy etcd packaged with the installation
Deploy etcd packaged by Bitnami
Using the built-in etcd store
HADR with multiple standbys for Db2 Warehouse
Configuring shared log archiving
Configuring the HADR API
Using the HADR API
Using a shared volume for backup storage
Configuring HADR with scripts
Creating services to expose the HADR endpoints
Creating HADR services for a single cluster topology
Creating HADR services for a multiple cluster topology
Determining the NodePorts for a multiple cluster configuration
Setting up Ingress HA proxy for different OpenShift clusters
Creating a network policy for built-in etcd
Setting up the HADR configuration
setup_config_hadr script
Example: Setting up HADR in a single OpenShift project
Example: Setting up HADR in a single OpenShift cluster across multiple OpenShift projects
Example: Setting up HADR in multiple OpenShift clusters
Example: Setting up HADR in a single IBM Cloud cluster
Modifying HADR configuration parameters
Enabling reads on standby
Starting an HADR configuration
Reinitializing an HADR configuration
Managing HADR with CLI commands
Showing the HADR status
Manually initiating an HADR takeover
Restarting HADR after a failover
Restarting HADR on the designated auxiliary standby database
Restarting HADR on the primary or principal standby databases after failover to an auxiliary standby database
Stopping HADR
Enabling HADR Automatic Client Reroute
Security
Updating the password secrets
Locating the keystore
Db2 Warehouse SCC capabilities
Required roles and permissions
Updating the SSL certificate after the Cloud Pak for Data self-signed certificate is updated
Audit logging
Log streaming
Backing up and restoring
Enabling automatic backup
Backing up
Using Db2 Warehouse container commands
Using cpd-cli backup-restore
Backing up with a custom resource
Online backup
Offline backup
Copying a backup to a different instance or remote cluster
Restoring
Restoring from a snapshot backup with Db2 Warehouse container commands
Performing a snapshot restore with Db2 container commands
Restoring Db2 Warehouse with a custom resource
Online backup
Offline backup
Using the restore script
Using commands
Backing up and restoring with IBM Spectrum Protect (TSM)
Scaling
Scaling in an installation
Scaling up
Changing your Db2 Warehouse deployment to use dedicated resources
Stopping and starting
Shutting down and restarting an instance
Support
Using the db2support tool
Viewing log files
Using the OpenShift must-gather utility
Node failure behavior and expectations
Managing diagnostic directories
Managing transaction logs
Running admin tasks from the command-line interface with ATS
Monitoring
Loading data
Uninstalling
Db2 REST service
Enabling the Db2 REST interface
Creating the route for the Db2 Warehouse REST service
Authenticating REST commands
Setting up Db2 Warehouse REST server metadata
Using the Db2 Warehouse REST interface
EDB Postgres
Preparing to install
Acquiring a license key
Installing
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Deploying with a custom resource
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Setting up access to service instances
Giving users access to the database
Creating a custom EDB Postgres user name and password
Performing physical backup and restore
Setting up continuous backup and restore
Configuring TLS with a custom resource
Uninstalling
Informix
Preparing to install
Setting up dedicated nodes for your Informix deployment
Installing
Post-installation setup
Creating a service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Connecting to Informix
Configuring TLS connections with Informix
Custom TLS certificate
Retrieving the Informix port number
Configuring the Informix NodePort with an Ingress controller
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
The in4mix and admin user passwords
Managing Informix users
Giving users access to the Informix database
Creating Informix user IDs that are not Cloud Pak for Data user IDs
Exec into the Informix pod
Stopping and starting an Informix instance
Restarting the Informix cluster
Monitoring Informix startup after rebooting a node
Security in Informix
Control Informix audit logging
Viewing Informix log files
Migrating data between clusters
Viewing Informix Data Management Console
Accessing the Informix Data Management Console
Monitoring database instance
Exploring databases
Exploring database summary
Exploring tables of the database
Exploring table summary
Running SQL statements
Managing sensors
Managing alerts
Monitoring sessions
Monitoring dbspaces
Monitoring checkpoints
Monitoring memory
Monitoring Online logs
Monitoring incidents
Changing the Informix license
Changing the license before deploying Informix
Changing the license after the Informix service instance creation
Uninstalling
MongoDB
Preparing to install
Setting up dedicated nodes
Installing
Post-installation setup
Provisioning a MongoDB Ops Manager service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Deploying with a custom resource
Provisioning a MongoDB database service instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Deploying with a custom resource
Upgrading
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Giving users access to the database
Administering with Ops Manager
Configuring backup
Restoring data from a backup
Configuring TLS
Configuring TLS using an internet connection
Configuring TLS without an internet connection
Uninstalling
watsonx.data
Installing
Post-installation setup
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Adding users
Uninstalling
Developer tool services
Anaconda Repository for IBM Cloud Pak for Data
Installing
Administering
Configuring conda and creating channels
Upgrading
RStudio Server Runtimes
Installing
Administering
Upgrading
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Uninstalling
Watson Studio Runtimes
Upgrading Watson Studio Runtimes
Upgrading from Version 4.7
Upgrading from Version 4.8
Upgrading from Version 5.0
Runtime 24.1 on Python 3.11
Administering
Uninstalling
Runtime 23.1 on Python 3.10
Installing
Administering
Uninstalling
Runtime 24.1 on Python 3.11 for GPU
Installing
Administering
Configuring multiple MIG profiles within a cluster
Uninstalling
Runtime 23.1 on Python 3.10 for GPU
Installing
Administering
Configuring multiple MIG profiles within a cluster
Uninstalling
Runtime 24.1 on R 4.3
Installing
Administering
Uninstalling
Runtime 23.1 on R 4.2
Installing
Administering
Uninstalling
Industry solutions services
OpenPages
Preparing to install
Setting up an external Db2 database
Setting up an external Oracle database
Installing
Post-installation setup
Creating a service instance
Preparing to provision an OpenPages instance
Creating a service instance from the web client
Creating a service instance with cpd-cli service-instance create
Creating a service instance programmatically
Post-provisioning setup
Post-provision setup for Loss Event Entry
Installing tools and utilities for OpenPages
Upgrading
Upgrading from Version 4.7
Replacing computed fields
Updating MRG and AI Factsheets
Updating the Knowledge Catalog import job
Updating the reporting schema
Importing reports and generating the framework
Upgrading from Version 4.8
Upgrading from Version 5.0
Administering
Configuring
Setting session inactivity timeout values for application servers
Changing the password of the Liberty keystore
Changing the settings for ORM
Managing users and groups
Adding users
Managing access to OpenPages instances
How users and groups are synchronized from Cloud Pak for Data to OpenPages
Configuring the user and group synchronization job
Checking running synchronization jobs
Changing the password of the OPSystem user
Cognos Analytics and OpenPages
Adding Cognos to OpenPages
Reporting framework
Administering an external database for OpenPages
Logging in to the database
Audit logging
Network security and the OpenPages service
Changing the data encryption key
Updating RabbitMQ credentials
Scaling options
Scaling the application server pod
Scaling the database server pod
Starting and stopping application servers
Backing up, restoring, and migrating
Backing up custom files
Migrating OpenPages to another instance
Migrating to another instance with an internal database
Migrating to another instance with an external Db2 database
Migrating to another instance with an external Oracle database
Migrating to Cloud Pak for Data (Oracle)
Troubleshooting backup and restore
Advanced configuration
Files and directories
Log files
Gathering logs with the LogCollector user interface
Uninstalling
Storage services
Industry accelerators
Integrations
Administering
Post-installation setup (Day 1)
Tracking and reporting use against license terms
Securing your environment
Changing the admin user password
Applying cluster HTTP proxy settings
Customizing and securing the route to the platform
Using custom TLS certificates to connect to the platform
Modifying the route
Turning off the host header injection check
Integrating with the Identity Management Service
Changing the cpadmin user to admin
Configuring Cloud Pak for Data to use the embedded LDAP integration
Configuring SSO
Configuring vault usage
Disabling the internal vault for the Cloud Pak for Data web client
Requiring users to use secrets for credentials when creating connections
Disabling secret sharing for Cloud Pak for Data vaults
Setting up security for connections
Configuring an external route to the Flight service
Enabling connections to use Kerberos authentication
Enabling connections to use Kerberos SSO authentication
Enabling users to use JDBC URLs stored in secrets
Enabling users to upload JDBC drivers
Managing certificates
Creating a secret to store shared custom certificates
Using custom TLS certificates to connect to the platform
CA certificate to connect to internal servers
Disabling shared credentials
Setting the idle session timeout
Limiting the number of concurrent user sessions
Limiting user information returned by API calls
Displaying a terms and conditions prompt
Monitoring user activity
Adding descriptions to user activity monitoring logs
Exporting audit records to an SIEM solution
Exporting audit records to Splunk
Exporting audit records to QRadar
Exporting audit records to Mezmo
Exporting audit records to Apache Kafka
Exporting audit records to the cluster
Enabling auditing for the Identity Management Service
Securing communication ports
Installing monitors
Installing service monitors
Installing privileged monitors
Setting up the platform for end users
Restricting the list of storage classes
Enabling email notifications
Enabling users to access the web client from emails
Enabling users to connect to external SMB storage volumes
Customizing the branding of the web client
Customizing the home page
Managing tours
Setting up a storage volume to store customizations for common core services
Disabling SELinux relabeling
Collecting telemetry data
Installing the IBM Metrics Operator
Installing the IBM Data Reporter Operator
Enabling telemetry data collection
Forwarding alerts
Forwarding alerts to email accounts
Forwarding alerts to an SNMP server
Forwarding alerts to Slack
Forwarding notifications
Forwarding notifications to email
Forwarding notifications to external applications
Moving cpd-cli images to a private container registry
Setting up services
Ongoing maintenance (Day 2)
Best practices for monitoring
Performing routine IBM Cloud Pak for Data monitoring
Setting up quotas and alerts
Performing routine cluster monitoring
Integrating with Prometheus
Exporting Cloud Pak for Data metrics to Prometheus
Running Prometheus queries from the command line
Running Prometheus queries from the OpenShift Console
watsonx Assistant
watsonx Orchestrate
Monitoring the available capacity of the global search persistent volume
Monitoring the platform
Getting the status of installed components
Changing the retention period for monitoring data
Monitors in Cloud Pak for Data
Monitoring and alerting
Setting up profiles
Setting up alerting rules
Configuring SMTP alerts
Configuring SNMP alerts
SNMP response codes
Configuring Slack alerts
Custom monitors
Example: Creating a PVC custom monitor
Cloud Pak for Data custom monitors
Monitoring objects by using labels
Creating announcements
Audit events
Cloud Pak for Data platform
Sample Cloud Pak for Data CADF Audit Records
AI Factsheets
Analytics Engine powered by Apache Spark
Cognos Analytics
Cognos Dashboards
Data Product Hub
Data Privacy
Data Refinery
DataStage
Data Virtualization
Db2 Big SQL
Db2 Data Management Console
Decision Optimization
IBM Knowledge Catalog
IBM Match 360 with Watson
Informix
OpenPages
Orchestration Pipelines
Planning Analytics
RStudio Server Runtimes
SPSS Modeler
Synthetic Data Generator
Watson Machine Learning
Watson Machine Learning Accelerator
Watson OpenScale
Watson Speech services
Watson Studio
watsonx.ai
watsonx.data
Managing resources
Scaling services
Manually scaling services
Automatically scaling services
Shutting down and restarting services
Shutting down services
Restarting services
Increasing nginx worker connections
Adding nodes to your cluster
Configuring node scoring for the scheduling service
Customizing pod specifications with RSI
Creating an RSI patch
Migrating data between instances
Services that support export-import
Preparing to use the export and import utility
Migrating data between installations
Managing storage volumes
Cleaning up storage volume resources
Managing secrets and vaults
Vaults and secrets APIs
Connecting to external vaults
Adding references to secrets in external vaults
Adding secrets to the internal vault
Administering connectors
Creating a custom JDBC connector
Supported services and tools for custom JDBC connectors
Managing custom JDBC connectors
Importing JDBC driver files
Increasing the file size limits for JDBC driver file uploads
Gathering diagnostic information
Changing the retention period for diagnostic logs
Managing users
Connecting to your identity provider
Enabling authentication against a custom authentication service
Predefined roles and permissions in Cloud Pak for Data
Managing roles in Cloud Pak for Data
Managing user groups
Specifying additional attributes for dynamic user groups
Using attributes from your IdP to create dynamic user groups
Adding the attributes to the Identity Management Service
Specifying which custom attributes to use
Using a custom attribute provider to specify additional attributes
Creating a custom attribute provider
Connecting to your custom attribute provider
Specifying which custom attributes to use
Managing access to the platform
Monitoring user activity
Disabling the default platform administrator
Backing up and restoring Cloud Pak for Data
Services that support backup and restore
Installing the backup and restore utilities
Setting up a client workstation
Installing the Cloud Pak for Data CLI
Installing the OpenShift CLI
Installing and configuring the OADP backup and restore utility
Moving images for the backup and restore utilities
Installing Cloud Pak for Data OADP backup and restore utility components
Installing the jq JSON command-line utility
Downloading the backup and restore scripts
Configuring the Cloud Pak for Data OADP backup and restore utility
Creating volume snapshot classes
Installing the OADP backup REST service
Installing the cpdbr service for IBM Storage Fusion integration
Installing the cpdbr service for NetApp Astra Control Center integration
Installing the cpdbr service for Portworx integration
Installing the Cloud Pak for Data volume backup and restore utility
Preparing the volume backup and restore utility
Setting the Cloud Pak for Data OADP backup and restore mode
Online backup and restore
Online backup and restore to the same cluster
Backup and restore with the Cloud Pak for Data OADP backup and restore utility
Backing up an instance of Cloud Pak for Data
Backup prerequisite tasks
Optional: Estimating how much storage to allocate for backups
Backing up the Cloud Pak for Data scheduling service
Creating an online backup
Restoring an instance of Cloud Pak for Data
Preparing to restore Cloud Pak for Data
Optional: Restoring the Cloud Pak for Data scheduling service
Restoring an online backup to the same cluster
Validating the restored Cloud Pak for Data instance
Post-restore tasks
Backup and restore with IBM Storage Fusion
Preparing to back up Cloud Pak for Data
Creating and scheduling online backups
Restoring an online backup to the same cluster
Post-restore tasks
Backup and restore with NetApp Astra Control Center
Preparing to back up Cloud Pak for Data
Backing up the scheduling service
Creating online backups
Optional: Restoring the scheduling service
Restoring an online backup to the same cluster
Post-restore tasks
Online backup and restore to a different cluster (disaster recovery)
Backup and restore with IBM Storage Fusion
Preparing to back up Cloud Pak for Data
Creating and scheduling online backups
Recovering Cloud Pak for Data
Preparing the target cluster
Updating node configurations
Restore prerequisite tasks
Restoring an online backup on the target cluster
Post-restore tasks
Backup and restore with NetApp Astra Control Center
Preparing to back up Cloud Pak for Data
Backing up the scheduling service
Creating online backups
Recovering Cloud Pak for Data
Preparing the target cluster
Updating node configurations
Restore prerequisite tasks
Restoring the scheduling service
Restoring an online backup on the target cluster
Post-restore tasks
Data replication with Portworx
Preparing the source cluster
Preparing the destination cluster
Configuring Portworx on the destination cluster
Configuring Portworx on the source cluster
Preparing to migrate Cloud Pak for Data projects
Migrating Cloud Pak for Data projects
Activating applications in the destination cluster
Post-restore tasks after activating applications
Offline backup and restore (OADP utility)
Offline backup and restore to the same cluster
Backing up an instance of Cloud Pak for Data
Backup prerequisite tasks
Excluding external volumes from backups
Optional: Estimating how much storage to allocate for backups
Backing up the Cloud Pak for Data scheduling service
Creating an offline backup
Post-backup tasks
Restoring an instance of Cloud Pak for Data
Preparing to restore Cloud Pak for Data
Optional: Restoring the Cloud Pak for Data scheduling service
Restoring an offline backup to the same cluster
Validating the restored Cloud Pak for Data instance
Post-restore tasks
Offline backup and restore to a different cluster
Backing up an instance of Cloud Pak for Data
Backup prerequisite tasks
Excluding external volumes from backups
Optional: Estimating how much storage to allocate for backups
Backing up the Cloud Pak for Data scheduling service
Creating an offline backup
Post-backup tasks
Restoring an instance of Cloud Pak for Data
Restore prerequisite tasks
Restoring the Cloud Pak for Data scheduling service
Restoring an offline backup to a different cluster
Validating the restored Cloud Pak for Data instance
Post-restore tasks
Scenario: Backing up and restoring Db2 Warehouse separately from Cloud Pak for Data
Offline backup and restore (volume backup utility)
Backing up volumes
Backup prerequisite tasks
Creating volume snapshots
Backing up volumes to a persistent volume claim or object store
Restoring volumes
Restoring volume snapshots
Restoring Cloud Pak for Data volume backups from a persistent volume claim or object store
Restoring Watson Studio
Restoring Watson Machine Learning
Uninstalling
Uninstalling OADP and Velero
Uninstalling the REST server
Uninstalling the cpdbr service
Cloud Pak for Data command-line interface (cpd-cli)
Installing the cpd-cli
Creating a cpd-cli profile
Gathering diagnostics
Managing services instances
Managing users
Projects
Creating a project
Importing a project
Importing project assets
Accessing a Git repository
Creating access token for collaboration
Configuring custom encryption for access tokens
Choosing a tool
Integrating with Git repositories
Default Git integration
Git operations
Project terminal
Connecting to data sources
Deprecated Git integration
Syncing projects
Syncing when working remotely
Administering projects
Managing collaborators
Project collaborator roles
Marking a project as sensitive
Logging project activity
Exporting project assets
Enabling folders (beta)
Managing assets in projects
Organizing assets with folders (beta)
Environments
Migrating Runtime 22.2 environments from Cloud Pak for Data 4.7 or 4.8
Notebook environments
Spark environments
Data Refinery environments
SPSS Modeler environments
RStudio environments
JupyterLab environments
GPU environments
Hadoop environments
Creating non-standard environment templates
Customizing environments
Customizing environment templates
Examples of customizations
Customizing with conda and mamba
Customizing an environment (Anaconda Repository for Cloud Pak for Data)
Customizing with pip
Building custom runtime images
Downloading the runtime configuration
Downloading the runtime image
Creating a custom image
Adding customizations to images
Best practices for adding customizations to images
Pushing the image to the registry
Image download optimization
Uploading the custom configuration
Managing environments
Promoting environments
Stopping active runtimes
Disabling or changing CPU, GPU and RStudio idle timeout
Updating and deleting custom runtime definitions
Viewing active runtimes across projects
Creating and managing jobs
Creating jobs in the Notebook editor
Creating jobs in Data Refinery
Creating jobs in IBM Match 360
Creating jobs in SPSS Modeler
Creating jobs for code files
Creating jobs in DataStage
Creating jobs with Masking flow
Creating jobs for running data quality rules
Creating jobs for Pipelines
Viewing jobs across projects
Leaving a project
Accessing data
Supported data sources
Connectors
Amazon RDS for MySQL connection
Amazon RDS for Oracle connection
Amazon RDS for PostgreSQL connection
Amazon Redshift connection
Amazon S3 connection
Setting up temporary credentials or a Role ARN for Amazon S3
Apache Cassandra connection
Apache Cassandra for DataStage connection
Apache Derby connection
Apache HBase connection
Apache HDFS connection
Apache Hive connection
Apache Impala connection
Apache Kafka connection
Box connection
DataStax Enterprise connection
Dremio connection
Dropbox connection
Elasticsearch connection
Elastic Cloud connection
Exasol connection
FTP connection
Generic JDBC connection
Generic S3 connection
Google BigQuery connection
Workload identity federation examples
Google Cloud Pub/Sub connection
Google Cloud Storage connection
Google Looker connection
Greenplum connection
HDFS via Execution Engine for Hadoop connection
Hive via Execution Engine for Hadoop connection
HTTP connection
IBM Cloud Data Engine connection
IBM Cloud Databases for MongoDB connection
IBM Cloud Databases for MySQL
IBM Cloud Databases for PostgreSQL connection
IBM Cloud Object Storage connection
IBM Cloud Object Storage (infrastructure) connection
IBM Cloudant connection
IBM Cognos Analytics connection
IBM Data Virtualization connection
IBM Data Virtualization Manager for z/OS connection
IBM Db2 connection
Setting up a Db2 connection that uses TLS and SSL
IBM Db2 for DataStage connection
IBM Db2 Big SQL connection
IBM Db2 for i connection
IBM Db2 for z/OS connection
IBM Db2 on Cloud connection
IBM Db2 Warehouse connection
IBM Informix connection
IBM Match 360 connection
IBM MQ connection
IBM Netezza Performance Server connection
IBM Netezza Performance Server for DataStage connection
IBM Planning Analytics connection
IBM Product Master connection
IBM SPSS Analytic Server connection
IBM watsonx.data Presto connection
Impala via Execution Engine for Hadoop connection
MariaDB connection
Microsoft Azure Blob Storage connection
Microsoft Azure Cosmos DB connection
Microsoft Azure Data Lake Storage connection
Microsoft Azure Databricks connection
Microsoft Azure File Storage connection
Microsoft Azure SQL Database connection
Microsoft Azure Synapse Analytics connection
Microsoft Power BI (Azure) connection
Microsoft Power BI (Local) connection
Microsoft SQL Server connection
Microsoft SQL Server Integration Services (SSIS) connection
Microsoft SQL Server Reporting Services (SSRS) connection
MicroStrategy connection
Milvus connection
MinIO connection
MongoDB connection
MySQL connection
OData connection
ODBC connection
Oracle connection
Oracle Database for DataStage connection
Oracle Business Intelligence Enterprise Edition connection
Oracle Data Integrator connection
PostgreSQL connection
Presto connection
Qlik Sense connection
Salesforce.com connection
Salesforce API for DataStage connection
SAP ASE connection
SAP BAPI connection
SAP Bulk Extract connection
SAP BusinessObjects connection
SAP Delta Extract connection
SAP HANA connection
SAP IDoc connection
SAP IQ connection
SAP OData connection
SingleStoreDB connection
Snowflake connection
Storage volume connection
Tableau connection
Teradata connection
Teradata database for DataStage connection
Vertica connection
Connecting to data sources
Managing collaborators on platform connections
Data protection with data source definitions
Protection solutions for data source definitions
Connectors that support data source definitions
Connectors with hard-coded data source identity properties
Roles and asset privacy settings for data source definitions
Creating a data source definition
Creating a data source definition from the Data source definition list
Adding endpoints to a new or existing data source definition
Editing, deactivating, activating, or deleting data source definitions
Adding data to a project
Adding connections to projects
Using secrets from vaults in connections
Adding data from a connection
Adding a dynamic view of data from a connection
Adding a connected folder asset from a connection
Accessing data sources with the Flight service
Using the Flight service in your applications
Flight data requests
Examples of Flight service data requests with Python
Examples of Flight service data requests with R
Flight service in Python notebooks
Flight service in R notebooks
Examples of Flight service with itc_utils and pandas
Best practices for loading large volumes of data from a file or connection
Downloading data assets
Adding catalog assets to a project
Publishing assets to a catalog
Searching for data assets
Discovering data products
Finding a data product
Subscribing to a data product
Subscribing to a data product that requires approval
Requesting a data product
Flight client example for accessing a data product
Preparing and integrating data
Virtualizing data
Connecting to data sources
Connection access restrictions
Transferring ownership of data sources
Connecting with personal credentials
Supported data sources
Connecting to Amazon S3
Connecting to Ceph
Connecting to IBM Cloud Object Storage
Connecting to Google BigQuery
Connecting to MinIO
Connecting to REST API
Connecting to SAP HANA
Connecting to SAP HANA on SAP Cloud Platform
Connecting to Snowflake
Connecting to Spark SQL
Connecting with generic JDBC driver
Filtering data
Status of data sources
Accessing data sources by using remote connectors
Installing remote connectors
Supported file types for remote data sources
Setting the character encoding scheme
Managing connectors on remote data sources
Discovering remote data sources
Creating virtual objects
Creating a virtualized table from a single data source
Creating a virtualized table from multiple data sources
Creating a virtualized table from files by using a remote connector
Creating a virtualized table from files in Cloud Object Storage
Creating schemas for virtual objects
Joining virtual objects
Joining virtual objects in SQL builder
Joining virtual objects in Data Virtualization
Removing a virtualized object
Managing access to virtual objects
Managing access to virtual objects
Revoking access to virtual objects
Managing visibility of virtual objects
Governing virtual data
Virtualizing data with business terms in default virtualization mode
Virtualizing data with business terms in strict virtualization mode
Enabling strict mode
Disabling strict mode
Publishing virtual data to a catalog
Manually creating a connection in the catalog
Governing virtual data with data protection rules
Authorization model for views
Allowing and denying access to data in Data Virtualization
Masking virtual data
Row-level filtering in Data Virtualization
Configuring PEP cache settings
Managing and customizing caches and queries
Adding data caches
Autocaching
Cache recommendations
Configuring cache recommendations
Finding cache recommendations
Viewing query history
Restrictions for caching
Improving query performance
Collecting statistics
Collecting statistics in the web client
Collecting statistics by using SQL
Checking for missing statistics
Collecting statistics for data in object storage
Collecting statistics for all other data sources
Data source specific limitations
Enabling caching
Creating function mappings
Specific data source type
Specific server
Monitoring and exploring the service
Monitoring integrated databases
Exploring integrated databases
Developing database applications
SQL interface
Running SQL
Data Virtualization procedures
VIRTUALIZENATIVEQUERY stored procedure
SETCONFIGPROPERTY stored procedure
SETCONFIGPROPERTY properties
TOGGLEPERSONALCREDENTIALSUSEINCACHE stored procedure
EXT_AUTHORIZER_EXPLAIN stored procedure
LISTCOSBUCKETS stored procedure
LISTCOSOBJECTS stored procedure
COLLECT_STATISTICS stored procedure
EXPLAIN_FORMAT stored procedure
MIGRATE_GROUP_AUTHZ stored procedure
removeCosConn stored procedure
removeRdbcX stored procedure
setCosConn stored procedure
setRdbcX stored procedure
setRdbcX stored procedure
Data Virtualization views
LISTNODES view
LISTRDBC view
Refining data
Adding data to Data Refinery
Validating your data
Visualizing your data
Managing Data Refinery flows
Target connection options
Refining data on the Hadoop cluster
Refining HDFS data
Refining data stored in tables in a Hive warehouse
Refining data stored in tables in Impala
GUI operations
Interactive code templates
Supported data sources for Data Refinery
Creating synthetic data
Creating synthetic data from imported data
Creating synthetic data from production data
Creating synthetic data from a custom data schema
Viewing synthetic data
Evaluating synthetic data
Exporting synthetic data
Curating data
Importing metadata
Designing metadata imports
Asset types created through metadata import
Discovering data
Importing ETL jobs
Preparing ETL job files
Importing BI reports
Preparing report files
Importing data models
Preparing data model files
Getting lineage
Getting ETL job lineage
Getting BI report lineage
Managing existing metadata imports
Metadata import jobs
Enriching your data assets
Designing metadata enrichments
Creating an enrichment asset
Random sampling concepts
Managing existing enrichments
Reviewing enrichment results
Profile details
Frequency distributions
Term assignment
Reviewing results in a spreadsheet program
Making bulk changes to assignments
Identifying primary keys
Identifying relationships
Advanced data profiling
Publishing enrichment results
Default enrichment settings
Adding a custom service for term assignment
Supported connectors
Managing data quality
Data quality dimensions
Predefined data quality checks
Data quality assets
Managing data quality definitions
Building blocks for rule logic
Sample data quality definitions
Sample rule expressions
Reserved words
Managing data quality rules
Creating rules from data quality definitions
Creating SQL-based rules
Sample SQL rules
Assessing data quality
Data quality analysis results
Data quality scores
Data quality SLA rule compliance and remediation
Transforming data with DataStage
Managing assets
Designing flows
Asset browser
DataStage stages
Aggregator
Fast path
Stage tab
Calculation and recalculation dependent properties
Bloom Filter
Stage tab
Change Apply
Example data (DataStage)
Fast path
Change Capture
Fast path
Stage tab
Checksum
Adding a Checksum column to your data (DataStage)
Properties for Checksum Stage (DataStage)
Mapping output columns (DataStage)
Specifying execution options (DataStage)
Column Export
Fast path
Stage tab
Input tab
Output tab
Column Generator
Fast path
Stage tab
Column Import
Examples (DataStage)
Fast path
Stage tab
Input tab
Output tab
Combine Records
Examples (DataStage)
Example 1 (DataStage)
Example 2 (DataStage)
Fast path
Stage tab
Properties section
Outputs section
Combine keys section
Options section
Advanced section
NLS Locale section
Compare
Fast path
Stage tab
Compress
Fast path
Stage tab
Copy
Fast path
Stage tab
Decode
Fast path
Difference
Fast path
Stage tab
Distributed Transaction
Encode
Fast path
Stage tab
Excel
Configuring the Excel stage as a source
Configuring the Excel stage as a target
Extracting the data from Microsoft Excel
Examples of extracting data from Microsoft Excel files
Examples of writing data to Microsoft Excel files
Reference
Properties reference: Excel
Data type conversions from Microsoft Excel to IBM DataStage
Data type conversions from DataStage to Microsoft Excel
Job abort conditions in Microsoft Excel
Consideration about end of wave
Expand
Fast path
Stage tab
External Filter
Fast path
Stage tab
External Source
Fast path
Stage tab
Advanced section
NLS Map section
Output tab
Properties section
Source section
Options section
Format section
Using RCP With External Source Stages (DataStage)
External Target
Fast path
Stage tab
Advanced section
NLS Map section
Input tab
Input link properties section
Target section
Options section
Format section
Output tab
Using RCP with External Target stages (DataStage)
Filter
Specifying the filter
Input data columns
Supported Boolean expressions and operators
Order of association (DataStage)
String comparison (DataStage)
Fast path
Stage tab
Funnel
Fast path
Stage tab
Link Ordering section
Generic
Fast path
Stage tab
Head
Fast path
Stage tab
Hierarchical data
Using the Hierarchical Data stage (DataStage)
Adding a Hierarchical Data stage to a (DataStage) flow
Configuring runtime properties for the Hierarchical Data stage (DataStage)
The assembly (DataStage)
Input step (DataStage)
Output step (DataStage)
Assembly Editor (DataStage)
Opening the Assembly Editor (DataStage)
Schema views
Mapping data (DataStage)
Working with the mapping table (DataStage)
Determining mapping candidates (DataStage)
Configuring how mapping candidates are determined (DataStage)
XML Composer step (DataStage)
XML Composer validation rules (DataStage)
XML Parser step (DataStage)
XML Parser validation rules (DataStage)
Setting default values for types (DataStage)
JSON transformation (DataStage)
Schema management (DataStage)
Opening the Schema Library Manager (DataStage)
Working with libraries and resources (DataStage)
Creating a JSON schema in the schema library (DataStage)
JSON Parser step (DataStage)
JSON Parser validation rules (DataStage)
JSON Composer step (DataStage)
JSON Composer validation rules (DataStage)
REST operator in (DataStage)
REST step pages
General
Security
Request
Response
Mappings
Output schema of the REST step
Passing multiple rows from an XML or JSON file
Transformation steps for the Hierarchical Data stage (DataStage)
Aggregate step (DataStage)
H-Pivot step (DataStage)
HJoin step (DataStage)
Order Join step (DataStage)
Regroup step (DataStage)
Sort step (DataStage)
Union step (DataStage)
V-Pivot step (DataStage)
Java Integration
Java Integration
Setting up your development environment
Implementing abstract methods of the Processor class
Compiling the Java code
Running the Java code on the Parallel Engine, Java Integration stage
Accessing Stage Configuration
Declaring the Capabilities of the Java code
Reading Records from Input Link
Writing Records to Output Link
Rejecting Records
Looking up data in the Sparse lookup mode
Data Types
Retrieving Column Metadata on the Link
Using user-defined properties
Runtime column propagation
Running the Java code on the conductor node
Transferring data from the conductor node to player nodes
Logging messages with the Java Integration stage
Terminating a job from the Java code
Using JavaBeans
User Defined Function
Designing jobs with the Java Integration stage
Configuring Java Integration stage as a source
Configuring Java Integration stage as a transformer
Configuring Java Integration stage as a target
Looking up data by using reference links
Setting up column definitions
Java libraries
Properties reference: Java Integration
Join
Join versus lookup
Fast path
Stage tab
Lookup
Lookup versus Join
Fast path
Properties
Stage tab
Input and Output tab
Make Subrecord
Fast path
Stage tab
Examples (DataStage)
Make Vector
Examples (DataStage)
Example 1 (DataStage)
Example 2 (DataStage)
Fast path
Stage tab
Properties section
Options section
Advanced section (DataStage)
Merge
Fast path
Stage tab
Modify
Fast path
Stage tab
Operational Decision Manager
Overview
Engine mode
Batch mode and key mode processing
Execution Object Model types
Fields and ruleset parameters mapping
Field propagation strategies
Java class path configuration
Designing a job
Creating a flow
Configuring the stage to invoke the ruleset
Configuring input and output link properties
Links and ruleset parameters association
Buffer links
Null value handling
Configuring the reject link
Compiling the flow
Configuration wizard
Configuring the stage with the wizard
Basic Java types and methods
DataStage type to Java type mapping
Java type to DataStage type mapping
Properties reference
Peek
Fast path
Stage tab
Pivot Enterprise
Specifying a horizontal pivot operation (DataStage)
Specifying a horizontal pivot operation and mapping output columns (DataStage)
Example of horizontally pivoting data (DataStage)
Specifying a vertical pivot operation (DataStage)
Specifying a vertical pivot operation and mapping output columns (DataStage)
Example of vertically pivoting data (DataStage)
Properties tab
Specifying execution options (DataStage)
Specifying where the stage runs (DataStage)
Specifying partitioning or collecting methods (DataStage)
Specifying a sort operation (DataStage)
Promote Subrecord
Examples (DataStage)
Example 1 (DataStage)
Example 2 (DataStage)
Fast path
Stage tab
Properties tab
Options section
Advanced tab
Remove Duplicates
Fast path
Stage tab
REST
Stage tab
Row Generator
Fast path
Stage tab
Output tab
Sample
Fast path
Stage tab
Link Ordering
Slowly Changing Dimension
Job design
Purpose codes
Surrogate keys
Editing
Defining the match condition
Selecting purpose codes
Purpose code definitions
Specifying information about a key source
Creating derivations for dimension columns
Dimension update action
Sort
Fast path
Stage tab
Split Subrecord
Fast path
Stage tab
Examples (DataStage)
Split Vector
Examples (DataStage)
Example 1 (DataStage)
Example 2 (DataStage)
Fast path
Stage tab
Properties tab
Options section
Advanced tab
Surrogate Key Generator
Creating the key source (DataStage)
Deleting the key source (DataStage)
Updating the state file (DataStage)
Generating surrogate keys (DataStage)
Switch
Example
Fast path
Stage tab
Tail
Fast path
Stage tab
Transformer
Basic concepts
Properties
Stage variables (DataStage)
Loop variables (DataStage)
Entering expressions (DataStage)
Loop example: converting a single row to multiple rows (DataStage)
Loop example: multiple repeating values in a single field (DataStage)
Loop example: generating new rows (DataStage)
Loop example: aggregating data (DataStage)
Surrogate Key tab (DataStage)
Link ordering (DataStage)
Triggers
Advanced (DataStage)
Input tab
Output tab
Runtime column propagation (DataStage)
System variables (DataStage)
Evaluation sequences for transformer expressions, stage variables, and loop variables (DataStage)
Reserved words
Parallel transform functions (DataStage)
Date and time functions (DataStage)
Logical functions (DataStage)
Mathematical functions (DataStage)
Null handling functions (DataStage)
Number functions (DataStage)
Raw functions (DataStage)
String functions (DataStage)
Retrieving substrings
Concatenating strings
Vector function (DataStage)
Type conversion functions (DataStage)
Utility functions (DataStage)
Operator functions (DataStage)
Function libraries
Wave Generator
Stage Tab
Properties
Input tab
Output tab
Web Service stage
Introduction to web services in DataStage
Encoding requests and responses
Required tasks in the Web Service stage
Other tasks
Setting up stage properties
Setting up input link properties
Setting up output link properties
Using a Web Service in DataStage
Write Range Map
Fast path
Stage tab
Input tab
XML Input
Stage tab
Transformation settings (DataStage)
XML Output
Using XML Output
About transforming tabular data
Validating documents and schemas
Aggregating input rows on output
Writing output to your file system
Processing NULLs and empty values
Selecting items for the XML Output
Setting the format of the XML output
Elements: Controlling the order and the repetition
Setting up stage properties
Setting up input and output link properties
DataStage connectors
Connecting to a data source in DataStage
Supported data sources in DataStage
Amazon RDS for PostgreSQL connector
Amazon S3 connector
Apache HDFS connector
Apache Hive connector
Apache Kafka connector
Apache Impala connector
FTP connector
Generic JDBC connector
Generic S3 connector
Greenplum connector
IBM Cloud Databases for PostgreSQL connector
IBM Informix connector
Oracle Database for DataStage connector
PostgreSQL connector
Snowflake connector
Teradata connector
File connectors in DataStage
Complex Flat File
Complex Flat File as a source
Defining record ID constraints in DataStage
Complex Flat File as a target in DataStage
Complex Flat File schema
Reject links in DataStage
Data set in DataStage
File set in DataStage
Input tab (DataStage)
Output tab (DataStage)
Lookup file set in DataStage
Sequential file in DataStage
Data service
Introduction
Topologies
Creating a job that includes a Data service connector
Running a job that includes a Data service connector
Using Before/After SQL Statements
Before SQL (DataStage)
Before SQL (node) in DataStage
After SQL (DataStage)
After SQL (node) in DataStage
Using stored procedures
Using multiple links
Connecting to Oracle data sources with LDAP, SEPS or TCPS
Connecting to Oracle data sources with LDAP authentication
Connecting to Oracle data sources with SEPS authentication
Connecting to Oracle data sources with TCPS authentication
Configuring Db2 connections to work with database sequences
Setting up the DB2 partition type
Quality stages in DataStage
Designing match specifications
Adding passes
Match comparisons in DataStage
Reverse matching
ABS_DIFF comparison
AN_DINT comparison
AN_INTERVAL comparison
CHAR comparison
CNT_DIFF comparison
D_INT comparison
D_USPS comparison
DATE8 comparison
DELTA_PERCENT comparison
DISTANCE comparison
INT_TO_INT comparison
INTERVAL_NOPAR comparison
INTERVAL_PARITY comparison
LR_CHAR comparison
LR_UNCERT comparison
MULT_ALIGN comparison
MULT_EXACT comparison
MULT_RANGE comparison
MULT_UNCERT comparison
NAME_UNCERT comparison
NUMERIC comparison
PREFIX comparison
PRORATED comparison
TIME comparison
UNCERT comparison
Adding QSM_MAT_UNCERT_VERSION environment variable
USPS comparison
USPS_DINT comparison
USPS_INT comparison
Testing passes
Matching data in DataStage
How matching is done
Match types
Match column selection in DataStage
Match passes in DataStage
Blocking in DataStage
Weights and record comparisons
Address Verification
Scenario: Improving international address data
Installing reference files
Parsing, validating, and transliterating address data
Errors
Investigate
Match Frequency
One-source Match
Two-source Match
Standardize
Standardize rule sets
Objects within rule sets
Classifications
Lookup tables
Output columns
Rules
Overrides
Quality stage pattern action reference
Introduction to the Pattern Action language
Parsing elements
Unconditional patterns
Identifying simple pattern classes
Conditional patterns
Simple conditional values
Conditional expressions
Using arithmetic expressions
Action statements
Copying information
Referencing dictionary fields from another rule set
Moving information
Concatenating information
Converting information
Retyping operands
Retyping multiple tokens
Patterning
Rule set extensions
User overrides for domain preprocessor rule sets
User overrides for rule sets
Setting margins
SOUNDEX phonetic coding
NYSIIS coding
Terminating pattern matching
Calling subroutines
Writing subroutines
Performing actions repetitively
Summary of sources and targets
Survive
Specifying your own stages
Defining build stages in DataStage
General tab
Properties tab
Build tab
Build stage macros
Informational macros (DataStage)
Flow-control macros (DataStage)
Input and output macros (DataStage)
Transfer Macros (DataStage)
How your code is executed (DataStage)
Inputs and outputs (DataStage)
Using multiple inputs (DataStage)
Example Build stage in DataStage
Header files
C++ classes - sorted by header file (DataStage)
C++ macros - sorted by header file (DataStage)
Defining custom stages
Compiling custom stages
Defining wrapped stages in DataStage
Running scripts in a wrapped stage
Creating a DataStage component
Defining data definitions
Reusable job design with subflows
Local subflows
Subflows
Parameters and parameter sets
Creating and using local parameters in DataStage
Creating and using parameter sets
Inserting parameters and parameter sets as properties in DataStage
Configuring runtime parameters in a flow
Configuring runtime parameters in a job
PROJDEF parameter set
Passing values from parameter sets into jobs by command-line interface
Environment variables
Managing environment variables
Guide to setting environment variables in DataStage
Buffering
Checkpoint
Compiler
Db2 Support
Debugging
Decimal support
Disk I/O
General Job Administration
Look up support
Miscellaneous
Network
NLS support
Oracle support
Partitioning
Reading and writing files environment variables in DataStage
Reporting environment variables in DataStage
SAS support
Sorting environment variables in DataStage
Sybase support environment variables in DataStage
Teradata support
Transport blocks
WLM
National Language Support (NLS)
Creating custom NLS maps
Reject links
Partitioning and collecting data
Writing partitioned data in file-storage connectors
Running jobs
Settings for the project, flow, and job level
Setting up before-job and after-job subroutines
Cataloging a Db2 database in the runtime container
Compile options with SQL Pushdown in DataStage
ELT run mode
ELT materialization policies in DataStage
Macros
Observability with Databand
Migrating jobs
Migrating connections
Migrating an ODBC connection
Migrating SAP connections
Migrating a Teradata connection
Migrating a Teradata database for DataStage connection with the Teradata program ID
Setting up vault connections for migrated jobs
Migrating a flow with user-defined functions
Migrating the Data Rules stage as a Quality Rule
Changing XML file locations after migration
Configuring the Hierarchical stage after migration in DataStage
Migrating a flow with a custom stage
Migrating the Java Transformer stage
Migrating BASIC routines
Migrating Web Service Transformer and Web Service Client stages
Asset import report (DataStage)
Downloading and importing flows
Development, testing, and production
Deployment spaces
DataStage command-line tools
DataStage APIs
Orchestrating flows with Orchestration Pipelines
Pipeline components for DataStage
CEL expressions and limitations
Migrating and constructing pipeline flows
Replacing BASIC routines
Examples
Sharing storage volumes
Setting the time zone
Managing master data
Creating a master data configuration asset
Data concepts in IBM Match 360
Matching algorithms in IBM Match 360
Working with governed data in IBM Match 360
Configuring master data
Customizing your data model
Adding data and mapping it to your data model
Connecting data to IBM Match 360
Adding master data from InfoSphere MDM
Defining attribute composition rules
Matching your data to create master data entities
Customizing and strengthening your matching algorithm
Advanced matching algorithm tuning using the API
Configuring IBM Match 360 workflows
Saving and loading configuration snapshots
Managing IBM Match 360 jobs
Streaming record and entity data changes
Event streaming message template
Completing pair reviews
Exploring master data
Defining the way records and attributes are displayed
Exploring master data entities and records
Adding and editing individual records and entities
Maintaining record and entity linkages
Exploring relationships between records
Remediating potential matches to improve data quality
Exporting master data
Advanced master data exports using the API
APIs available in IBM Match 360
Masking data with Masking flow
Creating masking flows
Running masking flow jobs
Managing job performance
Replicating data
Starting Data Replication
Supported Data Replication connections
Replicating Amazon RDS for PostgreSQL data
Supported PostgreSQL data types
Replicating IBM Db2 on Cloud data
Supported Db2 on Cloud data types
Replicating IBM Db2 Warehouse data
Replicating Apache Kafka data
Replicating Oracle data
Supported Oracle data types
Replicating PostgreSQL data
Monitoring your replication
Managing Data Replication
Managing feature groups
Managing data
The Db2 Data Management Console
Managing alerts
Creating and scheduling jobs
Creating monitor reports
Tuning
Creating a tuning task
Choosing EXPLAIN and tuning options
Generating access plan graph
Generating statistics recommendations
Generating index recommendations
Doing an index what-if analysis
Changing table organization
Setting up monitoring profile
Setting up alerts
Creating custom alerts
Setting up event monitor profile
Db2 databases
Db2 Big SQL databases
Data Gate and integrated databases
Db2 Warehouse databases
EDB Postgres databases
Informix databases
MongoDB databases
Using the watsonx.data console
Enterprise governance
Data governance
Planning to implement data governance
Differences between IBM Knowledge Catalog editions
Planning to set up IBM Knowledge Catalog
Planning to implement a governance framework
Planning to protect data with rules
Planning to author data protection rules
Planning to curate data
Planning to monitor IBM Knowledge Catalog
Catalogs
Administering a catalog
Creating a catalog
Duplicate asset handling
Changing catalog settings
Deleting a catalog
Saving searches for catalog assets
Managing access to a catalog
Catalog collaborator roles
Catalog assets
Finding and viewing an asset in a catalog
Adding assets to a catalog
Adding a data file
Adding a connection
Adding data from a connection
Adding a connected folder asset from a connection
Adding COBOL copybook assets
Adding and updating asset metadata
CSV file format for importing metadata asset details
CSV file format for importing asset relationships details
Downloading data assets
Editing asset properties
Relationships in a catalog
Asset relationships
Managing relationships in a catalog
Exploring relationships
Controlling access to an asset
Profiling an asset
Managing business lineage
Importing lineage relationship mapping files
Lineage relationship mapping file format
Removing an asset
Governance artifacts
Finding and viewing governance artifacts
Tags
Governance artifact properties
Managing governance artifacts
Import methods for governance artifacts
Importing artifacts by type with a CSV file
CSV import file format
Importing all governance artifacts with a ZIP file
Exporting governance artifacts
Workflows for governance artifacts
Categories
Predefined categories
Designing categories
Managing categories
Managing category collaborators
Category collaborator roles
Creating custom category collaborator roles
Importing or exporting categories
Policies
Designing policies
Governance rules
Designing governance rules
Data protection rules
Designing data protection rules
Filtering rows
Mask data
Advanced masking options
Redacting data method
Obfuscating data method
Preserve format method
Identifier masking method
Decrypting reversible masked data
Data protection rules enforcement
Managing data protection rules
Data quality SLA rules
Designing data quality SLA rules
Managing data quality SLA rules
Business terms
Designing business terms
Managing business terms
Authoring business terms
Classifications
Designing classifications
Predefined classifications
Data classes
Designing data classes
Adding matching methods to data classes
Predefined data classes
Predefined data classes details
Reference data
Designing reference data sets
Creating reference data sets with composite keys
Importing files for reference data sets
Relationships between reference data sets
Predefined reference data sets
Knowledge Accelerators
Notices
KA for Cross Industry
KA for Energy and Utilities
KA for Financial Services
KA for Healthcare
KA for Insurance
Getting started
Artifacts available to import
Cross Industry
Energy and Utilities
Financial Services
Healthcare
Insurance
Components
Use of governance artifacts
Category areas and subcategories
Business terms
Relationships
Custom attributes
Classifications and tags
Business Core Vocabulary
Subcategories
Concept terms
Property terms
Relationship terms
Business Performance Indicators
Subcategories
Performance analysis terms
Measures
Industry Alignment Vocabularies
Alignment area categories
Alignment topic categories
Alignment terms
Business Scopes
Subcategories
Business Scopes available for separate import
Cross Industry
Energy and Utilities
Financial Services
Healthcare
Insurance
Reference data sets
Data classes
Policies and rules
Synonyms
Using and customizing
Setting up reporting for IBM Knowledge Catalog
Data model
Managing reporting
Configuring reporting settings
Sample reporting queries
Reporting tables
Workspaces
Asset relationships
Categories
Governance artifacts
Artifact relationships
Data quality rules
Customizations
Workflow
User Profiles
Tags
Rules
Metadata imports and enrichments
Guardium Data Protection
Masking watsonx.data assets in IBM Knowledge Catalog
Publishing data products
Managing your task inbox as a producer
Creating a data product
Creating a data product from SQL
Creating a data product from a query
Creating a data product from a custom query
Creating a data product from a complex query
Creating a data product directly from a source
Creating a data product from a catalog
Creating a data product from a project
Creating a data product from a URL
Managing data products
Creating data source connections
Amazon S3 connection
Setting up temporary credentials or a Role ARN for Amazon S3
Google Cloud Storage connection
IBM Cloud Object Storage connection
Microsoft Azure File Storage connection
Microsoft Azure Blob Storage connection
Working with delivery methods
Delivery methods for connectors in Data Product Hub
Data Product Hub API
AI governance
Governing assets with watsonx.governance
Governing AI assets
Planning for AI governance
Scenario for governing an AI RAG solution
Evaluating AI models
Setup options for Watson OpenScale
Configuring Watson OpenScale with automatic setup
Configuring Watson OpenScale with manual setup
Configuring Watson OpenScale with advanced setup
Glossary
FAQs
Supported machine learning engines, frameworks, and models
IBM Watson Machine Learning
Microsoft Azure ML Studio frameworks
Microsoft Azure ML Service frameworks
Amazon SageMaker frameworks
Custom ML frameworks
IBM SPSS C&DS frameworks
Integrating 3rd-party ML engines with Watson OpenScale
Preparing to evaluate models
Adding deployments for evaluations
Providing model details
Configure asset deployments using JSON configuration files
Managing data for model evaluations
Managing payload data
Payload logging
Managing training data
Managing feedback data
Sending model transactions
Evaluating prompt templates in projects
Evaluating prompt templates in deployment spaces
Evaluating detached prompt templates in projects
Evaluating detached prompt templates in spaces
Evaluating prompt templates for non-IBM foundation models with notebooks
Configuring model evaluations
Quality evaluations
Fairness evaluations
Calculating fairness
Configuring for indirect bias
Drift evaluations
Drift metrics
Drift v2 evaluations
Generative AI quality evaluations
Model health monitor evaluations
Creating custom evaluations and metrics
Configuring explainability
Reviewing model insights
Reviewing evaluation results
Reviewing model transactions
Explaining model transactions
Reviewing debiased transactions
Batch processing
Preparing the batch processing environment in IBM Analytics Engine
Preparing the batch processing environment in IBM Analytics Engine on zLinux
Preparing the batch processing environment on the Hadoop Ecosystem
Configure batch processing in Watson OpenScale
Model risk management and model governance
Manage model risk
Metrics computation using Python SDK
Watson OpenScale Python client samples
Governing assets in AI use cases
Managing AI use cases
Managing report templates
Managing attachments
Setting up an AI use case
Associating workspaces with an AI use case
Tracking assets in an AI use case
Tracking prompt templates
Tracking machine learning models
Tracking external models
Tracking assets in a notebook
Viewing factsheets
Collaboration roles for governance
Customizing factsheet user details
Integrating AI use cases with Governance console
Managing risk and compliance with Governance console
Getting started
Configuring Governance console
Creating use cases and viewing models
Completing a risk assessment
Completing an applicability assessment
Solution components
Model Risk Governance solution
Operational Risk Management solution
Regulatory Compliance Management solution
Object types
Model Risk Governance workflows
Regulatory Compliance Management workflows
Operational Risk Management workflows
Managing workflows
Managing governance artifact workflows
Configuring IBM Match 360 workflows
Managing data quality remediation workflows
Working with data quality remediation tasks
Designing and creating custom workflows
Importing custom process definitions
Configuring custom workflows
Monitoring workflow tasks
Detecting workflow anomalies
Managing master data
Business analytics
Building interactive visualizations with Cognos Dashboards
Before you start
Supported data sources
Supported local file formats
Creating a dashboard
Editing a dashboard
Creating a relationship
Join operators
Join optimization
Sharing a dashboard
Publishing a dashboard to a catalog
Adding a dashboard from a catalog
Checking and refreshing data sources
Enabling dashboards to use governed virtual data
Migrating dashboards to Cognos Analytics
Analyzing and visualizing data with Cognos Analytics
Modeling and reporting patterns in data with Planning Analytics
Analyzing data and working with models
AI risk atlas
Non-disclosure
Lack of training data transparency
Model usage rights restrictions
Prompt injection attack
Incomplete advice
Lack of system transparency
Data usage restrictions
Impact on cultural diversity
Impact on education: plagiarism
Personal information in data
Improper usage
Extraction attack
Impact on Jobs
Jailbreaking
Data acquisition restrictions
Data bias
Uncertain data provenance
Unrepresentative risk testing
Data usage rights restrictions
Harmful code generation
Data contamination
Incomplete usage definition
Copyright infringement
Lack of data transparency
Impact on affected communities
Improper retraining
Spreading toxicity
Inaccessible training data
Impact on education: bypassing learning
Untraceable attribution
Evasion attack
Impact on the environment
Over- or under-reliance
Incorrect risk testing
Membership inference attack
Confidential data in prompt
Data privacy rights alignment
IP information in prompt
Prompt leaking
Hallucination
Legal accountability
Prompt priming
Reidentification
Attribute inference attack
Poor model accuracy
Data transfer restrictions
Generated content ownership and IP
Output bias
Dangerous use
Unexplainable output
Human exploitation
Toxic output
Data poisoning
Unreliable source attribution
Harmful output
Confidential information in data
Lack of model transparency
Unrepresentative data
Impact on human agency
Personal information in prompt
Nonconsensual use
Decision bias
Lack of testing diversity
Exposing personal information
Improper data curation
Revealing confidential information
Spreading disinformation
Developing generative AI solutions
Terms of use
Tokens
Supported foundation models
Supported embedding models
IBM Slate 125m v2 embedding model card
IBM Slate 125m embedding model card
IBM Slate 30m v2 embedding model card
IBM Slate 30m embedding model card
Choosing a model
Foundation model lifecycle
IBM foundation models
Prompting granite-13b-chat-v2
Prompting granite-7b-lab
IBM Granite 8b Japanese model card
IBM Granite 20b multilingual model card
IBM Granite 13b chat v2 model card
IBM Granite 13b instruct v2 model card
IBM Granite 13b v1 model card
IBM Granite 3b code instruct model card
IBM Granite 8b code instruct v2 model card
IBM Granite 20b code instruct model card
IBM Granite 34b code instruct model card
InstructLab foundation models
Prompt Lab
Saving prompts
Building reusable prompts
Model parameters for prompting
Sample prompts
Removing harmful content
Prompt tips
Avoiding undesirable output
Generating accurate output
Overcoming context length limitations
Tuning Studio
When to tune a model
Choosing a model to tune
Methods for tuning
Tuning a model
Creating a fine-tuning experiment
Creating a prompt-tuning experiment
Evaluating the tuning experiment
Data formats
Tuning parameters
Evaluating tuning experiment results
Improving a prompt-tuning experiment
Coding generative AI solutions
Credentials for programmatic access
Finding the project ID
Foundation model IDs
Python library
Getting foundation model information
Inferencing a foundation model
Inferencing with a prompt template
Tuning a foundation model
Converting text to text embeddings
Node.js SDK
REST API
Text embedding generation
Retrieval-augmented generation
Using vectorized text with RAG tasks
Notebooks and scripts
Planning your notebooks and scripts experience
Jupyter Notebook editor
Creating and managing notebooks
Parts of a notebook
Jupyter kernels and notebook environments
Coding and running notebooks
Markdown cheatsheet
JupyterLab
JupyterLab with default Git integration
Organizing your code files
JupyterLab with deprecated Git integration
Visual Studio Code
RStudio
Working in RStudio with default Git integration
Working in RStudio with deprecated Git integration
Working in RStudio with no Git integration
Using Spark in RStudio
Using libs from Anaconda Repository
Using libs from CRAN repositories
Accessing data in MySQL databases by using the RMariaDB library
Adding custom R packages
Libraries and scripts
Installing custom libraries
Importing scripts into a notebook
Watson Natural Language Processing
Working with pre-trained models
Library task catalog
Language detection
Syntax analysis
Noun phrase extraction
Keyword extraction and ranking
Entity extraction
Embeddings
HAP detection
Sentiment extraction
Tone classification
Emotion classification
Relations extraction
Hierarchical categorization
Category types
Creating your own models
Detecting entities with a custom dictionary
Detecting entities with regular expressions
Detecting entities with a custom transformer model
Classifying text with a custom classification model
Extracting sentiment with a custom transformer model
Extracting targets sentiment with a custom transformer model
Usage samples
Geospatial data analysis
Data skipping for Spark SQL
Parquet encryption
Key management by application
Key management by KMS
Time series analysis
Using the time series library
Time series key functionality
Time series functions
Time series lazy evaluation
Time reference system
SPSS predictive analytics algorithms
Data preparation
Classification and regression
Clustering
Forecasting
Survival analysis
Score
Loading and accessing data in a notebook
Loading data through generated code snippets
Adding data from a Planning Analytics connection
Accessing project assets with ibm-watson-studio-lib
ibm-watson-studio-lib for Python
ibm-watson-studio-lib for R
Migrating from project-lib for Python to ibm-watson-studio-lib
Migrating from project-lib for R to ibm-watson-studio-lib
Accessing project assets with project-lib
project-lib for Python (deprecated)
project-lib for R (deprecated)
Managing feature groups with assetframe-lib for Python
Managing the notebooks and scripts lifecycle
Automating the lifecycle of notebooks and scripts
Sharing notebooks
Hiding code in a notebook
AutoAI
AutoAI glossary
Building an AutoAI experiment
Configuring experiment settings
Configure a text analysis experiment
Building a time series forecast experiment
Tutorial: AutoAI univariate time series experiment
Tutorial: AutoAI supporting features time series experiment
Time series experiment implementation details
Scoring a time series model
Building a time series anomaly prediction experiment
Tutorial: AutoAI anomaly prediction experiment
Saving an AutoAI generated notebook
Running an AutoAI notebook on IBM Z and IBM LinuxONE
Selecting an AutoAI model
AutoAI implementation details
Configuring data imputation
Imputation details for time series experiments
Using incremental learning to train pipelines
Incremental learning details
Evaluating AutoAI experiments for fairness
Troubleshooting AutoAI experiments
Federated learning
Get started
Tutorials and samples
Tensorflow tutorial
Tensorflow samples
XGBoost tutorial
XGBoost samples
FHE sample
Frameworks, fusion methods, and Python versions
Hyperparameters
Creating the experiment
Set up your system
Creating the initial model
Create the data handler
Starting the aggregator (Admin)
Connecting to the aggregator (Party)
Monitoring and saving the model
Applying encryption
Limitations and troubleshooting
Deep Learning Experiment Builder
Deep learning experiment tutorial using MNIST data set
Deep learning experiment tutorial using CIFAR data set
Decision Optimization
Ways to use Decision Optimization
Supported data sources in Decision Optimization
Sample models and notebooks
Decision Optimization notebooks
Decision Optimization experiments
Creating a Decision Optimization experiment
Scenarios
Opening the scenario pane
Creating a scenario from scenario pane
Creating a scenario from Overview
Importing a scenario
Exporting a scenario
Viewing all scenarios
Generating a notebook from a scenario
Saving scenarios for deployment
Configuring a Decision Optimization experiment
Changing the deployment space for your experiment
Changing default environments and adding Python libraries
Preparing input data
Importing data into a scenario
Editing input data
Table search and filtering
Exporting data to a project
Building a model
Referencing imported data and defining output
Multiple model files
Importing a model
Configuring engine settings in a model
Solving a model
Intermediate solutions
Configuring the run parameters for a scenario
Selecting a different run environment for a scenario
Exploring the solution
Visualization view
Visualization widgets
Displaying multiple scenarios
Visualization widget syntax
Note widgets
Table widgets
Chart widgets
Vega Chart widgets
Gantt widgets
Pivot table widgets
Python DOcplex models
Input and output data
Solving and analyzing a model: the diet problem
Changing the scenario data
Working with multiple scenarios
Generating multiple scenarios
Engine settings
Modeling Assistant models
Selecting a Decision domain in the Modeling Assistant
Formulating and running a model: house construction scheduling
Adding multi-concept constraints and custom decisions: shift assignment
Creating advanced custom constraints with Python
OPL models
Decision Optimization Java models
SPSS Modeler
Tutorials
Introduction to modeling
Building the flow
Browsing the model
Evaluating the model
Scoring records
Summary
Automated modeling for a flag target
Historical data
Building the flow
Generating and comparing models
Summary
Automated modeling for a continuous target
Training data
Building the flow
Comparing the models
Summary
Automated data preparation
Building the flow
Comparing the models
Drug treatment - exploratory graphs
Reading in text data
Creating a distribution chart
Creating a scatterplot
Creating a web chart
Creating advanced visualizations
Deriving a new field
Building a model
Browsing the model
Using an Analysis node
Screening predictors
Building the flow
Building the models
Reducing input data string length
Reclassifying the data
Classifying telecommunications customers
Building the flow
Browsing the model
Telecommunications churn
Building the flow
Browsing the model
Forecasting bandwidth utilization
Forecasting with the Time Series node
Creating the flow
Examining the data
Defining the dates
Defining the targets
Setting the time intervals
Creating the model
Examining the model
Summary
Forecasting catalog sales
Creating the flow
Examining the data
Exponential smoothing
ARIMA
Summary
Making offers to customers (self-learning)
Building the flow
Browsing the model
Retail sales promotion
Examining the data
Learning and testing
Condition monitoring
Examining the data
Data preparation
Learning
Testing
Hotel satisfaction example for Text Analytics
Text Mining node
Tuning results on the Text Analytics Workbench
Building and deploying the model
Text Link Analysis node
Nodes palette
Import
Data Asset node
User Input node
Sim Gen node
Extension Import node
Record Operations
Select node
Sample node
Sort node
Balance node
Distinct node
Aggregate node
Merge node
Append node
Streaming Time Series node
SMOTE node
RFM Aggregate node
Space-Time-Boxes node
Extension Transform node
Streaming TCM node
CPLEX Optimization node
Field Operations
Auto Data Prep node
Type node
Viewing and setting information about types
Measurement levels
Geospatial measurement sublevels
Converting continuous data
What is instantiation?
Data values
Setting options for values
Specifying values and labels for continuous data
Specifying values and labels for nominal and ordinal data
Specifying values for a flag
Specifying values for collection data
Specifying values for geospatial data
Defining missing values
Checking type values
Setting the field role
Setting field format options
Filter node
Derive node
Filler node
Reclassify node
Binning node
RFM Analysis node
Ensemble node
Partition node
Set to Flag node
Restructure node
Transpose node
Field Reorder node
History node
Time Intervals node
Anonymize node
Graphs
Charts node
Plot node
Multiplot node
Time Plot node
Distribution node
Histogram node
Collection node
Web node
Evaluation node
Modeling
Auto Classifier node
Continuous machine learning
Auto Numeric node
Auto Cluster node
TCM node
Bayes Net node
C5.0 node
C&R Tree node
CHAID node
QUEST node
Tree-AS node
Random Trees node
Random Forest node
Decision List node
Time Series node
GenLin node
GLMM node
GLE node
Linear node
Linear-AS node
Regression node
LSVM node
Logistic node
Neural Net node
KNN node
Cox node
PCA/Factor node
SVM node
Feature Selection node
Discriminant node
SLRM node
Association Rules node
Apriori node
CARMA node
Sequence node
Kohonen node
Anomaly node
K-Means node
TwoStep cluster node
TwoStep-AS cluster node
Isotonic-AS node
XGBoost-AS node
K-Means-AS node
XGBoost Tree node
XGBoost Linear node
Gaussian Mixture node
KDE node
One-Class SVM node
MultiLayerPerceptron-AS node
HDBSCAN node
Extension Model node
Extension model nugget
Text Analytics
About text mining
How extraction works
How categorization works
Language Identifier node
Text Link Analysis node
Expert options
TLA node output
Text Mining node
Text Mining model nuggets
Text Analytics Workbench
Concepts tab
Text links tab
Categories tab
Resource editor tab
Setting options
Advanced linguistic settings
Advanced frequency settings
Generating a model nugget
Linguistic resources
Reusing custom linguistic resources
Reusing custom category sets
Reusing a library
Reusing a template
Reusing a text analysis package (TAP)
Outputs
Table node
Matrix node
Analysis node
Data Audit node
Transform node
Statistics node
Means node
Report node
Set Globals node
Sim Fit node
Sim Eval node
KDE Simulation node
Extension Output node
Export
Data Asset Export node
Extension Export node
Extension nodes
R scripts
Python for Spark scripts
Scripting with Python for Spark
Analytic Server Context
Data metadata
Date, time, timestamp
Exceptions
Examples
Native Python APIs
Decision tree nodes
The interactive tree builder
Growing and pruning the tree
Defining custom splits
Saving tree models and results
Building a tree model directly
Decision tree nodes
SuperNodes
Reference information
Tips and shortcuts
Scripting and automation
Scripting overview
Types of scripts
Flow scripts
Flow script example: Training a neural net
Jython code size limits
Running and interrupting scripts
The scripting language
Python and Jython
Python scripting
Operations
Lists
Strings
Remarks
Statement syntax
Identifiers
Blocks of code
Passing arguments to a script
Examples
Mathematical methods
Using non-ASCII characters
Object-oriented programming
Defining a class
Creating a class instance
Adding attributes to a class instance
Defining class attributes and methods
Hidden variables
Inheritance
Scripting in SPSS Modeler
Flows, SuperNode streams, and diagrams
Flows
SuperNode flows
Diagrams
Running a flow
The scripting context
Referencing existing nodes
Finding nodes
Setting properties
Creating nodes and modifying flows
Creating nodes
Linking and unlinking nodes
Importing, replacing, and deleting nodes
Traversing through nodes in a flow
Getting information about nodes
The scripting API
Example: Searching for nodes using a custom filter
Metadata: Information about data
Parameters
Global values
Error reporting for flows
Scripting tips
Looping through nodes
Accessing flow run results
Table content model
XML content model
JSON content model
Column Statistics content model and Pairwise Statistics content model
Properties reference overview
Syntax for properties
Structured properties
Abbreviations
Node and flow property examples
Node properties overview
Common node properties
Flow properties
Data Asset Import node properties
dataassetimport properties
extensionimportnode properties
simgennode properties
userinputnode properties
Record Operations node properties
appendnode properties
aggregatenode properties
balancenode properties
cplexoptnode properties
derive_stbnode properties
distinctnode properties
extensionprocessnode properties
mergenode properties
rfmaggregatenode properties
samplenode properties
selectnode properties
sortnode properties
streamingtimeseries properties
Field Operations node properties
anonymizenode properties
autodataprepnode properties
astimeintervalsnode properties
binningnode properties
derivenode properties
ensemblenode properties
fillernode properties
filternode properties
historynode properties
partitionnode properties
reclassifynode properties
reordernode properties
restructurenode properties
rfmanalysisnode properties
settoflagnode properties
transposenode properties
typenode properties
Modeling node properties
Common modeling node properties
anomalydetectionnode properties
apriorinode properties
associationrulesnode properties
autoclassifiernode properties
Setting algorithm properties
autoclusternode properties
autonumericnode properties
bayesnetnode properties
c50node properties
carmanode properties
cartnode properties
chaidnode properties
coxregnode properties
decisionlistnode properties
discriminantnode properties
extensionmodelnode properties
factornode properties
featureselectionnode properties
genlinnode properties
glmmnode properties
gle properties
kmeansnode properties
kmeansasnode properties
knnnode properties
kohonennode properties
linearnode properties
linearasnode properties
logregnode properties
lsvmnode properties
neuralnetworknode properties
questnode properties
randomtrees properties
regressionnode properties
sequencenode properties
slrmnode properties
svmnode properties
tcmnode properties
ts properties
treeas properties
twostepnode properties
twostepAS properties
Model nugget node properties
applyanomalydetectionnode properties
applyapriorinode properties
applyassociationrulesnode properties
applyautoclassifiernode properties
applyautoclusternode properties
applyautonumericnode properties
applybayesnetnode properties
applyc50node properties
applycarmanode properties
applycartnode properties
applychaidnode properties
applycoxregnode properties
applydecisionlistnode properties
applydiscriminantnode properties
applyextension properties
applyfactornode properties
applyfeatureselectionnode properties
applygeneralizedlinearnode properties
applyglmmnode properties
applygle properties
applygmm properties
applykmeansnode properties
applyknnnode properties
applykohonennode properties
applylinearnode properties
applylinearasnode properties
applylogregnode properties
applylsvmnode properties
applyneuralnetworknode properties
applyocsvmnode properties
applyquestnode properties
applyrandomtrees properties
applyregressionnode properties
applyselflearningnode properties
applysequencenode properties
applysvmnode properties
applytcmnode properties
applyts properties
applytreeas properties
applytwostepnode properties
applytwostepAS properties
applyxgboosttreenode properties
applyxgboostlinearnode properties
hdbscannugget properties
kdeapply properties
Graph node properties
collectionnode properties
distributionnode properties
dvcharts properties
evaluationnode properties
histogramnode properties
multiplotnode properties
plotnode properties
timeplotnode properties
webnode properties
Output node properties
analysisnode properties
dataauditnode properties
extensionoutputnode properties
kdeexport properties
matrixnode properties
meansnode properties
reportnode properties
setglobalsnode properties
simfitnode properties
statisticsnode properties
tablenode properties
transformnode properties
Export node properties
dataassetexport properties
extensionexportnode properties
Python node properties
gmm properties
hdbscannode properties
kdemodel properties
kdeexport properties
ocsvmnode properties
rfnode properties
smotenode properties
xgboostlinearnode properties
xgboosttreenode properties
Spark node properties
isotonicasnode properties
kmeansasnode properties
multilayerperceptronnode properties
xgboostasnode properties
SuperNode properties
CLEM (legacy) language reference
Building CLEM (legacy) expressions
About CLEM
CLEM examples
Values and data types
Expressions and conditions
Working with strings
Handling blanks and missing values
Working with numbers
Working with times and dates
Summarizing multiple fields
Working with multiple-response data
The Expression Builder
Accessing the Expression Builder
Creating expressions
Selecting functions
Database functions
Selecting fields
Viewing or selecting values
Checking CLEM expressions
Find
CLEM datatypes
Integers
Reals
Characters
Strings
Lists
Fields
Dates
Time
CLEM operators
Functions reference
Conventions in function descriptions
Bitwise integer operations
Comparison functions
Conversion functions
Date and time functions
Converting date and time values
Functions handling blanks and null values
Global functions
Information functions
Logical functions
Numeric functions
Probability functions
Random functions
Sequence functions
SoundEx functions
Spatial functions
Special fields
String functions
Trigonometric functions
SPSS algorithms
Working with your data
Missing data values
Handling missing values
Handling records with missing values
Handling fields with missing values
Handling records with system missing values
Functions available for missing values
Temporary storage
Supported data sources for SPSS Modeler
Building custom images to install ODBC drivers
Building custom images to install R and Python packages
Configuring Teradata to use LDAP
Adding comments and annotations
Deploying SPSS Modeler flows and models
Creating and scheduling jobs
Flow scripting
Flow scripting example
Flow and SuperNode parameters
SQL optimization
How does SQL pushback work?
Tips for maximizing SQL pushback
Nodes supporting SQL pushback
CLEM expressions and operators supporting SQL pushback
Generating SQL from model nuggets
Disabling or caching nodes in a flow
Disabling nodes in a flow
Caching options for nodes
Importing an SPSS Modeler stream
Setting properties for flows
Expression Builder
Selecting functions
Visualizing your data
Chart types
3D charts
Bar charts
Box plots
Bubble charts
Candlestick charts
Circle packing charts
Custom charts
Dendrogram charts
Dual Y-axes charts
Error bar charts
Evaluation charts
Heat map charts
Histogram charts
Line charts
Map charts
Math curve charts
Multi-chart charts
Multiple series charts
Parallel charts
Pareto charts
Pie charts
Population pyramid charts
Q-Q plots
Radar charts
Relationship charts
Scatter plots and dot plots
Scatter matrix charts
Series array charts
Sunburst charts
t-SNE charts
Time plots
Theme River charts
Tree charts
Treemap charts
Word cloud charts
Global visualization preferences
Db2 Big SQL
Analyzing data stored on S3 or S3 compatible object store services
Using a Jupyter notebook with Db2 Big SQL
Apache Hadoop
Machine learning models in Jupyter Python
Using Hadoop utilities
Using delegation token endpoints
Apache Spark
Getting started with Spark applications
Submitting Spark jobs
Spark jobs API syntax, parameters and return codes
spark.local.dir configuration parameter
Supported application languages and versions
Run Spark applications interactively
Adding custom configurations
Configuring Spark log level information
Persisting applications
Spark labs - Development environment
Enabling application autoscaling
Disaggregated Shuffle
Run Spark Streaming applications
Accessing data from applications
Monitoring Spark jobs through the Spark user interface
Accessing and customizing the Spark history server
Accessing Spark job driver logs
Using custom packages
Customizing Spark applications and notebooks using the analyticsengine CR configuration
Customizing using service volume instances
Using an external metastore for Spark SQL
Using IBM Cloud Data Engine
Spark libraries
Data skipping for Spark SQL
Time series analysis
Using the time series library
Time series key functionality
Time series functions
Time series lazy evaluation
Time reference system
Geospatial data analysis
Topological relations
Geohashing functions
Geospatial indexing
Ellipsoidal metrics
Routing functions
Parquet modular encryption
Key management by application
Key management by KMS
Spark service CLI
Configuring an Analytics Engine powered by Apache Spark instance for watsonx.data
Integrating with watsonx.data on Cloud Pak for Data
Integrating with watsonx.data on IBM Cloud
Integrating with watsonx.data stand-alone
Getting started with watsonx.data Spark use cases
Watson Machine Learning Accelerator
Supported data sources
Deploying and managing AI assets
Deploying AI assets
Considerations for upgrading Watson Machine Learning and watsonx.ai
Deployment spaces
Creating deployment spaces
Deployment space collaborator roles
Importing spaces and projects
Exporting space assets
Deleting deployment spaces
Assets in deployment spaces
Promoting assets to a deployment space
Adding data assets to a deployment space
Adding models to a deployment space
Adding code packages to a deployment space
Accessing asset details
Managing asset versions
Enabling model tracking with AI Factsheets
Deploying AI assets programmatically
Watson Machine Learning Python client example notebooks
Deploying foundation model assets
Deploying a prompt template
Deploying a tuned foundation model
Deploying a detached prompt template
Deploying a custom foundation model
Planning to deploy a custom foundation model
Supported hardware, model architectures, and performance settings
Creating custom hardware specifications
Setting up storage and uploading the model
Setting up storage and uploading a model that is located on Hugging Face
Setting up storage and uploading a model that is located in your environment
Registering a custom foundation model
Properties and parameters for your custom foundation model
Shutting down and restarting custom foundation model deployment
Creating a deployment for a custom foundation model
Managing a deployment for a custom foundation model
Prompting a custom foundation model
Deploying machine learning assets
Creating online deployments
Creating batch deployments
Compute requirements for batch deployment jobs
Data sources for scoring batch deployments
Batch deployment input details by framework
AutoAI models
Decision Optimization models
Python functions
Python scripts
Pytorch models
R scripts
Scikit-learn and XGBoost models
Spark and PMML models
SPSS models
Tensorflow models
Using multiple inputs for an SPSS job
Deploying Python functions
Writing deployable Python functions
Deploying Shiny apps
Deploying Shiny Apps from the GUI
Deploying Shiny Apps with curl and REST API
Deploying Shiny Apps with cpdctl
Connecting your Shiny application to a persistent storage volume
Example script to deploy a Shiny app
Deploying code packages
Deploying NLP models
Deploying scripts
Creating a deployment job
Creating jobs in deployment spaces for SPSS Modeler flows
Managing deployment jobs
Creating deployment jobs to run code packages
Default hardware configurations for deployments
Updating a deployment
Scaling a deployment
Evaluating deployments in spaces
Deleting a deployment
Deploying Decision Optimization models
Deploying a model by using the user interface
Deployment steps
Model deployment
Model execution
Model input and output data file formats
Model input and output data adaptation
Output data definition
Solve parameters
Running jobs
REST API example
Deploying Java models
Changing Python version in a deployed model with REST API
Python client examples
Delegating the CPLEX engine solve to Watson Machine Learning
Frameworks and software specifications
Supported software specifications
Constricted software specifications
Discontinued software specifications
Managing outdated software specifications or frameworks
Customizing deployment runtimes
Requirements for using custom components in ML models
Customizing with third-party and private Python libraries
Customizing with conda
Customizing with pip
Building deployment runtime images
Getting registry URL
Downloading the runtime definition
Downloading the runtime image
Creating a custom image
Customizing Python and SPSS deployment images
Pushing the custom image to image registry
Creating a new base software specification
Uploading the changed configuration
Managing AI Lifecycle with ModelOps
Deployments dashboard
Managing AI lifecycle with CPDCTL
Example: Creating a script asset with CPDCTL
Orchestrating tasks with Pipelines
Getting started with Pipelines
Planning a pipeline
Run the built-in sample pipeline
Creating a pipeline
Configuring pipeline nodes
Managing pipeline settings
Storage and data access
Configuring global objects
Adding conditions to a pipeline
Functions used in pipelines Expression Builder
DataStage functions used in pipelines Expression Builder
Handling pipeline errors
Programming a pipeline
Samples for programming a pipeline
Creating custom components
Running and saving pipelines
AI solutions
Voice Gateway
Watson Assistant for Voice Interaction
Watson Discovery
Watson Speech services
watsonx Assistant
watsonx Code Assistant for Red Hat Ansible Lightspeed
watsonx Code Assistant for Z
watsonx Code Assistant for Z Code Explanation
watsonx Orchestrate
Reference
Cloud Pak for Data APIs
Available APIs
Generating an API auth token
Generating a bearer token
cpd-cli command reference
cpd-cli command syntax conventions
asset
asset attachment create
asset attachment delete
asset attachment download
asset attachment get
asset attachment mark-complete
asset attachment update
asset attribute create
asset attribute delete
asset attribute get
asset attribute list
asset attribute update
asset create
asset data-asset create
asset data-asset get
asset data-asset upload
asset delete
asset export cancel
asset export download
asset export get
asset export list
asset export start
asset export wait
asset file download
asset file list
asset file upload
asset get
asset import cancel
asset import get
asset import list
asset import start
asset import wait
asset promote
asset script create
asset search
asset type get
asset type list
asset type replace
asset update
backup-restore
init
quiesce
repository
repository list
repository validate
reset
snapshot
snapshot create
snapshot delete
snapshot list
snapshot status
snapshot-restore create
snapshot-restore delete
snapshot-restore list
snapshot-restore status
unquiesce
version
volume-backup
volume-backup create
volume-backup delete
volume-backup download
volume-backup list
volume-backup logs
volume-backup purge
volume-backup status
volume-backup unlock
volume-backup upload
volume-restore
volume-restore create
volume-restore delete
volume-restore list
volume-restore logs
volume-restore status
code-package
code-package create
code-package delete
code-package get
code-package list
code-package promote
code-package revision create
code-package revision delete
code-package revision get
code-package revision list
config
profiles
profiles get
profiles list
profiles set
profiles unset
users
users get
users list
users set
users unset
version
connection
connection create
connection datasource-type get
connection datasource-type list
connection delete
connection discover-adhoc
connection discover-data-asset
connection discover
connection get
connection list-actions
connection list
connection perform-action
connection update
datastage
datastage codegen generate-buildop
datastage flow clone
datastage flow compile
datastage flow create
datastage flow delete
datastage flow get
datastage flow list
datastage flow update
datastage migration create-from-zip
datastage migration create
datastage migration delete-zip
datastage migration delete
datastage migration export-flows-with-dependencies
datastage migration get-zip
datastage migration get
datastage subflow clone
datastage subflow create
datastage subflow delete
datastage subflow get
datastage subflow list
datastage subflow update
datastage table-definition clone
datastage table-definition create
datastage table-definition delete
datastage table-definition get
datastage table-definition list
datastage table-definition replace
datastage table-definition update
datastage xml-schema-library clone
datastage xml-schema-library create
datastage xml-schema-library delete-files
datastage xml-schema-library delete
datastage xml-schema-library download
datastage xml-schema-library get
datastage xml-schema-library list
datastage xml-schema-library rename
datastage xml-schema-library upload
diag
delete-job
download-logs
gather-diagnostics
healthcheck
list-jobs
list-services
version
environment
environment create
environment delete
environment get
environment hardware-specification create
environment hardware-specification delete
environment hardware-specification get
environment hardware-specification list
environment hardware-specification update
environment list
environment package-extension create
environment package-extension delete
environment package-extension get
environment package-extension list
environment package-extension update
environment package-extension upload-complete
environment software-specification add-package-extensions
environment software-specification create
environment software-specification delete
environment software-specification get
environment software-specification list
environment software-specification remove-package-extensions
environment software-specification update
environment update
export-import
export
export create
export delete
export download
export list
export logs
export purge
export status
export upload
import
import create
import delete
import list
import logs
import status
init
list
list aux-module
reset
schedule-export
schedule-export create
schedule-export delete
schedule-export download
schedule-export list
schedule-export logs
schedule-export purge
schedule-export status
schedule-export upload
version
health
cluster
network-connectivity
network-performance
nodes
operands
operators
runcommand
service-functionality
service-functionality cleanup
service-functionality list
storage-performance
storage-validation
version
job
job create
job delete
job get
job list
job run cancel
job run create
job run delete
job run download-results
job run get
job run list
job run logs
job run pause
job run resume
job run wait
job serving-name
job update
manage
Component IDs
add-cred-to-global-pull-secret
add-icr-cred-to-global-pull-secret
apply-cluster-component-scale-config
apply-cluster-components
apply-cr
apply-db2-kubelet
apply-entitlement
apply-icsp
apply-olm
apply-privileged-monitoring-service
apply-rsi-patches
apply-scale-config
apply-scc
apply-scheduler
apply-service-monitor
authorize-instance-topology
case-download
clean-workspace
collect-state
copy-image
create-physical-location
create-proxy-config
create-rsi-patch
delete-cr
delete-images
delete-olm-artifacts
delete-physical-location
delete-platform-ca-certs
delete-privileged-monitoring-service
delete-proxy
delete-rsi-patch
delete-service-monitor
deploy-knative-eventing
disable-proxy
edit-physical-location
enable-cpd-config-ac
enable-proxy
enable-rsi
gen-platform-ca-certs
get-cluster-component-scale-config
get-cpd-instance-details
get-cr-status
get-k8s-details
get-license
get-olm-artifacts
get-proxy-config
get-rsi-patch-info
get-rsi-patch-logs
get-physical-locations
get-scale-config
install-cpd-config-ac
install-rsi
list-components
list-dependents
list-deployed-components
list-images
list-platform-ca-certs-pods
list-prereqs
load-image
login-entitled-registry
login-private-registry
login-to-ocp
mirror-images
pull-image
register-physical-location
remove-entitlement
remove-knative-eventing
restart
restart-container
save-image
setup-appconnect
setup-iam-integration
setup-instance-topology
setup-mcg
setup-route
setup-tethered-ns
show-minimum-rbac
Giving a user the minimum RBAC to install the scheduling service
Giving the NamespaceScope operator the minimum RBAC to manage an instance Cloud Pak for Data
Giving a user the minimum RBAC to setup and install Cloud Pak for Data
Applying the required permissions to projects (namespaces) as an instance administrator
Giving a user the minimum RBAC to install Cloud Pak for Data components
shutdown
test-image
uninstall-cpd-config-ac
uninstall-rsi
update-cr
ml
ml deployment compute-predictions
ml deployment create
ml deployment delete
ml deployment get
ml deployment-job create
ml deployment-job-definition create-revision
ml deployment-job-definition create
ml deployment-job-definition delete
ml deployment-job-definition get
ml deployment-job-definition list-revisions
ml deployment-job-definition list
ml deployment-job-definition update
ml deployment-job delete
ml deployment-job get
ml deployment-job list
ml deployment list
ml deployment update
ml deployment wait
ml experiment create-revision
ml experiment create
ml experiment delete
ml experiment get
ml experiment list-revisions
ml experiment list
ml experiment update
ml function create-revision
ml function create
ml function delete
ml function download-code
ml function get
ml function list-revisions
ml function list
ml function update
ml function upload-code
ml model create-revision
ml model create
ml model-definition create-revision
ml model-definition create
ml model-definition delete
ml model-definition download-model
ml model-definition get
ml model-definition list-revisions
ml model-definition list
ml model-definition update
ml model-definition upload-model
ml model delete-content
ml model delete
ml model download-content
ml model filtered-download
ml model get
ml model list-attachments
ml model list-revisions
ml model list
ml model update
ml model upload-content
ml model wait
ml pipeline create-revision
ml pipeline create
ml pipeline delete
ml pipeline get
ml pipeline list-revisions
ml pipeline list
ml pipeline update
ml training create
ml training-definition create-revision
ml training-definition create
ml training-definition delete
ml training-definition get
ml training-definition list-revisions
ml training-definition list
ml training-definition update
ml training delete
ml training get
ml training list
ml training wait
notebook
notebook create
notebook delete
notebook revert
notebook update
notebook version create
notebook version delete
notebook version get
notebook version list
notebook version update
oadp
backup
backup create
backup delete
backup download
backup list
backup logs
backup posthooks
backup precheck
backup prehooks
backup status
backup-location
backup-location create
backup-location delete
backup-location list
checkpoint
checkpoint create
checkpoint list
checkpoint reset
client
client config get
client config set
dpa
dpa create
dpa delete
dpa list
dpa status
du-pv
gather
gather info
install
reset
restore
restore create
restore delete
restore list
restore logs
restore posthooks
restore precheck
restore preworkloadhooks
restore status
snapshot-location
snapshot-location create
snapshot-location list
uninstall
version
pipeline
pipeline commit
pipeline copy
pipeline delete
pipeline get
pipeline get-template
pipeline list
pipeline upload
pipeline validate
pipeline cleanup delete-artifacts
pipeline cleanup delete-cache
pipeline run get
pipeline run get-artifact
pipeline run get-log
pipeline run get-logs
pipeline run get-status
pipeline run list-parameter
pipeline run migrate-cache
pipeline run task-result get
pipeline run task-result head
pipeline run task-result list
pipeline version delete
pipeline version get
pipeline version get-template
pipeline version get-template-format
pipeline version list
pipeline version upload
project
project create
project delete
project get
project list
project member create
project member delete
project member get
project member list
project member update
project update
service-instance
apply-acl
create
delete
get
get-acl
list
status
upgrade
version
space
space create
space delete
space get
space list
space member create
space member delete
space member get
space member list
space member update
space update
space wait
user-mgmt
add-ldap-group-to-cpd-group
add-ldap-user-to-cpd-group
bulk-upsert-ud-roles
bulk-upsert-users
config-ldap
get-group
get-user
list-groups
list-ud-roles
list-users
upsert-group
upsert-user
version
wx-ai
Deployments
wx-ai deployment create
wx-ai deployment list
wx-ai deployment get
wx-ai deployment update
wx-ai deployment delete
wx-ai deployment text-generate
wx-ai deployment text-generate-stream
wx-ai deployment wait
Foundation model specifications
wx-ai foundation-model list-models
wx-ai foundation-model list-tasks
Prompts
wx-ai prompt create
wx-ai prompt get
wx-ai prompt update
wx-ai prompt delete
wx-ai prompt update-lock
wx-ai prompt get-lock
wx-ai prompt get-input
wx-ai prompt add-chat-item
Prompt sessions
wx-ai prompt-session create
wx-ai prompt-session get
wx-ai prompt-session update
wx-ai prompt-session delete
wx-ai prompt-session add-entry
wx-ai prompt-session list-entries
wx-ai prompt-session add-chat-item
wx-ai prompt-session update-lock
wx-ai prompt-session get-lock
wx-ai prompt-session get-entry
wx-ai prompt-session delete-entry
Training
wx-ai training create
wx-ai training list
wx-ai training get
wx-ai training delete
wx-ai training wait
Text
wx-ai text generate
wx-ai text generate-stream
wx-ai text tokenize
wx-ai text calculate-embeddings
Troubleshooting
Troubleshooting the platform
Cannot log in to the web client
Finding and removing hidden pods
Kubelet configurations are not applied correctly
Cluster imbalance overwhelms worker nodes
Worker nodes show status of disk pressure
Requests time out when you upload or download files from volumes
Failure uploading assets to a project
Recovering access to projects
Troubleshooting global search
OpenSearch cluster nodes get out of sync
PostgreSQL cluster replicas get out of sync
Increasing the number of nginx worker connections
EDB Postgres OCP Upgrade
Troubleshooting installation and upgrade
Cannot run cpd-cli manage commands
Cannot access CASE packages when using a proxy server
The setup-instance-topology command fails
Troubleshooting the apply-olm command
Upgrades fail when global search is configured incorrectly
Troubleshooting backup and restore
Troubleshooting problems with the Cloud Pak for Data OADP backup and restore utility
Troubleshooting problems with IBM Storage Fusion
Troubleshooting problems with Portworx data replication
OADP backup fails with request signature error
IBM Knowledge Catalog glossary artifacts are not correctly restored
Restore with IBM Storage Fusion fails with no space left on device error
OADP backup or checkpoint operation fails with error message that one is already in progress
Offline backup fails due to cpdbr-vol-mnt pod stuck in Pending state
OADP online backup partially fails with timeout error
Post-restore hooks cannot run due to Informix service is not deployed
Insufficient storage error when restoring an online backup to a different cluster
Resource quotas cause backup errors
Offline restore fails due to partially successful posthooks
Unable to restore Data Virtualization with Portworx disaster recovery
Offline restore fails with Create OperandRequest Timeout warnings
Restore with Portworx DR fails with operand request and subscription timeout warnings
Restore fails due to missing resources
Offline post-restore hooks fail
Creating a backup with IBM Storage Fusion fails at precheck-backup step
OpenSearch pods not running after restore
StatefulSet error when restoring Cloud Pak for Data
Checkpoint job fails with no space left on device error
EDB Postgres cluster is in an unhealthy state after a failed online backup
Backups stored on Ceph storage take a long time and then fail
Restoring Cloud Pak for Data with IBM Storage Fusion fails or is only partially successful
PVC files remain after restoring an online backup on IBM Storage Scale storage fails
SignatureDoesNotMatch error when running cpd-cli oadp commands
Unable to run an online backup or restore operation
Backup image is permanently in Deleting state
Unable to initialize the volume backup and restore utility
Problems when stopping a volume backup or restore job before it finishes
Unable to synchronize a deprecated Git project in Watson Studio after restoring an online backup
Unable to restore volume backup
Speech export utility fails during data export
Connections
Analytics Engine powered by Apache Spark
Cognos Dashboards
Database services
Enabling the DMC to activate replication
Firewall issues with database ports
Troubleshooting Db2 and Db2 Warehouse
Updating the Db2 SSL certificate after the Cloud Pak for Data self-signed certificate is updated
Db2 licensing issues
NFS Version 4
PureData System workloads
Reinitializing an HADR configuration to resolve errors
Pod restarts repeatedly
Unable to connect to Db2 through Db2Rest
Data Gate
Fixing incorrect Db2 configuration settings
Instance not fully removed
Data Gate instance failed after upgrade
Upgrade flag not deleted after upgrade
Data Gate server is in error or inactive state
The Data Gate server is unavailable
Security configuration fails
Synchronization security configuration error AQT10503E
Updating the access token for the target database
Timeout when publishing to IBM Knowledge Catalog
Catalog access 403 error
Managing ICH70001I messages
Avoiding table state ERROR
Target database in BACKUP PENDING state
Failed to extract certificates from the PKCS12 certificate file
Creating a Data Gate instance does not show a valid target database
The Data Gate instance is stuck in UPGRADE_IN_PROGRESS state
The Data Gate service upgrade fails with InProgress error message
Messages
Data Product Hub
Data Refinery
Data Virtualization
Service statuses
Troubleshooting general issues
Resolving metastore issues after storage outage or pods restart
SQL6031N error in the db2nodes.cfg file
Validation hangs when you run a query after you scale worker pods
Worker pods cannot connect
Troubleshooting installation and upgrade issues
BIGSQL database remains in a "backup pending" state
Upgrading a Data Virtualization instance that uses the ocs-storagecluster-cephfs storage class delays the instance upgrade
Upgrading a Data Virtualization instance fails because of missing or incorrect stg-class-file
Provisioning a Data Virtualization instance does not complete
Service instance fails to start
SQL30082N error after you update
Data Virtualization fails to start after provisioning
Virtualized table list needs manual updates after upgrade to 5.0 and later
Query of a virtualized table fails with Presto after upgrade
Troubleshooting backup and restore issues
Data Virtualization online restore post hooks time out
Data Virtualization pods are unreachable
Failure on backup and restore
Refresh dmcuser password after backup and restore
Reset a secret after backup and restore
Troubleshooting virtualization issues
Japanese column names are not displayed correctly
Virtualizing a table with many columns fails
Error message when you try to use an unsupported file format in Cloud Object Storage
Speed up loading of tables when you virtualize
Reveal hidden tables when you virtualize
Listing of virtual objects is slow
Viewing of objects in your cart is slow
Troubleshooting governance issues
Cannot grant users access to a view
Access to a table is denied by policies
Cannot access assets in the catalog
Data protection rules are not enforced
Cannot see business term that is assigned to data asset
A virtualized object cannot be used in Cognos Dashboards without credentials and an appropriate role
Troubleshooting data source connections
ERRORCODE=-4214, SQLSTATE28000 error when you create a connection to Data Virtualization with API key
CID and TDEF columns are not returned
Restoring a connection that was removed from Platform connections
Adding a connection to Amazon RDS for MySQL fails
Error when you remove a virtualized table in object storage
Adding a remote data source with vaulted credentials fails
A data source cannot be added with personal credentials from a vault
Cannot push down string functions with string units on Db2 remote data source
Snowflake connection times out
Cannot connect to data source
Cannot preview data from Salesforce
Cannot push down join views
Errors when you delete a connection
Remote connector does not start after restart
Stack overflow errors
Troubleshooting data caches and queries
Caches stay in deleting state
Cannot delete schemas or virtual objects
Errors finding cache recommendations
Troubleshooting queries
SQL messages
SQL5105N error when you run a query
SQL20478 error when you run a query
SQL0727N error when you query view results
Slow performance for Data Virtualization queries that are linked to pushdown issues for data sources
Errors when you run a query or virtualize objects
SQL1822N error when you run a query
Concurrent queries are slow or fail
Incorrect query results for Db2 remote data sources
Inaccurate query results for Netezza remote data sources
Inaccurate query results for Db2 z/OS remote data sources
Inaccurate query results for Hive data sources
Data type STRING in Hive tables is assigned CLOB data type
Performance issues in queries with subqueries
SUM() or AVG() function returns an error
Unexpected preview or query results for virtualized tables over flat files
Troubleshooting statistics collection
Collecting statistics on virtualized tables over flat files might fail
Column-level statistics not collected due to data type mismatch
DataStage
Db2 Big SQL
Troubleshooting Db2 SQL messages
Operations against an object store are hanging or timing out
Investigating problems that are encountered during interactions with an object store
SQL5197N or SQL5199N error during INSERT or SELECT operations
Access error after changing the role of a Db2 Big SQL user
Db2 Data Management Console
Tables are not getting displayed after upgrade
Cannot connect to databases
Load logs are cleared after upgrading the console
Event monitors might not work correctly after reinstalling the console
Db2 Data Management Console is not displayed in the web interface
Resolving timeout error
Cannot connect to Redis
Configuring the JDBC properties to the target database
Adding an instance to Db2 Data Management Console manually
Setting up a repository fails
Explain pods fail to start
Resolving Redis connection issues
Redis pods fail to start
Resolving administrator privilege issues
SQL pods fail to start
Hadoop environments
Troubleshooting with the Execution Engine for Apache Hadoop diagnostics tool
IBM Knowledge Catalog
Solving governance artifacts import problems
Synchronize the data policy service (DPS) category caches
Processes time out before completing
Re-creating predefined data classes
Cannot submit profile jobs or preview assets
New columns do not appear in preview of a connected asset
Relationship analysis doesn't complete within the default timeout period
Running concurrent metadata import jobs on multiple metadata-discovery pods
IBM Match 360 with Watson
OpenPages
Setting up a tethered project fails
Service times out
Changing locales does not refresh all browser windows and tabs
ESG dashboards do not display
Error generating the reporting schema
Product Master
Database connection errors
Host unreachable error
SPSS Modeler
Synthetic Data Generator
Watson OpenScale
Watson Studio
Watson Machine Learning
watsonx Assistant
watsonx.data
watsonx.governance
Glossary