SSQNUZ_5.4.x - Documentation Index

Table of Contents

• Welcome
• Overview of IBM Cloud Pak for Data
  • What's new
    • AI Factsheets
    • Analytics Engine powered by Apache Spark
    • Cognos Dashboards
    • Common core services
    • Data Gate
    • Data Privacy
    • Data Refinery
    • Data Replication
    • DataStage
    • Data Virtualization
    • Db2
    • Db2 Big SQL
    • Db2 Data Management Console
    • Db2 Warehouse
    • Decision Optimization
    • EDB Postgres
    • Execution Engine for Apache Hadoop
    • IBM Knowledge Catalog
    • IBM Manta Data Lineage
    • IBM Master Data Management
    • Informix
    • MANTA Automated Data Lineage
    • MongoDB
    • OpenPages
    • Orchestration Pipelines
    • RStudio Server Runtimes
    • SPSS Modeler
    • Watson Machine Learning
    • Watson OpenScale
    • Watson Studio
    • Watson Studio Runtimes
  • Use cases
    • Data intelligence use case
    • Data integration use case
    • Data science and MLOps use case
    • AI governance use case
  • Asset types and properties
    • Searching for assets
    • Previews
    • Profiles
    • Data quality
    • Relationships
    • Visualizations
    • Activities
  • AI augmentations in curation tools
  • Known issues
• Installing and administering Cloud Pak for Data
  • Administering AI Factsheets
  • Administering Data Privacy
  • Administering Data Virtualization
    • Managing roles for users and groups
      • Assigning roles to users
      • Assigning roles to groups
    • Administering auditing in Data Virtualization
      • Monitoring user activity
      • Enabling and disabling auditing
      • Auditing user-initiated SQL statements in Data Virtualization
      • Customizing audit policies
      • Db2 audit events in Data Virtualization
    • Administering queries and caches
      • Fetch phase warnings and errors
        • Enabling and disabling fetch phase warnings
      • Enabling autocaching
      • Setting Federation Server options
      • Setting the query mode
  • Administering Db2 Big SQL
    • Monitoring Db2 Big SQL
  • Administering IBM Knowledge Catalog
    • Custom properties, relationships, and asset types
      • Creating custom asset types
      • Creating custom properties
      • Creating custom relationships
      • Importing custom properties or relationships from a file
      • Managing custom properties, relationships, and asset types
    • Managing rule settings
    • Connecting to a watsonx.ai instance
    • Updating projects and metadata with a new embedding model
  • Administering Watson OpenScale
• Getting started and tutorials
  • Roadmaps for getting started
  • Setting up your console and first steps
    • Setting your preferences
    • Finding services
    • Checking your permissions
    • Identifying tasks that need to be completed
  • Generating API keys
    • Generating ZenApiKey authorization tokens
    • Generating a bearer token
    • Generating an authorization token for the admin user
  • Available APIs
  • Switching between experiences
  • Quick start tutorials
    • Prepare data
      • Refine data
      • Transform data
      • Virtualize data
    • Analyze data
      • Analyze data in a Jupyter notebook
    • Build machine learning models
      • Build a model with AutoAI
      • Build a model in a Jupyter notebook
      • Build a model with SPSS Modeler
      • Build a Decision Optimization model
    • Govern AI
      • Evaluate a deployment in spaces
      • Evaluate a machine learning model
  • Use case tutorials
    • Build and deploy a model
    • Test and validate the model
    • Orchestrate an AI pipeline with data integration
    • Orchestrate an AI pipeline with model monitoring
    • Integrate data
    • Virtualize external data
    • Replicate data
    • Configure a 360 degree view
    • Curate high quality data
    • Protect your data
    • Consume your data
    • Govern virtualized data
  • Industry accelerators
  • Video library
  • Getting started with watsonx.data intelligence MCP server
    • MCP configuration file and environment variables
    • Configuring IBM Bob
    • Configuring Claude Desktop
    • Configuring watsonx Orchestrate
    • Configuring GitHub Copilot in Visual Studio Code
    • Setting up and using skills
• Projects
  • Creating a project
    • Importing a project
    • Importing project assets
    • Accessing a Git repository
    • Creating access token for collaboration
    • Configuring custom encryption for access tokens
  • Choosing a tool
  • Integrating with Git repositories
    • Default Git integration
      • Git operations
      • Project terminal
      • Connecting to data sources
    • Deprecated Git integration
      • Syncing projects
      • Syncing when working remotely
  • Administering projects
    • Managing collaborators
    • Project collaborator roles
    • Marking a project as sensitive
    • Logging project activity
    • Exporting project assets
    • Enabling folders
    • Defining default settings for tools
      • Data intelligence
        • Providing additional context for SQL generation
      • Data quality
      • Job execution windows
      • Metadata enrichment
        • Adding a custom service for term assignment
        • CSV file for rule-based term assignment
        • Tuning term assignment
        • Custom abbreviation files for name generation
  • Managing assets in projects
    • Organizing assets with folders
  • Environments
    • Notebook environments
    • Spark environments
    • Data Refinery environments
    • SPSS Modeler environments
    • RStudio environments
    • JupyterLab environments
    • GPU environments
    • Hadoop environments
    • Creating non-standard environment templates
    • Customizing environments
      • Customizing environment templates
        • Customizing environment templates with pip
        • Examples of customizations
    • Managing environments
    • Viewing active runtimes across projects
  • Creating and managing jobs
    • Creating jobs in the Notebook editor
    • Creating jobs in Data Refinery
    • Creating jobs in IBM Master Data Management
    • Creating jobs in SPSS Modeler
    • Creating jobs for code files
    • Creating jobs in DataStage
    • Creating jobs with Masking flow
    • Creating jobs for running data quality rules
    • Creating jobs for Pipelines
    • Viewing jobs across projects
  • Leaving a project
  • Markdown cheatsheet
• Deployment spaces
  • Creating deployment spaces
  • Deployment space collaborator roles
  • Deleting deployment spaces
  • Assets in deployment spaces
    • Promoting assets to a deployment space
    • Importing spaces and projects
    • Exporting space assets
    • Adding data assets to a deployment space
    • Importing models to a deployment space
    • Adding code packages to a deployment space
    • Accessing asset details
    • Managing asset versions
• Managing data
  • The Db2 Data Management Console
    • Managing alerts
    • Creating and scheduling jobs
    • Creating monitor reports
    • Tuning
      • Creating a tuning task
      • Choosing EXPLAIN and tuning options
      • Generating access plan graph
      • Generating statistics recommendations
      • Generating index recommendations
      • Doing an index what-if analysis
      • Changing table organization
    • Setting up monitoring profile
      • Setting up alerts
      • Creating custom alerts
    • Setting up event monitor profile
  • Db2 databases
  • Db2 Big SQL databases
  • Data Gate
  • Db2 Warehouse databases
  • EDB Postgres databases
  • Informix databases
  • MongoDB databases
• Preparing data
  • Supported data sources
    • Connectors
      • Amazon Aurora for MySQL
      • Amazon Aurora for PostgreSQL
      • Amazon DynamoDB
      • Amazon RDS for MySQL connection
      • Amazon RDS for Oracle connection
      • Amazon RDS for PostgreSQL connection
      • Amazon Redshift connection
      • Amazon S3 connection
        • Setting up temporary credentials or a Role ARN for Amazon S3
      • Apache Cassandra connection
      • Apache Cassandra for DataStage connection
      • Apache Derby connection
      • Apache HBase connection
      • Apache HDFS connection
      • Apache Hive connection
      • Apache Hive Metastore connection
      • Apache Impala connection
      • Apache Kafka connection
      • Apache Spark SQL connection
      • AWS Databricks
      • Box connection
      • ClickHouse connection
      • Collibra connection
      • Confluence connection
      • DataStax Enterprise connection
      • DataStax HCD connection
      • Denodo connection
      • Dremio connection
      • Dropbox connection
      • Elasticsearch connection
      • Elastic Cloud connection
      • Exasol connection
      • FTP connection
      • Generic JDBC connection
      • Generic S3 connection
      • Google BigQuery connection
        • Workload identity federation examples
      • Google Cloud Pub/Sub connection
      • Google Cloud Storage connection
      • Google Drive connection
      • Google Looker connection
      • Greenplum connection
      • HDFS via Execution Engine for Hadoop connection
      • Hive via Execution Engine for Hadoop connection
      • HTTP connection
      • IBM Cloud Data Engine connection
      • IBM Cloud Databases for MongoDB connection
      • IBM Cloud Databases for MySQL
      • IBM Cloud Databases for PostgreSQL connection
      • IBM Cloud Object Storage connection
      • IBM Cloud Object Storage (infrastructure) connection
      • IBM Cloudant connection
      • IBM Cognos Analytics connection
      • IBM Data Replication Access Server connection
      • IBM Data Virtualization connection
      • IBM Data Virtualization Manager for z/OS connection
      • IBM DataStage for Cloud Pak for Data connection
      • IBM Db2 connection
        • Setting up a Db2 connection that uses TLS and SSL
      • IBM Db2 for DataStage connection
      • IBM Db2 Big SQL connection
      • IBM Db2 for i connection
      • IBM Db2 for z/OS connection
      • IBM Db2 on Cloud connection
      • IBM Db2 Warehouse connection
      • IBM FileNet P8 connection
      • IBM Informix connection
      • IBM Master Data Management connection
      • IBM MQ connection
      • IBM Netezza Performance Server connection
      • IBM Netezza Performance Server for DataStage connection
      • IBM Planning Analytics connection
      • IBM Product Master connection
      • IBM SPSS Analytic Server connection
      • IBM watsonx.data Milvus connection
      • IBM watsonx.data Presto connection
      • IBM watsonx.data SharePoint connection
      • Iceberg metastore connection
      • Impala via Execution Engine for Hadoop connection
      • Informatica PowerCenter connection
      • MariaDB connection
      • Microsoft Azure Blob Storage connection
      • Microsoft Azure Cosmos DB connection
      • Microsoft Azure Data Lake Storage connection
      • Microsoft Azure Databricks connection
      • Microsoft Azure Fabric Warehouse connection
      • Microsoft Azure File Storage connection
      • Microsoft Azure PostgreSQL connection
      • Microsoft Azure SQL Database connection
      • Microsoft Azure Synapse Analytics connection
      • Microsoft OneDrive connection
      • Microsoft Power BI (Azure) connection
      • Microsoft Power BI (Local) connection
      • Microsoft SharePoint Files connection
      • Microsoft SharePoint connection
      • Microsoft SQL Server connection
      • Microsoft SQL Server Integration Services (SSIS) connection
      • Microsoft SQL Server Reporting Services (SSRS) connection
      • MicroStrategy connection
      • Milvus connection
      • MinIO connection
      • MongoDB connection
      • MySQL connection
      • OData connection
      • ODBC connection
      • OpenLineage connection
      • OpenSearch connection
      • OpenSearch IBM Cloud connection
      • Oracle connection
      • Oracle Database for DataStage connection
      • Oracle Business Intelligence Enterprise Edition connection
      • Oracle Data Integrator connection
      • PostgreSQL connection
      • Presto connection
      • Qlik Sense connection
      • Salesforce.com connection
      • Salesforce API for DataStage connection
      • SAP ASE connection
      • SAP BAPI connection
      • SAP Bulk Extract connection
      • SAP BusinessObjects connection
      • SAP BW Extract connection
        • Creating a connection for SAP BW Extract with SNC encryption
      • SAP Delta Extract connection
      • SAP HANA connection
      • SAP IDoc connection
      • SAP IQ connection
      • SAP OData connection
      • SingleStoreDB connection
      • Slack connection
      • Snowflake connection
      • Statistical Analysis System (SAS) connection
      • Storage volume connection
      • Tableau connection
      • Talend Data connection
      • Teradata connection
      • Teradata database for DataStage connection
      • Trino connection
      • Vertica connection
  • Adding data to a project
    • Adding connections to projects
      • Using secrets from vaults in connections
    • Adding integrated service connectors
    • Adding data from a connection
    • Adding a dynamic view of data from a connection
      • Adding a query-based asset
        • Providing additional context for text-to-SQL conversions
      • Adding segmented assets
    • Adding a connected folder asset from a connection
    • Accessing data sources with Flight service
      • Flight data requests
      • Supported connectors in Flight service
      • Flight service in Python notebooks
        • Examples of Flight service data requests with Python
        • Connecting to Flight service manually in Python
        • Using itc_utils with your own code
        • Examples of Flight service with itc_utils and pandas
        • Resolving timeouts in Flight service
        • Using Flight with Apache Spark
      • Flight service in R notebooks
        • Examples of Flight service data requests with R
        • Connecting to Flight service manually in R
      • Best practices for loading large volumes of data from a file or connection
    • Downloading data assets
    • Adding catalog assets to a project
    • Publishing assets to a catalog
  • Adding platform connections
    • Managing collaborators on platform connections
  • Building custom connectors with Connector forge
    • Creating and deploying custom connectors
    • Troubleshooting custom connectors
  • Parametrized connections
  • Data protection with data source definitions
    • Protection solutions for data source definition
    • Connectors that support data source definitions
    • Connectors with hard-coded data source identity properties
    • Roles and asset privacy settings for data source definitions
    • Creating a data source definition
      • Creating a data source definition from the Data source definition list
      • Adding endpoints to a new or existing data source definition
    • Creating a connection from the Data source definitions list
    • Setting connection limits for data source definitions
    • Editing, deactivating, activating, or deleting data source definitions
  • Refining data
    • Adding data to Data Refinery
    • Validating your data
    • Visualizing your data
    • Managing Data Refinery flows
    • Target connection options
    • Refining data on the Hadoop cluster
      • Refining HDFS data
      • Refining data stored in tables in a Hive warehouse
      • Refining data stored in tables in Impala
    • GUI operations
    • Interactive code templates
    • Supported data sources for Data Refinery
  • Curating structured data
    • Importing metadata
      • Importing asset metadata
      • Importing metadata with MANTA Automated Data Lineage
        • Designing metadata imports
        • Asset types created through metadata import
        • Discovering data
        • Importing ETL jobs
          • Preparing ETL job files
        • Importing BI reports
          • Preparing report files
        • Importing data models
          • Preparing data model files
        • Getting lineage
        • Getting ETL job lineage
        • Getting BI report lineage
        • Managing existing metadata imports
        • Metadata import jobs
      • Importing metadata with IBM Manta Data Lineage
        • Designing metadata imports
          • Configuring metadata import for data integration assets
        • Creating metadata imports
        • Managing existing metadata imports
        • Metadata import jobs
    • Enriching your data assets
      • Designing metadata enrichments
        • Term assignment
          • Best practices
        • Random sampling concepts
      • Creating an enrichment asset
      • Managing existing enrichments
      • Managing enrichment jobs
        • Monitoring job runs
      • Reviewing enrichment results
        • Profile details
        • Frequency distributions
      • Reviewing results in a spreadsheet program
      • Making bulk changes to assignments
      • Managing data quality checks
      • Identifying primary keys
      • Identifying relationships
      • Advanced data profiling
      • Publishing enrichment results
    • Supported connectors for discovery, enrichment, and data quality
    • Supported connectors for lineage import
      • Amazon RDS for Oracle lineage configuration
      • Amazon RDS for PostgreSQL lineage configuration
      • Amazon Redshift lineage configuration
      • Apache Hive lineage configuration
      • Google BigQuery lineage configuration
      • Greenplum lineage configuration
      • IBM Cloud Databases for PostgreSQL lineage configuration
      • IBM Cognos Analytics lineage configuration
      • IBM Data Virtualization lineage configuration
      • IBM DataStage for Cloud Pak for Data lineage configuration
      • IBM Db2 lineage configuration
      • IBM Db2 for z/OS lineage configuration
      • IBM Db2 on Cloud lineage configuration
      • IBM Netezza Performance Server lineage configuration
      • Informatica PowerCenter lineage configuration
      • Microsoft Azure Databricks lineage configuration
      • Microsoft Azure SQL Database lineage configuration
      • Microsoft Power BI (Azure) lineage configuration
      • Microsoft Power BI Report Server lineage configuration
      • Microsoft SQL Server lineage configuration
      • Microsoft SQL Server Analysis Services (SSAS) lineage configuration
      • Microsoft SQL Server Integration Services (SSIS) lineage configuration
      • MicroStrategy lineage configuration
      • OpenLineage lineage configuration
      • Oracle lineage configuration
      • PostgreSQL lineage configuration
      • Qlik Sense lineage configuration
      • SAP BusinessObjects lineage configuration
      • SAP HANA lineage configuration
      • Snowflake lineage configuration
      • Statistical Analysis System (SAS) lineage configuration
      • Tableau lineage configuration
      • Talend lineage configuration
      • Teradata lineage configuration
      • YugabyteDB lineage configuration
    • Supported connectors for importing lineage with MANTA Automated Data Lineage
  • Managing data quality
    • Data quality dimensions
    • Types of predefined data quality checks
    • Data quality assets
    • Managing data quality definitions
      • Building blocks for rule logic
      • Sample data quality definitions
      • Sample rule expressions
      • Reserved words
    • Managing data quality rules
      • Creating rules from data quality definitions
      • Creating SQL-based rules
        • Sample SQL rules
      • Configuring output settings for rules
    • Managing data quality rule groups
    • Assessing data quality with rules
    • Data quality analysis results
      • Data quality scores
    • Data quality SLA compliance and remediation
    • Ensuring data quality with data contracts
  • Managing feature groups
  • Transforming data with DataStage
    • Administering
    • Designing flows
      • Asset browser
      • Advanced find search
      • DataStage stages
        • Aggregator
          • Fast path
          • Stage tab
            • Calculation and recalculation dependent properties
        • Bloom Filter
          • Stage tab
        • Change Apply
          • Example data (DataStage)
          • Fast path
        • Change Capture
          • Fast path
          • Stage tab
        • Checksum
          • Adding a Checksum column to your data (DataStage)
          • Properties for Checksum Stage (DataStage)
          • Mapping output columns (DataStage)
          • Specifying execution options (DataStage)
        • Column Export
          • Fast path
          • Stage tab
          • Input tab
          • Output tab
        • Column Generator
          • Fast path
          • Stage tab
        • Column Import
          • Examples (DataStage)
          • Fast path
          • Stage tab
          • Input tab
          • Output tab
        • Combine Records
          • Examples (DataStage)
            • Example 1 (DataStage)
            • Example 2 (DataStage)
          • Fast path
          • Stage tab
            • Properties section
              • Outputs section
              • Combine keys section
              • Options section
            • Advanced section
            • NLS Locale section
        • Compare
          • Fast path
          • Stage tab
        • Compress
          • Fast path
          • Stage tab
        • Copy
          • Fast path
          • Stage tab
        • Decode
          • Fast path
        • Difference
          • Fast path
          • Stage tab
        • Distributed Transaction
        • Encode
          • Fast path
          • Stage tab
        • Excel
          • Configuring the Excel stage as a source
          • Configuring the Excel stage as a target
          • Extracting the data from Microsoft Excel
            • Examples of extracting data from Microsoft Excel files
            • Examples of writing data to Microsoft Excel files
          • Reference
            • Properties reference: Excel
            • Data type conversions from Microsoft Excel to IBM DataStage
            • Data type conversions from DataStage to Microsoft Excel
            • Job abort conditions in Microsoft Excel
            • Consideration about end of wave
        • Expand
          • Fast path
          • Stage tab
        • External Filter
          • Fast path
          • Stage tab
        • External Source
          • Fast path
          • Stage tab
            • Advanced section
            • NLS Map section
          • Output tab
            • Properties section
              • Source section
              • Options section
            • Format section
            • Using RCP With External Source Stages (DataStage)
        • External Target
          • Fast path
          • Stage tab
            • Advanced section
            • NLS Map section
          • Input tab
            • Input link properties section
              • Target section
              • Options section
            • Format section
              • Output tab
          • Using RCP with External Target stages (DataStage)
        • Filter
          • Specifying the filter
            • Input data columns
            • Supported Boolean expressions and operators
              • Order of association (DataStage)
            • String comparison (DataStage)
          • Fast path
          • Stage tab
        • Funnel
          • Fast path
          • Stage tab
            • Link Ordering section
        • Generic
          • Fast path
          • Stage tab
        • Head
          • Fast path
          • Stage tab
        • Hierarchical data
          • Using the Hierarchical Data stage (DataStage)
            • Adding a Hierarchical Data stage to a (DataStage) flow
            • Configuring runtime properties for the Hierarchical Data stage (DataStage)
          • The assembly (DataStage)
            • Input step (DataStage)
            • Output step (DataStage)
            • Assembly Editor (DataStage)
            • Opening the Assembly Editor (DataStage)
          • Schema views
          • Mapping data (DataStage)
            • Working with the mapping table (DataStage)
            • Determining mapping candidates (DataStage)
            • Configuring how mapping candidates are determined (DataStage)
          • XML Composer step (DataStage)
            • XML Composer validation rules (DataStage)
          • XML Parser step (DataStage)
            • XML Parser validation rules (DataStage)
            • Setting default values for types (DataStage)
          • JSON transformation (DataStage)
            • Schema management (DataStage)
              • Opening the Schema Library Manager (DataStage)
              • Working with libraries and resources (DataStage)
              • Creating a JSON schema in the schema library (DataStage)
            • JSON Parser step (DataStage)
              • JSON Parser validation rules (DataStage)
            • JSON Composer step (DataStage)
              • JSON Composer validation rules (DataStage)
          • REST operator in (DataStage)
            • REST step pages
              • General
              • Security
              • Request
              • Response
              • Mappings
            • Output schema of the REST step
          • Passing multiple rows from an XML or JSON file
          • Transformation steps for the Hierarchical Data stage (DataStage)
            • Aggregate step (DataStage)
            • H-Pivot step (DataStage)
            • HJoin step (DataStage)
            • Order Join step (DataStage)
            • Regroup step (DataStage)
            • Sort step (DataStage)
            • Union step (DataStage)
            • V-Pivot step (DataStage)
        • Java Integration
          • Java code integration
            • Setting up your development environment
            • Implementing abstract methods of the Processor class
            • Compiling the Java code
            • Running the Java code on the Parallel Engine, Java Integration stage
            • Accessing Stage Configuration
            • Declaring the Capabilities of the Java code
            • Reading Records from Input Link
            • Writing Records to Output Link
            • Rejecting Records
            • Looking up data in the Sparse lookup mode
            • Data Types
            • Retrieving Column Metadata on the Link
            • Using user-defined properties
            • Runtime column propagation
            • Running the Java code on the conductor node
            • Transferring data from the conductor node to player nodes
            • Logging messages with the Java Integration stage
            • Terminating a job from the Java code
            • Using JavaBeans
            • User Defined Function
          • Designing jobs with the Java Integration stage
            • Configuring Java Integration stage as a source
            • Configuring Java Integration stage as a transformer
            • Configuring Java Integration stage as a target
            • Looking up data by using reference links
            • Setting up column definitions
          • Java libraries
          • Properties reference: Java Integration
        • Join
          • Join versus lookup
          • Fast path
          • Stage tab
        • Lookup
          • Lookup versus Join
          • Fast path
          • Properties
            • Stage tab
            • Input and Output tab
          • Sparse and normal Lookup
        • Make Subrecord
          • Fast path
          • Stage tab
          • Examples (DataStage)
        • Make Vector
          • Examples (DataStage)
            • Example 1 (DataStage)
            • Example 2 (DataStage)
          • Fast path
          • Stage tab
            • Properties section
              • Options section
            • Advanced section (DataStage)
        • Merge
          • Fast path
          • Stage tab
        • Modify
          • Fast path
          • Stage tab
        • Operational Decision Manager
          • Overview
            • Engine mode
            • Batch mode and key mode processing
            • Execution Object Model types
            • Fields and ruleset parameters mapping
            • Field propagation strategies
          • Java class path configuration
          • Designing a job
            • Creating a flow
            • Configuring the stage to invoke the ruleset
            • Configuring input and output link properties
              • Links and ruleset parameters association
              • Buffer links
              • Null value handling
            • Configuring the reject link
            • Compiling the flow
          • Configuration wizard
            • Configuring the stage with the wizard
          • Basic Java types and methods
            • DataStage type to Java type mapping
            • Java type to DataStage type mapping
          • Properties reference
        • Peek
          • Fast path
          • Stage tab
        • Pivot Enterprise
          • Specifying a horizontal pivot operation (DataStage)
            • Specifying a horizontal pivot operation and mapping output columns (DataStage)
            • Example of horizontally pivoting data (DataStage)
          • Specifying a vertical pivot operation (DataStage)
            • Specifying a vertical pivot operation and mapping output columns (DataStage)
            • Example of vertically pivoting data (DataStage)
          • Properties tab
            • Specifying execution options (DataStage)
            • Specifying where the stage runs (DataStage)
            • Specifying partitioning or collecting methods (DataStage)
            • Specifying a sort operation (DataStage)
        • Promote Subrecord
          • Examples (DataStage)
            • Example 1 (DataStage)
            • Example 2 (DataStage)
          • Fast path
          • Stage tab
            • Properties tab
              • Options section
            • Advanced tab
        • Remove Duplicates
          • Fast path
          • Stage tab
        • REST
          • Stage tab
          • Examples of using REST stage
        • Row Generator
          • Fast path
          • Stage tab
          • Output tab
        • Sample
          • Fast path
          • Stage tab
            • Link Ordering
        • Slowly Changing Dimension
          • Job design
          • Purpose codes
          • Surrogate keys
          • Editing
            • Defining the match condition
            • Selecting purpose codes
              • Purpose code definitions
            • Specifying information about a key source
            • Creating derivations for dimension columns
          • Dimension update action
        • Sort
          • Fast path
          • Stage tab
        • Split Subrecord
          • Fast path
          • Stage tab
          • Examples (DataStage)
        • Split Vector
          • Examples (DataStage)
            • Example 1 (DataStage)
            • Example 2 (DataStage)
          • Fast path
          • Stage tab
            • Properties tab
              • Options section
            • Advanced tab
        • Stored Procedure
          • Teradata Stored Procedure
        • Surrogate Key Generator
          • Creating the key source (DataStage)
          • Deleting the key source (DataStage)
          • Updating the state file (DataStage)
          • Generating surrogate keys (DataStage)
        • Switch
          • Example
          • Fast path
          • Stage tab
        • Tail
          • Fast path
          • Stage tab
        • Transformer
          • Basic concepts
          • Properties
            • Stage variables (DataStage)
            • Loop variables (DataStage)
              • Entering expressions (DataStage)
              • Loop example: converting a single row to multiple rows (DataStage)
              • Loop example: multiple repeating values in a single field (DataStage)
              • Loop example: generating new rows (DataStage)
              • Loop example: aggregating data (DataStage)
            • Surrogate Key tab (DataStage)
            • Link ordering (DataStage)
            • Triggers
            • Advanced (DataStage)
            • Input tab
            • Output tab
              • Runtime column propagation (DataStage)
            • System variables (DataStage)
            • Evaluation sequences for transformer expressions, stage variables, and loop variables (DataStage)
            • Reserved words
          • Parallel transform functions (DataStage)
            • Date and time functions (DataStage)
            • Logical functions (DataStage)
            • Mathematical functions (DataStage)
            • Null handling functions (DataStage)
            • Number functions (DataStage)
            • Raw functions (DataStage)
            • String functions (DataStage)
              • Retrieving substrings
              • Concatenating strings
            • Vector function (DataStage)
            • Type conversion functions (DataStage)
            • Utility functions (DataStage)
            • Operator functions (DataStage)
          • Function libraries
        • Wave Generator
          • Stage Tab
            • Properties
          • Input tab
            • Output tab
        • Web Service stage
          • Introduction to web services in DataStage
          • Encoding requests and responses
          • Required tasks in the Web Service stage
          • Other tasks
          • Setting up stage properties
          • Setting up input link properties
          • Setting up output link properties
          • Using a Web Service in DataStage
        • Write Range Map
          • Fast path
          • Stage tab
          • Input tab
        • XML Input
          • Stage tab
            • Transformation settings (DataStage)
        • XML Output
          • Using XML Output
            • About transforming tabular data
            • Validating documents and schemas
            • Aggregating input rows on output
            • Writing output to your file system
            • Processing NULLs and empty values
            • Selecting items for the XML Output
            • Setting the format of the XML output
            • Elements: Controlling the order and the repetition
            • Setting up stage properties
            • Setting up input and output link properties
      • DataStage connectors
        • Connecting to a data source in DataStage
        • Supported data sources in DataStage
          • Amazon RDS for PostgreSQL connector
          • Amazon S3 connector
          • Apache HDFS connector
          • Apache Hive connector
          • Apache Kafka connector
          • Apache Impala connector
          • FTP connector
          • Generic JDBC connector
          • Google BigQuery connector
          • Generic S3 connector
          • Greenplum connector
          • IBM Cloud Databases for PostgreSQL connector
          • IBM Data Virtualization Manager for z/OS connector
          • IBM Db2 connector
          • IBM Db2 for DataStage connector
          • IBM Db2 for z/OS connector
          • IBM Informix connector
          • IBM Master Data Management connector
          • Oracle connector
          • Oracle Database for DataStage connector
          • PostgreSQL connector
          • Salesforce API for DataStage connector
          • Snowflake connector
          • Teradata connector
          • Teradata database for DataStage connector
          • IBM watsonx.data Presto connector
        • File connectors in DataStage
          • Complex Flat File
            • Complex Flat File as a source
              • Defining record ID constraints in DataStage
            • Complex Flat File as a target in DataStage
            • Complex Flat File schema
            • Reject links in DataStage
          • Data set in DataStage
          • File set in DataStage
            • Input tab (DataStage)
            • Output tab (DataStage)
          • Lookup file set in DataStage
          • Sequential file in DataStage
        • Data service
          • Introduction
          • Topologies
          • Creating a job that includes a Data service connector
          • Running a job that includes a Data service connector
        • Using Before/After SQL Statements
          • Before SQL (DataStage)
          • Before SQL (node) in DataStage
          • After SQL (DataStage)
          • After SQL (node) in DataStage
        • Using stored procedures
          • Syntax
        • Using multiple links
        • Connecting to Oracle data sources with LDAP, SEPS or TCPS
          • Connecting to Oracle data sources with LDAP authentication
          • Connecting to Oracle data sources with SEPS authentication
          • Connecting to Oracle data sources with TCPS authentication
        • Configuring Db2 connections to work with database sequences
        • Setting up the DB2 partition type
        • Common properties for DataStage connectors
        • Decimal rounding modes
      • Quality stages in DataStage
        • Designing match specifications
          • Adding passes
            • Match comparisons in DataStage
              • Reverse matching
              • ABS_DIFF comparison
              • AN_DINT comparison
              • AN_INTERVAL comparison
              • CHAR comparison
              • CNT_DIFF comparison
              • D_INT comparison
              • D_USPS comparison
              • DATE8 comparison
              • DELTA_PERCENT comparison
              • DISTANCE comparison
              • INT_TO_INT comparison
              • INTERVAL_NOPAR comparison
              • INTERVAL_PARITY comparison
              • LR_CHAR comparison
              • LR_UNCERT comparison
              • MULT_ALIGN comparison
              • MULT_EXACT comparison
              • MULT_RANGE comparison
              • MULT_UNCERT comparison
              • NAME_UNCERT comparison
              • NUMERIC comparison
              • PREFIX comparison
              • PRORATED comparison
              • TIME comparison
              • UNCERT comparison
                • Adding QSM_MAT_UNCERT_VERSION environment variable
              • USPS comparison
              • USPS_DINT comparison
              • USPS_INT comparison
          • Testing passes
          • Matching data in DataStage
            • How matching is done
              • Match types
              • Match column selection in DataStage
              • Match passes in DataStage
              • Blocking in DataStage
            • Weights and record comparisons
        • Address Verification
          • Scenario: Improving international address data
          • Installing reference files
          • Parsing, validating, and transliterating address data
          • Errors
        • Investigate
        • Match Frequency
        • One-source Match
        • Two-source Match
        • Standardize
          • Standardize rule sets
            • Objects within rule sets
              • Classifications
              • Lookup tables
              • Output columns
              • Rules
              • Overrides
          • Quality stage pattern action reference
            • Introduction to the Pattern Action language
            • Parsing elements
            • Unconditional patterns
              • Identifying simple pattern classes
            • Conditional patterns
              • Simple conditional values
              • Conditional expressions
              • Using arithmetic expressions
            • Action statements
              • Copying information
              • Referencing dictionary fields from another rule set
              • Moving information
              • Concatenating information
              • Converting information
              • Retyping operands
              • Retyping multiple tokens
              • Patterning
              • Rule set extensions
              • User overrides for domain preprocessor rule sets
              • User overrides for rule sets
              • Setting margins
              • SOUNDEX phonetic coding
              • NYSIIS coding
              • Terminating pattern matching
              • Calling subroutines
              • Writing subroutines
              • Performing actions repetitively
              • Summary of sources and targets
        • Survive
        • Data Quality Rule
      • Specifying your own stages
        • Extending DataStage by using included samples
        • Defining build stages in DataStage
          • General tab
          • Properties tab
          • Build tab
          • Build stage macros
            • Informational macros (DataStage)
            • Flow-control macros (DataStage)
            • Input and output macros (DataStage)
            • Transfer Macros (DataStage)
            • How your code is executed (DataStage)
            • Inputs and outputs (DataStage)
              • Using multiple inputs (DataStage)
            • Example Build stage in DataStage
          • Header files
            • C++ classes - sorted by header file (DataStage)
            • C++ macros - sorted by header file (DataStage)
        • Defining custom stages
          • Compiling custom stages
          • Using custom stages
        • Defining wrapped stages in DataStage
          • Running scripts in a wrapped stage
      • Creating a DataStage component
        • Message handlers
      • Defining data definitions
      • Reusable job design with subflows
        • Local subflows
        • Subflows
      • Parameters and parameter sets
        • Creating and using local parameters in DataStage
        • Creating and using parameter sets
        • Inserting parameters and parameter sets as properties in DataStage
        • Configuring parameters for vaulted properties
        • Configuring runtime parameters in a flow
        • Configuring runtime parameters in a job
        • PROJDEF parameter set
        • Passing values from parameter sets into jobs by command-line interface
      • Environment variables
        • Managing environment variables
        • Guide to setting environment variables in DataStage
        • Buffering
        • Checkpoint
        • Compiler
        • Db2 Support
        • Debugging
        • Decimal support
        • Disk I/O
        • General Job Administration
        • Look up support
        • Miscellaneous
        • Network
        • NLS support
        • Oracle support
        • Partitioning
        • Reading and writing files environment variables in DataStage
        • Reporting environment variables in DataStage
        • SAS support
        • Sorting environment variables in DataStage
        • Sybase support environment variables in DataStage
        • Teradata support
        • Transport blocks
        • WLM
      • National Language Support (NLS)
        • Creating custom NLS maps
      • Reject links
      • Partitioning and collecting data
        • Writing partitioned data in file-storage connectors
    • Running jobs
      • Settings for the project, flow, and job level
      • Setting up before-job and after-job subroutines
        • Cataloging a Db2 database in the runtime container
      • Scheduling DataStage jobs
      • Compile options with SQL Pushdown in DataStage
      • ELT run mode
        • ELT materialization policies in DataStage
      • Macros
      • Observing DataStage jobs
      • Performance factors overview
      • Job run metrics
    • Migrating jobs
      • Migrating connections
        • Migrating an ODBC connection
        • Migrating SAP connections
        • Migrating a Teradata connection
        • Migrating a Teradata database for DataStage connection with the Teradata program ID
        • Configuring DataStage jobs with Kerberos connection properties after migration
        • Setting up vault connections for migrated jobs
      • Migrating a flow with user-defined functions
      • Migrating the Data Rules stage as a Quality Rule
      • Changing XML file locations after migration
      • Configuring the Hierarchical stage after migration in DataStage
      • Migrating a flow with a custom stage
      • Migrating the Java Transformer stage
      • Migrating BASIC routines
      • Migrating Web Service Transformer and Web Service Client stages
      • Naming storage volumes after migrating jobs
      • Asset import report (DataStage)
      • Accessing the routine script
    • Downloading and importing flows
    • Development, testing, and production
      • Deployment spaces
    • DataStage Anywhere
      • Creating a remote engine
      • Managing a remote engine
        • Test connection on remote engines
        • Troubleshooting a remote runtime engine for DataStage Anywhere
      • Frequently asked questions for DataStage Anywhere
      • Security for remote engines with DataStage Anywhere
      • Sharing the existing remote engine across projects in IBM Cloud Pak for Data
      • ODBC support for the remote engine
    • DataStage command-line tools
    • DataStage APIs
    • Orchestrating flows with Orchestration Pipelines
      • Pipeline components for DataStage
      • CEL expressions and limitations
      • Migrating and constructing pipeline flows
        • Using Send email script in migrated pipeline flows
      • Replacing BASIC routines
        • Examples
      • Sharing storage volumes
      • DataStage jobs in pipeline flows example
      • DataStage optimized runner
    • DevOps for DataStage
      • Managing assets
      • Getting started with Unit Testing
      • Configuring test data storage
      • Creating a DataStage test case
        • Capturing test data
        • Recapturing test result baseline
      • Editing a DataStage test case
        • DataStage specification format
          • Testing flow by using date/time references
          • Row count comparisons
          • Excluding columns from tests
          • Using Cluster Keys for high volume DataStage tests
      • Running a DataStage test case
      • Verifying a DataStage test results
      • Migrating test cases from older DataStage versions
  • Virtualizing data
    • Adding and connecting to data sources in Data Virtualization
      • Data source connection access privileges in Data Virtualization
        • Transferring ownership of data sources
      • Connecting with personal credentials
      • Supported data sources
      • Connecting to Amazon S3
      • Connecting to Ceph
      • Connecting to IBM Cloud Object Storage
      • Connecting to Google BigQuery
      • Connecting to MinIO
      • Connecting to REST API
        • Configuring pagination for REST API connector
      • Connecting to SAP HANA
      • Connecting to SAP HANA on SAP Cloud Platform
      • Connecting to Snowflake
      • Connecting to Spark SQL
      • Connecting with generic JDBC driver
      • Filtering data
      • Status of data sources
      • Limitations for data sources in Data Virtualization
    • Accessing data sources by using remote connectors
      • Installing remote connectors
      • Supported file types for remote data sources
      • Setting the character encoding scheme
      • Managing connectors on remote data sources
      • Discovering remote data sources
    • Creating virtual objects
      • Creating a virtualized table from a single data source table
      • Creating a virtualized table from a single data source native query
      • Creating a virtualized table from a filesystem file accessed by a remote agent
      • Creating a virtualized table from multiple data sources
      • Creating a virtualized table from files in Cloud Object Storage
      • Creating schemas for virtual objects
      • Joining virtual objects
        • Joining virtual objects in SQL builder
        • Joining virtual objects in Data Virtualization
      • Removing a virtualized object
    • Managing access to virtual objects
      • Managing access to virtual objects
      • Revoking access to virtual objects
      • Managing visibility of virtual objects
    • Governing virtual data
      • Virtualizing data with business terms in default virtualization mode
      • Virtualizing data with business terms in strict virtualization mode
        • Enabling strict mode
        • Disabling strict mode
      • Publishing virtual data to a catalog
      • Manually creating a connection in the catalog
      • Governing virtual data with data protection rules
        • Authorization model for views
        • Allowing and denying access to data in Data Virtualization
        • Masking virtual data
          • Masking data with Mask at Read semantics in Data Virtualization
        • Row-level filtering in Data Virtualization
        • Configuring PEP cache settings
    • Managing and customizing caches and queries
      • Adding data caches
      • Autocaching
      • Cache recommendations
        • Configuring cache recommendations
        • Finding cache recommendations
      • Viewing query history
      • Restrictions for caching
    • Improving query performance
      • Collecting statistics
        • Collecting statistics in the web client
        • Collecting statistics by using SQL
          • Checking for missing statistics
          • Collecting statistics for data in object storage
          • Collecting statistics for all other data sources
        • Data source specific limitations
      • Enabling caching
      • Creating function mappings
        • Supported data sources for function mappings in Data Virtualization
        • Specific data source type
        • Specific SSPD server
    • Monitoring and exploring the service
      • Monitoring integrated databases
      • Exploring integrated databases
    • Developing database applications
    • SQL interface
      • Running SQL
      • Data Virtualization procedures
        • REMOVESCHEMAFROMASSETS stored procedure
        • SETCONNECTIONCONFIGPROPERTIES stored procedure
          • SETCONNECTIONCONFIGPROPERTY properties
        • SETCONNECTIONCONFIGPROPERTY stored procedure
        • VIRTUALIZCOSOBJECT stored procedure
        • VIRTUALIZENATIVEQUERY stored procedure
        • SETCONFIGPROPERTY stored procedure
          • SETCONFIGPROPERTY properties
        • TOGGLEPERSONALCREDENTIALSUSEINCACHE stored procedure
        • EXT_AUTHORIZER_EXPLAIN stored procedure
        • COLLECT_STATISTICS stored procedure
        • EXPLAIN_FORMAT stored procedure
        • MIGRATE_GROUP_AUTHZ stored procedure
        • removeCosConn stored procedure
        • removeRdbcX stored procedure
        • setCosConn stored procedure
        • setRdbcX stored procedure
        • setRdbcX stored procedure
        • VIRTUALIZETABLE stored procedure
        • LISTCOSBUCKETS stored procedure
        • LISTCOSOBJECTS stored procedure
        • VIRTUALIZEFILE stored procedure
      • Data Virtualization views
        • LISTCOLUMNS view
        • NUMTABSREMOTE view
        • LISTCONNECTIVITYTESTWARNINGS view
        • LISTTABLES view
        • LISTSCHEMAS view
        • LISTCATALOGS view
        • LISTTHREADS view
        • NUMTHREADS view
        • LISTNODES view
        • LISTRDBC view
        • LISTRBDCDETAILS view
        • LISTREMOTEWARNINGS view
        • LISTCONFIG view
        • LISTSOURCES view
  • Masking data with Masking flow
    • Creating masking flows
    • Running masking flow jobs
      • Managing job performance
  • Managing master data
    • Creating a master data configuration asset
    • Configuring access to data
      • Configuring token-based access control
      • Configuring attribute-based access control
    • Data concepts
    • Matching algorithms
    • Working with governed data
    • Migrating data into IBM Master Data Management
      • Migrating from InfoSphere MDM
        • Enabling proxy communication with IBM MDM
          • Configuring additional proxy transactions
          • Configuring proxy mapping
          • Configuring proxy properties
          • Configuring proxy XML templates
          • Configuring proxy query logic
      • Migrating from InfoSphere Big Match
    • Configuring master data
      • Customizing your data types
      • Adding data and mapping it to your data types
      • Connecting data to IBM Master Data Management
      • Adding master data from InfoSphere MDM
      • Defining attribute composition rules
      • Matching your data to create master data entities
      • Customizing and strengthening your matching algorithm
        • Advanced matching algorithm tuning using the API
      • Discovering relationships in master data
      • Defining hierarchy types
      • Defining group types
      • Configuring history tracking
      • Managing master data tags
      • Configuring IBM Master Data Management workflows
        • Configuring a potential match workflow
        • Configuring a potential overlay workflow
      • Saving and loading configuration snapshots
      • Managing IBM Master Data Management jobs
      • Streaming record and entity data changes
        • Event streaming message template
    • Completing pair reviews
    • Exploring master data
      • Defining the way records and attributes are displayed
      • Exploring master data entities and records
      • Adding and editing records and entities
      • Maintaining record and entity linkage
      • Analyzing and comparing master data
      • Completing data stewardship tasks
        • Remediating potential matches to improve data quality
        • Remediating potential overlay issues
      • Exploring relationships in master data
      • Exploring hierarchies in master data
      • Exploring groups in master data
      • Exploring historical master data
      • Exporting master data
        • Advanced master data exports using the API
    • Managing reference data with code tables
      • Understanding code tables
        • Main language code table
      • Creating a code table
      • Adding new types of reference data
      • Updating reference data
      • Supporting multiple languages
      • Integrating code tables with data types
      • Creating hierarchical code tables
    • APIs available in IBM Master Data Management
  • Replicating data
    • Running replication jobs
      • Large object handling options
    • Supported Data Replication connections
      • Replicating Amazon RDS for PostgreSQL data
        • Supported PostgreSQL data types
      • Replicating Apache Kafka data
      • Replicating data with an IBM Data Replication Access Server connection
      • Replicating IBM Db2 data
        • Supported IBM Db2 data types
      • Replicating IBM Db2 on Cloud data
        • Supported IBM Db2 on Cloud data types
      • Replicating IBM Db2 Warehouse data
      • Replicating IBM Db2 for z/OS data
      • Replicating IBM watsonx.data data
      • Replicating Microsoft SQL Server data
        • Supported Microsoft SQL Server data types
      • Replicating Oracle data
        • Supported Oracle data types
      • Replicating PostgreSQL data
    • Choosing a business goal
      • Configuring the Change Log business goal
        • Change Log types
        • Change Log tables
    • Monitoring replication jobs
    • Managing replication jobs
• Data science solutions
  • Notebooks and scripts
    • Planning your notebooks and scripts experience
    • Jupyter Notebook editor
      • Creating and managing notebooks
      • Parts of a notebook
      • Jupyter kernels and notebook environments
      • Coding and running notebooks
      • Markdown cheatsheet
    • JupyterLab
      • JupyterLab with default Git integration
        • Organizing your code files
      • JupyterLab with deprecated Git integration
    • Visual Studio Code
    • RStudio
      • Working in RStudio with default Git integration
      • Working in RStudio with deprecated Git integration
      • Working in RStudio with no Git integration
      • Using Spark in RStudio
      • Using libs from Anaconda Repository
      • Accessing data in MySQL databases by using the RMariaDB library
      • Adding custom R packages
    • Libraries and scripts
      • Installing custom libraries
      • Importing scripts into a notebook
    • Watson Natural Language Processing
      • Working with pre-trained models
      • Library task catalog
        • Language detection
        • Syntax analysis
        • Noun phrase extraction
        • Keyword extraction and ranking
        • Entity extraction
        • Embeddings
        • HAP detection
        • Sentiment extraction
        • Tone classification
        • Emotion classification
        • Relations extraction
        • Hierarchical categorization
          • Category types
      • Creating your own models
        • Detecting entities with a custom dictionary
        • Detecting entities with regular expressions
        • Detecting entities with a custom transformer model
        • Classifying text with a custom classification model
        • Extracting sentiment with a custom transformer model
        • Extracting targets sentiment with a custom transformer model
      • Usage samples
    • Geospatial data analysis
    • Data skipping for Spark SQL
    • Parquet encryption
      • Key management by application
      • Key management by KMS
    • Time series analysis
      • Using the time series library
      • Time series key functionality
      • Time series functions
      • Time series lazy evaluation
      • Time reference system
    • SPSS predictive analytics algorithms
      • Data preparation
      • Classification and regression
      • Clustering
      • Forecasting
      • Survival analysis
      • Score
    • Loading and accessing data in a notebook
      • Loading data through generated code snippets
      • Adding data from a Planning Analytics connection
      • Accessing project assets with ibm-watson-studio-lib
        • ibm-watson-studio-lib for Python
        • ibm-watson-studio-lib for R
        • Migrating from project-lib for Python to ibm-watson-studio-lib
        • Migrating from project-lib for R to ibm-watson-studio-lib
      • Accessing project assets with project-lib
        • project-lib for Python (deprecated)
        • project-lib for R (deprecated)
      • Managing feature groups with assetframe-lib for Python
    • Managing the notebooks and scripts lifecycle
      • Automating the lifecycle of notebooks and scripts
      • Sharing notebooks
        • Hiding code in a notebook
  • Building interactive visualizations with Cognos Dashboards
    • Before you start
      • Supported data sources
      • Supported local file formats
    • Creating a dashboard
    • Editing a dashboard
    • Creating a relationship
      • Join operators
      • Join optimization
    • Sharing a dashboard
    • Publishing a dashboard to a catalog
    • Adding a dashboard from a catalog
    • Checking and refreshing data sources
    • Enabling dashboards to use governed virtual data
    • Migrating dashboards to Cognos Analytics
  • AutoAI for machine learning
    • AutoAI glossary
    • Automating a machine learning experiment
      • Configuring experiment settings
      • Configure a text analysis experiment
    • Automating a time series forecast experiment
      • Tutorial: AutoAI univariate time series experiment
      • Tutorial: AutoAI supporting features time series experiment
      • Time series experiment implementation details
      • Scoring a time series model
    • Saving an AutoAI generated notebook
    • Running an AutoAI notebook on IBM Z and IBM LinuxONE
    • Selecting an AutoAI model
    • AutoAI library
    • AutoAI implementation details
    • Configuring data imputation
      • Imputation details for time series experiments
    • Using incremental learning to train pipelines
      • Incremental learning details
    • Evaluating AutoAI experiments for fairness
    • Troubleshooting AutoAI experiments
  • Deep learning
    • Deep learning experiment tutorial using MNIST data set
    • Deep learning experiment tutorial using CIFAR data set
  • Decision Optimization
    • Ways to use Decision Optimization
    • Supported data sources in Decision Optimization
    • Sample models and notebooks
    • Decision Optimization notebooks
    • Decision Optimization experiments
      • Creating a Decision Optimization experiment
      • Scenarios
        • Opening the scenario pane
        • Creating a scenario from scenario pane
        • Creating a scenario from Overview
        • Importing a scenario
        • Exporting a scenario
        • Viewing all scenarios
        • Generating a notebook from a scenario
        • Comparing scenarios
        • Comparing scenario tables
        • Comparing scenario models and log files
        • Saving scenarios for deployment
      • Configuring a Decision Optimization experiment
        • Changing the deployment space for your experiment
        • Changing default environments and adding Python libraries
      • Preparing input data
        • Importing data into a scenario
        • Editing input data
        • Table search and filtering
        • Exporting data to a project
      • Building a model
        • Code snippets for building models
        • Referencing imported data and defining output
        • Multiple model files
        • Importing a model
        • Configuring engine settings in a model
      • Solving a model
        • Intermediate solutions
        • Configuring the run parameters for a scenario
        • Selecting a different run environment for a scenario
      • Exploring the solution
      • Visualization view
        • Visualization widgets
        • Displaying multiple scenarios
        • Visualization widget syntax
          • Note widgets
          • Table widgets
          • Chart widgets
          • Vega Chart widgets
          • Gantt widgets
          • Pivot table widgets
      • Python DOcplex models
        • Input and output data
        • Solving and analyzing a model: the diet problem
          • Changing the scenario data
        • Working with multiple scenarios
          • Generating multiple scenarios
        • Engine settings
      • Modeling Assistant models
        • Selecting a Decision domain in the Modeling Assistant
        • Formulating and running a model: house construction scheduling
        • Adding multi-concept constraints and custom decisions: shift assignment
        • Creating advanced custom constraints with Python
      • OPL models
    • Decision Optimization Java models
  • SPSS Modeler
    • Getting started in SPSS Modeler
      • Mining data in SPSS Modeler
      • Mining text data with Text Analytics
        • About text mining
        • How extraction works
        • How categorization works
    • Tutorials
      • Introduction to modeling
      • Automate modeling for a flag target
      • Automate modeling for a continuous target
      • Automate data preparation
      • Explore graphs for drug treatment
      • Screen predictors
      • Reduce input data string length
      • Classify telecommunications customers
      • Predict telecommunications churn
      • Forecast bandwidth utilization
      • Forecast catalog sales
      • Make offers to customers (self-learning)
      • Predict retail sales promotions
      • Condition Monitoring
      • Analyze text for hotel satisfaction
    • Understanding and preparing data
      • Importing an SPSS Modeler stream
      • SQL optimization
        • How does SQL pushback work?
        • Tips for maximizing SQL pushback
        • Nodes supporting SQL pushback
        • CLEM expressions and operators supporting SQL pushback
        • Generating SQL from model nuggets
      • Missing data values
        • Handling missing values
          • Handling records with missing values
          • Handling fields with missing values
          • Handling records with system missing values
        • Functions available for missing values
      • Temporary storage
    • Supported data sources for SPSS Modeler
    • Building flows and models
      • Disabling or caching nodes in a flow
        • Disabling nodes in a flow
        • Caching options for nodes
      • Adding comments and annotations
      • Setting properties for flows
      • Creating and scheduling jobs
      • Chart builder
      • Flow scripting
        • Flow scripting example
        • Flow and SuperNode parameters
    • Nodes palette
      • Import
        • Data Asset node
        • Extension Import node
        • Sim Gen node
        • User Input node
      • Record Operations
        • Aggregate node
        • Append node
        • Balance node
        • CPLEX Optimization node
        • Distinct node
        • Extension Transform node
        • Merge node
        • RFM Aggregate node
        • Sample node
        • Select node
        • SMOTE node
        • Space-Time-Boxes node
        • Sort node
        • Streaming TCM node
        • Streaming Time Series node
      • Field Operations
        • Anonymize node
        • Auto Data Prep node
        • Binning node
        • Derive node
        • Ensemble node
        • Field Reorder node
        • Filler node
        • Filter node
        • History node
        • Partition node
        • Reclassify node
        • Restructure node
        • RFM Analysis node
        • Set to Flag node
        • Time Intervals node
        • Transpose node
        • Type node
          • Viewing and setting information about types
          • Measurement levels
            • Geospatial measurement sublevels
          • Converting continuous data
          • What is instantiation?
          • Data values
            • Setting options for values
            • Specifying values and labels for continuous data
            • Specifying values and labels for nominal and ordinal data
            • Specifying values for a flag
            • Specifying values for collection data
            • Specifying values for geospatial data
          • Defining missing values
          • Checking type values
          • Setting the field role
          • Setting field format options
      • Graphs
        • Charts node
        • Collection node
        • Distribution node
        • Evaluation node
        • Histogram node
        • Plot node
        • Multiplot node
        • Time Plot node
        • Web node
      • Modeling
        • Anomaly node
        • Apriori node
        • Association Rules node
        • Auto Classifier node
          • Continuous machine learning
        • Auto Numeric node
        • Auto Cluster node
        • Bayes Net node
        • C5.0 node
        • C&R Tree node
        • CARMA node
        • CHAID node
        • Cox node
        • Decision List node
        • Discriminant node
        • Extension Model node
          • Extension model nugget
        • Feature Selection node
        • Gaussian Mixture node
        • GenLin node
        • GLE node
        • GLMM node
        • HDBSCAN node
        • Isotonic-AS node
        • KDE node
        • K-Means node
        • K-Means-AS node
        • KNN node
        • Kohonen node
        • Linear node
        • Linear-AS node
        • Logistic node
        • LSVM node
        • Neural Net node
        • MultiLayerPerceptron-AS node
        • One-Class SVM node
        • PCA/Factor node
        • QUEST node
        • Random Trees node
        • Random Forest node
        • Regression node
        • Sequence node
        • SLRM node
        • SVM node
        • TCM node
        • Time Series node
        • Tree-AS node
        • TwoStep cluster node
        • TwoStep-AS cluster node
        • XGBoost-AS node
        • XGBoost Linear node
        • XGBoost Tree node
      • Text Analytics
        • Language Identifier node
        • Text Link Analysis node
          • Expert options
          • TLA node output
        • Text Mining node
          • Text Mining model nuggets
          • Text Analytics Workbench
            • Concepts tab
            • Text links tab
            • Categories tab
            • Resource editor tab
              • Rule lab
              • Creating TLA rules with the Rule lab
            • Setting options
              • Advanced linguistic settings
              • Advanced frequency settings
            • Generating a model nugget
        • Linguistic resources
          • Reusing custom linguistic resources
            • Reusing custom category sets
            • Reusing a library
            • Reusing a template
            • Reusing a text analysis package (TAP)
      • Outputs
        • Analysis node
        • Data Audit node
        • Extension Output node
        • KDE Simulation node
        • Matrix node
        • Means node
        • Report node
        • Set Globals node
        • Sim Fit node
        • Sim Eval node
        • Statistics node
        • Table node
        • Transform node
      • Export
        • Data Asset Export node
        • Extension Export node
      • Extension nodes
        • Native Python APIs
        • Python for Spark scripts
          • Scripting with Python for Spark
            • Analytic Server Context
            • Data metadata
            • Date, time, timestamp
            • Exceptions
            • Examples
        • R scripts
      • Decision tree nodes
        • The interactive tree builder
          • Growing and pruning the tree
          • Defining custom splits
          • Saving tree models and results
        • Building a tree model directly
        • Decision tree nodes
      • SuperNodes
    • Promoting SPSS Modeler flows and models
    • Scripting and automation
      • Scripting overview
        • Types of scripts
        • Flow scripts
          • Flow script example: Training a neural net
          • Jython code size limits
        • Running and interrupting scripts
      • The scripting language
        • Python and Jython
        • Python scripting
          • Operations
          • Lists
          • Strings
          • Remarks
          • Statement syntax
          • Identifiers
          • Blocks of code
          • Passing arguments to a script
          • Examples
          • Mathematical methods
          • Using non-ASCII characters
        • Object-oriented programming
          • Defining a class
          • Creating a class instance
          • Adding attributes to a class instance
          • Defining class attributes and methods
          • Hidden variables
          • Inheritance
      • Scripting in SPSS Modeler
        • Flows, SuperNode streams, and diagrams
          • Flows
          • SuperNode flows
          • Diagrams
        • Running a flow
        • The scripting context
        • Referencing existing nodes
          • Finding nodes
          • Setting properties
        • Creating nodes and modifying flows
          • Creating nodes
          • Linking and unlinking nodes
          • Importing, replacing, and deleting nodes
          • Traversing through nodes in a flow
        • Getting information about nodes
      • The scripting API
        • Example: Searching for nodes using a custom filter
        • Metadata: Information about data
        • Parameters
        • Global values
        • Error reporting for flows
      • Scripting tips
        • Looping through nodes
        • Accessing flow run results
          • Table content model
          • XML content model
          • JSON content model
          • Column Statistics content model and Pairwise Statistics content model
      • Properties reference overview
        • Syntax for properties
          • Structured properties
          • Abbreviations
        • Node and flow property examples
        • Node properties overview
          • Common node properties
      • Flow properties
      • Data Asset Import node properties
        • dataassetimport properties
        • extensionimportnode properties
        • simgennode properties
        • userinputnode properties
      • Record Operations node properties
        • appendnode properties
        • aggregatenode properties
        • balancenode properties
        • cplexoptnode properties
        • derive_stbnode properties
        • distinctnode properties
        • extensionprocessnode properties
        • mergenode properties
        • rfmaggregatenode properties
        • samplenode properties
        • selectnode properties
        • sortnode properties
        • streamingtimeseries properties
      • Field Operations node properties
        • anonymizenode properties
        • autodataprepnode properties
        • astimeintervalsnode properties
        • binningnode properties
        • derivenode properties
        • ensemblenode properties
        • fillernode properties
        • filternode properties
        • historynode properties
        • partitionnode properties
        • reclassifynode properties
        • reordernode properties
        • restructurenode properties
        • rfmanalysisnode properties
        • settoflagnode properties
        • transposenode properties
        • typenode properties
      • Modeling node properties
        • Common modeling node properties
        • anomalydetectionnode properties
        • apriorinode properties
        • associationrulesnode properties
        • autoclassifiernode properties
          • Setting algorithm properties
        • autoclusternode properties
        • autonumericnode properties
        • bayesnetnode properties
        • c50node properties
        • carmanode properties
        • cartnode properties
        • chaidnode properties
        • coxregnode properties
        • decisionlistnode properties
        • discriminantnode properties
        • extensionmodelnode properties
        • factornode properties
        • featureselectionnode properties
        • genlinnode properties
        • glmmnode properties
        • gle properties
        • kmeansnode properties
        • kmeansasnode properties
        • knnnode properties
        • kohonennode properties
        • linearnode properties
        • linearasnode properties
        • logregnode properties
        • lsvmnode properties
        • neuralnetworknode properties
        • questnode properties
        • randomtrees properties
        • regressionnode properties
        • sequencenode properties
        • slrmnode properties
        • svmnode properties
        • tcmnode properties
        • ts properties
        • treeas properties
        • twostepnode properties
        • twostepAS properties
      • Model nugget node properties
        • applyanomalydetectionnode properties
        • applyapriorinode properties
        • applyassociationrulesnode properties
        • applyautoclassifiernode properties
        • applyautoclusternode properties
        • applyautonumericnode properties
        • applybayesnetnode properties
        • applyc50node properties
        • applycarmanode properties
        • applycartnode properties
        • applychaidnode properties
        • applycoxregnode properties
        • applydecisionlistnode properties
        • applydiscriminantnode properties
        • applyextension properties
        • applyfactornode properties
        • applyfeatureselectionnode properties
        • applygeneralizedlinearnode properties
        • applyglmmnode properties
        • applygle properties
        • applygmm properties
        • applykmeansnode properties
        • applyknnnode properties
        • applykohonennode properties
        • applylinearnode properties
        • applylinearasnode properties
        • applylogregnode properties
        • applylsvmnode properties
        • applyneuralnetworknode properties
        • applyocsvmnode properties
        • applyquestnode properties
        • applyrandomtrees properties
        • applyregressionnode properties
        • applyselflearningnode properties
        • applysequencenode properties
        • applysvmnode properties
        • applytcmnode properties
        • applyts properties
        • applytreeas properties
        • applytwostepnode properties
        • applytwostepAS properties
        • applyxgboosttreenode properties
        • applyxgboostlinearnode properties
        • hdbscannugget properties
        • kdeapply properties
      • Graph node properties
        • collectionnode properties
        • distributionnode properties
        • dvcharts properties
        • evaluationnode properties
        • histogramnode properties
        • multiplotnode properties
        • plotnode properties
        • timeplotnode properties
        • webnode properties
      • Output node properties
        • analysisnode properties
        • dataauditnode properties
        • extensionoutputnode properties
        • kdeexport properties
        • matrixnode properties
        • meansnode properties
        • reportnode properties
        • setglobalsnode properties
        • simfitnode properties
        • statisticsnode properties
        • tablenode properties
        • transformnode properties
      • Export node properties
        • dataassetexport properties
        • extensionexportnode properties
      • Python node properties
        • gmm properties
        • hdbscannode properties
        • kdemodel properties
        • kdeexport properties
        • ocsvmnode properties
        • rfnode properties
        • smotenode properties
        • xgboostlinearnode properties
        • xgboosttreenode properties
      • Spark node properties
        • isotonicasnode properties
        • kmeansasnode properties
        • multilayerperceptronnode properties
        • xgboostasnode properties
      • SuperNode properties
    • Expression Builder
      • Selecting functions
    • Reference information
      • Tips and shortcuts
      • CLEM (legacy) language reference
        • Building CLEM (legacy) expressions
          • About CLEM
          • CLEM examples
          • Values and data types
          • Expressions and conditions
          • Working with strings
          • Handling blanks and missing values
          • Working with numbers
          • Working with times and dates
          • Summarizing multiple fields
          • Working with multiple-response data
          • The Expression Builder
            • Accessing the Expression Builder
            • Creating expressions
            • Selecting functions
              • Database functions
            • Selecting fields
            • Viewing or selecting values
            • Checking CLEM expressions
          • Find
        • CLEM datatypes
          • Integers
          • Reals
          • Characters
          • Strings
          • Lists
          • Fields
          • Dates
          • Time
        • CLEM operators
        • Functions reference
          • Conventions in function descriptions
          • Bitwise integer operations
          • Comparison functions
          • Conversion functions
          • Date and time functions
            • Converting date and time values
          • Functions handling blanks and null values
          • Global functions
          • Information functions
          • Logical functions
          • Numeric functions
          • Probability functions
          • Random functions
          • Sequence functions
          • SoundEx functions
          • Spatial functions
          • Special fields
          • String functions
          • Trigonometric functions
      • SPSS algorithms
  • Visualizing your data
    • Chart types
      • 3D charts
      • Bar charts
      • Box plots
      • Bubble charts
      • Candlestick charts
      • Circle packing charts
      • Custom charts
      • Dendrogram charts
      • Dual Y-axes charts
      • Error bar charts
      • Evaluation charts
      • Heat map charts
      • Histogram charts
      • Line charts
      • Map charts
      • Math curve charts
      • Multi-chart charts
      • Multiple series charts
      • Parallel charts
      • Pareto charts
      • Pie charts
      • Population pyramid charts
      • Q-Q plots
      • Radar charts
      • Relationship charts
      • Scatter plots and dot plots
      • Scatter matrix charts
      • Series array charts
      • Sunburst charts
      • t-SNE charts
      • Time plots
      • Theme River charts
      • Tree charts
      • Treemap charts
      • Word cloud charts
    • Global visualization preferences
  • Db2 Big SQL
    • Analyzing data stored on S3 or S3 compatible object store services
    • Using a Jupyter notebook with Db2 Big SQL
  • Apache Hadoop
    • Machine learning models in Jupyter Python
    • Using Hadoop utilities
    • Using delegation token endpoints
  • Apache Spark
    • Getting started with Spark applications
      • Submitting Spark jobs
        • Spark jobs API syntax, parameters and return codes
        • spark.local.dir configuration parameter
        • Supported application languages and versions
      • Run Spark applications interactively
        • Adding custom configurations
        • Configuring Spark Environment Variables
      • Configuring Spark log level information
      • Persisting applications
    • Spark labs - Development environment
    • Enabling application autoscaling
    • Disaggregated Shuffle
    • Run Spark Streaming applications
    • Accessing data from applications
    • Monitoring Spark jobs through the Spark user interface
    • Accessing and customizing the Spark history server
    • Accessing Spark job driver logs
    • Using custom packages
      • Customizing Spark applications and notebooks using the analyticsengine CR configuration
      • Customizing using service volume instances
    • Using an external metastore for Spark SQL
      • Using IBM Cloud Data Engine
    • Spark libraries
      • Data skipping for Spark SQL
      • Time series analysis
        • Using the time series library
        • Time series key functionality
        • Time series functions
        • Time series lazy evaluation
        • Time reference system
      • Geospatial data analysis
        • Topological relations
        • Geohashing functions
        • Geospatial indexing
        • Ellipsoidal metrics
        • Routing functions
      • Parquet modular encryption
        • Key management by application
        • Key management by KMS
    • Spark service CLI
    • Configuring an Analytics Engine powered by Apache Spark instance for watsonx.data
      • Integrating with watsonx.data on Cloud Pak for Data
      • Integrating with watsonx.data on IBM Cloud
      • Integrating with watsonx.data stand-alone
      • Getting started with watsonx.data Spark use cases
• Deploying and managing AI assets
  • Deploying AI assets
    • Considerations for upgrading Watson Machine Learning and watsonx.ai
    • Deploying machine learning assets
      • Deploying machine learning assets programmatically
        • Python client example notebooks
      • Creating online deployments
      • Creating batch deployments
        • Compute requirements for batch deployment jobs
        • Data sources for scoring batch deployments
        • Batch deployment input details by framework
          • AutoAI models
          • Decision Optimization models
          • Python functions
          • Python scripts
          • Pytorch models
          • R scripts
          • Scikit-learn and XGBoost models
          • Spark and PMML models
          • SPSS models
          • Tensorflow models
          • Using multiple inputs for an SPSS job
      • Deploying Python functions
        • Writing deployable Python functions
      • Deploying Shiny apps
        • Deploying Shiny Apps from the GUI
        • Deploying Shiny Apps with curl and REST API
        • Deploying Shiny Apps with cpdctl
        • Connecting your Shiny application to a persistent storage volume
        • Example script to deploy a Shiny app
      • Deploying code packages
      • Deploying NLP models
      • Deploying scripts
      • Creating deployments with GPU hardware specifications
      • Creating a deployment job
      • Creating jobs in deployment spaces for SPSS Modeler flows
      • Managing deployment jobs
      • Creating deployment jobs to run code packages
      • Managing hardware specifications for deployments
      • Updating a deployment
      • Scaling a deployment
      • Evaluating deployments
      • Deleting a deployment
    • Deploying Decision Optimization models
      • Deploying a model by using the user interface
      • Deploying a model programmatically
        • Deploying a model
        • Model input and output data file formats
          • OPL model input and output data file formats
          • Python model input and output data file formats
        • Batch deployment and model execution
          • Input data for deployment jobs
          • Output data for deployment jobs
          • Run parameters
          • Running jobs
        • REST API example
      • Deploying Java models
      • Changing Python version in a deployed model with REST API
      • Python client examples
      • Delegating the CPLEX engine solve to Watson Machine Learning
    • Enabling model tracking with AI Factsheets
    • Frameworks and software specifications
      • Supported software specifications
      • Managing outdated software specifications or frameworks
    • Customizing deployment runtimes
      • Customizing runtimes with external libraries and packages
        • Requirements for using custom components with ML models
        • Customizing environment templates with pip
  • Managing AI Lifecycle with ModelOps
    • Deployments dashboard
    • Managing AI lifecycle with CPDCTL
      • Example: Creating a script asset with CPDCTL
  • Orchestrating tasks with Pipelines
    • Getting started with Pipelines
      • Planning a pipeline
    • Run the built-in sample pipeline
    • Creating a pipeline
      • Configuring pipeline nodes
      • Managing pipeline settings
      • Storage and data access
      • Configuring global objects
      • Adding conditions to a pipeline
        • Functions used in pipelines Expression Builder
        • DataStage functions used in pipelines Expression Builder
      • Handling pipeline errors
      • Programming a pipeline
        • Samples for programming a pipeline
      • Creating custom components
    • Running and saving pipelines
    • Configuration management for Orchestration Pipelines
• Enterprise governance
  • Data governance
    • Planning to implement data governance
      • Differences between IBM Knowledge Catalog editions
      • Planning to set up IBM Knowledge Catalog
      • Planning to implement a governance framework
      • Planning to protect data with rules
      • Planning to author data protection rules
      • Planning to curate data
      • Planning to monitor IBM Knowledge Catalog
    • Catalogs
      • Administering a catalog
        • Creating a catalog
          • Duplicate asset handling
        • Changing catalog settings
        • Deleting a catalog
        • Saving searches for catalog assets
      • Managing access to a catalog
        • Catalog collaborator roles
      • Catalog assets
        • Finding and viewing an asset in a catalog
        • Browsing asset hierarchies
        • Adding assets to a catalog
          • Adding a data file
          • Adding a connection
          • Adding data from a connection
          • Adding a connected folder asset from a connection
          • Adding COBOL copybook assets
          • Importing and exporting assets and asset metadata with a CSV file
            • CSV file format for importing and exporting asset metadata
            • CSV file format for importing and exporting asset relationships
        • Downloading data assets
        • Editing asset properties
        • Relationships in a catalog
          • Asset relationships
          • Managing relationships in a catalog
          • Exploring relationships
        • Controlling access to an asset
        • Profiling an asset
        • Managing business lineage
          • Importing lineage relationship mapping files
            • Lineage relationship mapping file format
        • Removing an asset
      • Identical data assets
    • Governance artifacts
      • Finding and viewing governance artifacts
        • Tags
      • Governance artifact properties
      • Versioning of governance artifacts
      • Managing governance artifacts
        • Import methods for governance artifacts
          • Importing artifacts by type with a CSV file
          • CSV import file format
          • Importing all governance artifacts with a ZIP file
        • Exporting governance artifacts
      • Workflows for governance artifacts
      • Categories
        • Predefined categories
        • Designing categories
        • Managing categories
        • Managing category collaborators
        • Category collaborator roles
        • Creating custom category collaborator roles
        • Importing or exporting categories
      • Policies
        • Designing policies
      • Governance rules
        • Designing governance rules
      • Data protection rules
        • Designing data protection rules
          • Filtering rows
          • Mask data
            • Advanced masking options
              • Redacting data method
              • Obfuscating data method
                • Preserve format method
                • Identifier masking method
        • Data protection rules enforcement
        • Managing data protection rules
      • Data quality SLAs
        • Designing data quality SLAs
        • Managing data quality SLAs
      • Business terms
        • Designing business terms
        • Predefined business terms
        • Managing business terms
        • Authoring business terms
        • Generating business terms
        • Creating glossaries with MCP
      • Classifications
        • Designing classifications
        • Predefined classifications
      • Data classes
        • Designing data classes
        • Adding matching methods to data classes
          • Creating custom Java class packages
        • Predefined data classes
          • Predefined data classes details
      • Reference data
        • Designing reference data sets
        • Creating reference data sets with composite keys
        • Importing files for reference data sets
        • Relationships between reference data sets
        • Predefined reference data sets
        • Storing reference data sets in an external database
      • Reporting queries
      • Knowledge Accelerators
        • Notices
        • KA for Cross Industry
        • KA for Energy and Utilities
        • KA for Financial Services
        • KA for Healthcare
        • KA for Insurance
        • Getting started
          • Artifacts available to import
            • Cross Industry
            • Energy and Utilities
            • Financial Services
            • Healthcare
            • Insurance
        • Components
          • Use of governance artifacts
            • Category areas and subcategories
            • Business terms
            • Relationships
            • Custom attributes
            • Classifications and tags
          • Business Core Vocabulary
            • Subcategories
            • Concept terms
            • Property terms
            • Relationship terms
          • Business Performance Indicators
            • Subcategories
            • Performance analysis terms
            • Measures
          • Industry Alignment Vocabularies
            • Alignment area categories
            • Alignment topic categories
            • Alignment terms
          • Business Scopes
            • Subcategories
            • Business Scopes available for separate import
              • Cross Industry
              • Energy and Utilities
              • Financial Services
              • Healthcare
              • Insurance
          • Reference data sets
          • Data classes
          • Policies and rules
          • Synonyms
        • Using and customizing
    • Data lineage
      • Preparing data for data lineage
      • Viewing data lineage
      • Managing data lineage graph
      • Configuring alias assignments
      • OpenLineage integration
        • Mapping OpenLineage events
          • Designing mappings for OpenLineage events
          • Creating mappings for OpenLineage events
          • OpenLineage mappings tutorial
      • Configuring agents for lineage metadata import
      • Exporting data lineage
        • Exporting data lineage to Collibra
          • Configuring Collibra export
        • Exporting data lineage to OpenLineage format
        • Designing lineage export
        • Creating an export asset and managing jobs
    • Setting up reporting
      • Database requirements
      • Data model
      • Managing reporting
      • Sample reporting queries
      • Reporting tables
        • Workspaces
        • Asset relationships
        • Categories
        • Governance artifacts
        • Artifact relationships
        • Data quality rules
        • Customizations
        • Workflow
        • User Profiles
        • Tags
        • Rules
        • Metadata imports and enrichments
    • Masking watsonx.data assets in watsonx.data intelligence
    • Resynchronizing assets and artifacts in the knowledge graph
  • Governing AI
    • Evaluating AI models
      • Setup options for model evaluations
        • Configuring model evaluations with automatic setup
        • Configuring model evaluations with manual setup
        • Configuring model evaluations with advanced setup
      • Terms for evaluations
      • FAQs
      • Supported machine learning engines, frameworks, and models
        • IBM Watson Machine Learning
        • Microsoft Azure ML Studio frameworks
        • Microsoft Azure ML Service frameworks
        • Amazon SageMaker frameworks
        • Custom ML frameworks
        • IBM SPSS C&DS frameworks
        • Integrating 3rd-party ML engines for model evaluations
      • Preparing to evaluate models
        • Adding deployments for evaluations
        • Providing model details
        • Configure asset deployments using JSON configuration files
      • Managing data for model evaluations
        • Managing payload data
          • Payload logging
        • Managing training data
        • Managing feedback data
        • Sending model transactions
      • Evaluating detached prompt templates in projects
      • Evaluating detached prompt templates in spaces
      • Evaluating generative AI output in multiple languages
      • Evaluation metrics
        • Accuracy
        • API latency
        • API throughput
        • Area under PR
        • Area under ROC
        • Average absolute odds difference
        • Average odds difference
        • Disparate impact
        • Error rate difference
        • False discovery rate difference
        • False negative rate difference
        • False omission rate difference
        • False positive rate difference
        • Feature drift
        • Impact score
        • Model quality drift
        • Output drift
        • Output metadata drift
        • Payload size
        • Records
        • Scoring requests
        • Statistical parity difference
        • Users
      • Configuring quality evaluations
      • Configuring fairness evaluations
        • Calculating fairness
        • Configuring for indirect bias
      • Configuring drift v2 evaluations
      • Configuring model health monitor evaluations
      • Creating custom evaluations and metrics
      • Creating custom evaluations with LLM as a judge
      • Configuring explainability
      • Setting up alerts
      • Reviewing model insights
        • Reviewing evaluation results
        • Reviewing model transactions
          • Explaining model transactions
          • Reviewing debiased transactions
      • Batch processing
        • Preparing the batch processing environment in IBM Analytics Engine
        • Preparing the batch processing environment in IBM Analytics Engine on zLinux
        • Preparing the batch processing environment on the Hadoop Ecosystem
        • Configure batch processing
      • Model risk management and model governance
        • Manage model risk
      • Metrics computation for ML models using the Python SDK
    • Governing assets in AI use cases
      • Setting up AI governance
        • Creating and managing inventories
        • Managing report templates
        • Managing attachments
      • Setting up an AI use case
      • Tracking assets in an AI use case
        • Tracking machine learning models
        • Tracking Python functions
        • Tracking external models
        • Tracking assets in a notebook
      • Viewing factsheets
      • Collaboration roles for governance
      • Adding custom facts to a factsheet
      • Coding custom facts
      • Integrating AI use cases with Governance console
  • Managing workflows
    • Managing governance artifact workflows
    • Configuring IBM Master Data Management workflows
    • Managing data quality remediation workflows
      • Working with data quality remediation tasks
    • Designing and creating custom workflows
      • Importing custom process definitions
      • Configuring custom workflows
    • Importing and exporting workflow configurations
    • Monitoring workflow tasks
    • Detecting workflow anomalies
• Troubleshooting
  • Common core services
  • Connections
  • Analytics Engine powered by Apache Spark
  • Cognos Dashboards
  • Database services
  • Data Gate
  • Data Refinery
  • DataStage
  • Data Virtualization
    • Troubleshooting virtualization issues
      • Speed up loading of Presto tables and columns in Data Virtualization
      • Virtualization fails if the table column count exceeds the limit
      • Data Virtualization automatically restarts at 1PM UTC
      • Japanese column names are not displayed correctly
      • Virtualizing a table with many columns fails
      • Error message when you try to use an unsupported file format in Cloud Object Storage
      • Speed up loading of tables when you virtualize
      • Reveal hidden tables when you virtualize
      • Listing of virtual objects is slow
      • Viewing of objects in your cart is slow
    • Troubleshooting governance issues
      • Cannot grant users access to a view
      • Access to a table is denied by policies
      • Cannot access assets in the catalog
      • Data protection rules are not enforced
      • Cannot see business term that is assigned to data asset
      • A virtualized object cannot be used in Cognos Dashboards without credentials and an appropriate role
    • Troubleshooting data source connections
      • ERRORCODE=-4214, SQLSTATE28000 error when you create a connection to Data Virtualization with API key
      • Virtualized table list needs manual updates after upgrade to 5.0 and later
      • CID and TDEF columns are not returned
      • Restoring a connection that was removed from Platform connections
      • Adding a connection to Amazon RDS for MySQL fails
      • Error when you remove a virtualized table in object storage
      • Adding a remote data source with vaulted credentials fails
      • A data source cannot be added with personal credentials from a vault
      • Cannot push down string functions with string units on Db2 remote data source
      • Snowflake connection times out
      • Cannot connect to data source
      • Cannot preview data from Salesforce
      • Cannot push down join views
      • Errors when you delete a connection
      • Remote connector does not start after restart
      • Stack overflow errors
    • Troubleshooting data caches and queries
      • Cannot see updated information in the cache dashboard
      • Caches stay in deleting state
      • Cannot delete schemas or virtual objects
      • Errors finding cache recommendations
    • Troubleshooting queries
      • Query of a virtualized table fails with Presto after upgrade
      • SQL messages
      • SQL5105N error when you run a query
      • SQL20478 error when you run a query
      • SQL0727N error when you query view results
      • Slow performance for Data Virtualization queries that are linked to pushdown issues for data sources
      • Errors when you run a query or virtualize objects
      • SQL1822N error when you run a query
      • Concurrent queries are slow or fail
      • Incorrect query results for Db2 remote data sources
      • Inaccurate query results for Netezza remote data sources
      • Inaccurate query results for Db2 z/OS remote data sources
      • Inaccurate query results for Hive data sources
      • Data type STRING in Hive tables is assigned CLOB data type
      • Performance issues in queries with subqueries
      • SUM() or AVG() function returns an error
      • Unexpected preview or query results for virtualized tables over flat files
    • Troubleshooting statistics collection
      • Collecting statistics on virtualized tables over flat files might fail
      • Column-level statistics not collected due to data type mismatch
  • Db2 Big SQL
    • Troubleshooting Db2 Big SQL messages
    • Access error after changing the role of a Db2 Big SQL user
  • Db2 Data Management Console
  • Hadoop environments
    • Troubleshooting with the Execution Engine for Apache Hadoop diagnostics tool
  • Data governance and quality
    • Solving governance artifacts import problems
    • Synchronize the data policy service (DPS) category caches
    • Processes time out before completing
    • Re-creating predefined data classes
    • Cannot submit profile jobs or preview assets
    • New columns do not appear in preview of a connected asset
    • Relationship analysis doesn't complete within the default timeout period
    • Running concurrent metadata import jobs on multiple metadata-discovery pods
    • Recovering from a failed scale-up in a deployment with IBM Manta Data Lineage
  • IBM Master Data Management
  • OpenPages
    • Service times out
    • Changing locales does not refresh all browser windows and tabs
    • ESG dashboards do not display
    • Error generating the reporting schema
  • SPSS Modeler
  • Watson OpenScale
  • Watson Machine Learning
  • Watson Studio
• Glossary