Choosing compute resources for running tools in projects
You use compute resources in projects when you run jobs and most tools. Depending on the tool, you might have a choice of compute resources for the runtime for the tool.
Compute resources are known as either environment templates or hardware and software specifications. In general, compute resources with larger hardware configurations incur larger usage costs. Many tools in projects use the watsonx.data intelligence service for compute resources, but some tools use other services. Each service tracks and bills compute usage separately.
These tools have multiple choices for configuring runtimes that you can choose from:
This tool requires that you create an environment to configure the runtime:
These tools have one runtime configuration that is assigned automatically:
- Metadata import
- Metadata enrichment
- Advanced profiling
- Primary key or in-depth relationship analysis
- Data quality rules
- Profiling
Metadata import
The amount of CUH per hour that metadata import jobs consume depends on the number of assets that are imported. For each 50,000 assets, 1 CUH is consumed.
Metadata enrichment
The amount of CUH per hour that metadata enrichment jobs consume depends on the enrichment objectives that you select.
| Metadata enrichment objectives | Capacity units per hour (CUH) |
|---|---|
| Profile data | 4 |
| Expand metadata | 1.5 per 1,000 asset with 100 columns each |
| Assign terms and classifications | 0.75 per 1,000 assets Use of gen AI for term assignment will incur additional charges. |
| Set relationships | 4 |
| Identify data quality checks | 0.06 per 1,000 assets with 100 columns each |
| Run data quality analysis | 4 |
You can track compute usage for the metadata enrichment objectives on the Resource usage page of the Manage tab of the project.
CUH consumption for profiling and running data quality analysis is aggregated and shown under Profiling. CUH consumption for other selected objectives in metadata enrichment is aggregated and shown under Metadata enrichment.
Advanced profiling
This type of analysis requires the DataStage service. Advanced profiling runs as a DataStage flow with the Default DataStage PX S environment, which consumes 1 CUH per hour, with a minimum of 1 minute of CUH.
The runtime for advanced profiling appears as a DataStage flow on the Resource usage page of the Manage tab of the project.
Primary key or in-depth relationship analysis
These types of analysis require the DataStage service. A primary key or in-depth relationship analysis runs as a DataStage flow with the Default DataStage PX S environment, which consumes 1 CUH per hour, with a minimum of 1 minute of CUH.
The runtime for primary key or in-depth relationship analyses appears as a DataStage flow on the Resource usage page of the Manage tab of the project.
Data quality rules
Data quality rules require the DataStage service. A data quality rule job runs as a DataStage flow with the Default DataStage PX S environment, which consumes 1 CUH per hour, with a minimum of 1 minute of CUH.
The runtime for data quality rules appears as a DataStage flow on the Resource usage page of the Manage tab of the project.
Profiling data assets
Profiling a data asset in a project or a catalog consumes 4 CUH per hour.
You can track compute usage for profiling outside of metadata enrichment on the Resource usage page of the Manage tab of the project.
Changing the environment runtime for a tool
For tools that support multiple runtime environments, you can select a larger environment runtime if you notice that processing is slow.
To change an environment runtime:
- Save any data from your current session before switching to another environment.
- Stop the active runtime under Tool runtimes on the Environments page on the Manage tab of your project.
- Restart the tool and select another environment with the compute power and memory capacity that better meets your requirements.