Integrating with Data Product Hub

Data assets and data products

Data assets can be SQL queries, SQL tables, ML models, or BI dashboards. A data product is a collection of assets packaged together for a specific business use case.

Data producers and data consumers

Data producers are responsible for packaging the data assets into data products, defining the data contract, and making the product accessible to consumers. Data consumers are the users who subscribe to these data products. Data consumers can be both technical users and business users.

Features of integration

Accessing datasource

DPH connects with watsonx.data through a Presto connector to create a data asset using the metadata information, which is packaged into a data product. The data is packaged securely without actual data movement.

To access data from watsonx.data:

1. Create a connection asset to access the data. See Presto connection.
2. Create a data product to package the asset. See Creating a data product directly from a source.
3. Publish the data product. See Publishing your data product.

Parameterized querying

You can package the metadata information of SQL queries in watsonx.data and deliver the results in formats such as CSV or Parquet. Data producers can create a data product with a customizable query. See Creating a data product from a customizable query.

Data delivery methods

Flight service

The Flight service provides read access to various data sources in watsonx.data. This delivery method is only used by the technical data consumer (for example, data scientists). See Flight Service.

Data extract

Business consumers use data refinery service to create a CSV data extract file and download it. See Data Extract delivery method.

Deliver as a table in watsonx.data

This delivery method allows consumers to access their data product as a table in watsonx.data. See Deliver as a table in watsonx.data delivery method.

Prerequisites — you must have the following accesses in watsonx.data:

1. Access to a Spark engine connected to the target catalog.
2. Access to a Presto engine (required to make the connection).
3. Access to a target Iceberg catalog with role to the full catalog or an access policy.
4. Write access to storage used for the target catalog.
5. Access to Cloud Object Storage added as a component in the watsonx.data instance. See IBM Cloud Object Storage.

For managing access in watsonx.data, see Managing data policy.

Access in watsonx.data

This delivery method allows consumers to access their data product directly in watsonx.data. See Data Product Hub documentation.