Platform architecture

The watsonx.data experience is part of the IBM watsonx platform on IBM Software Hub clusters. The IBM watsonx platform has multiple integrated experiences that share services and workspaces. The experiences that you can access depend on which services are installed on your IBM Software Hub cluster. An experience provides focused access to the tools for specific tasks.

The IBM watsonx platform includes these integrated experiences:

  • watsonx, which contains the watsonx.ai Studio, Watson Machine Learning, and IBM watsonx.governance services for building and governing AI solutions.
  • Data Fabric, which contains the watsonx.data intelligence and watsonx.data integration services for transforming and sharing high-quality, trusted data products.
  • watsonx.data, which contains the watsonx.data Premium, watsonx.data intelligence, watsonx.data integration, watsonx.ai, and related services for preparing unstructured data for AI.
  • Cloud Pak for Data, which contains many of the same services as the other experiences but without generative AI or unstructured data processing capabilities.
  • Data Product Hub, which contains the Data Product Hub service for sharing data products without the rest of the Data Fabric capabilities.

Projects are shared between the experiences so that users with different tasks can work together. You can switch between experiences that you have permission to access to use different tools. Users who are collaborating in the same project can work in different experiences. For example, suppose a data engineer and an AI engineer are collaborators in the same project. The data engineer ingests data sources into the lakehouse in the watsonx.data experience and integrates the data in the Data Fabric experience to prepare a data asset. The AI engineer, who is working in the watsonx experience, uses the data asset to train a model. See Switching between experiences.

The following illustration shows the architecture of the integrated experiences on the IBM Software Hub platform, the services and capabilities for each experience, and the shared functionality that provides an integrated user experience.

alt=""

Shared functionality

The platform includes the following functionality that is shared between services and experiences for secure and scalable collaboration:

  • Connectivity
  • Administration
  • Storage
  • Workspaces

Connectivity

You can create connections to remote data sources and import connected data. You can configure connections with personal or shared credentials. For a list of supported connectors, see Connectors.

You can share connections with others across the platform in the Platform assets catalog.

Administration

Your cluster administrators manage the experience through the IBM Software Hub platform. Administrators can perform the following types of tasks:

  • Installing, upgrading, or migrating the software
  • Backing up or restoring the software
  • Monitoring the platform
  • Securing the environment
  • Auditing events
  • Forwarding alerts, notifications, and announcements
  • Setting up services
  • Managing resources
  • Managing users

See Administering IBM Software Hub in the IBM Software Hub documentation.

Storage

The IBM Software Hub platform requires a persistent storage solution that is accessible to your Red Hat OpenShift cluster. All the assets that you create with watsonx.ai and watsonx.governance are stored in that persistent storage solution.

See Storage requirements in the IBM Software Hub documentation.

Workspaces

The platform is organized as a set of collaborative workspaces where you can work with your team or organization. Each workspace has a set of members with roles that provide permissions to perform actions.

Most users work with assets, which are items that are created or added to workspaces by users. Assets can represent data, flows, experiments, or other types of code or information. See Asset types and properties.

You can work in these types of platform workspaces in the watsonx.data experience:

  • Projects
  • Catalogs
  • Platform connections
  • Categories
  • Data lineage

You can search for assets across all workspaces that you belong to.

Projects

Projects are where your data science and model builder teams work with data to create assets, such as, saved prompts, notebooks, models, or pipelines.

Your projects are shared across the integrated experiences. However, you can view and run only those assets that are valid in the current experience. For example, in the watsonx experience, you can't enrich the metadata of a data asset.

The following image shows what the Overview page of a project might look like.

Overview page for a project

Platform connections

Platform connections is a view of the Platform assets catalog that lists connection assets. You can access platform connections in any project.

The Platform assets catalog is shared across all integrated experiences. However, in each experience you can view and access only those assets, items, or views that are applicable to that experience.

The following image shows what the Connections page of the Platform connections might look like.

Connections page for Platform connections

Catalogs

Catalogs are where your organization finds and stores high-quality, trusted data, and other assets, such as data quality assets. You can find data assets in a catalog and move them into a project to work with the data. Or you can curate data in projects and publish the high-quality data assets to a catalog for others to use.

Catalogs are available in the watsonx.data and Data Fabric experiences.

The following image shows what the Assets page of a catalog might look like.

A catalog contains a view of assets.

Categories

Categories are where your governance team creates and manages governance artifacts that enrich data assets in catalogs.

Categories are available in the watsonx.data and Data Fabric experiences.

The following image shows what a category might look like.

A category contains governance artifacts.

Data lineage

Data lineage is a visualization of how your data moves through the organization. It shows where your data originates, how it transforms and where it continues to flow. Review data lineage to verify data accuracy, find compliance issues, and determine the impact of data.

Data lineage is available in the watsonx.data and Data Fabric experiences.

A data lineage graph shows the flow of data.

Data product sharing

In Data Product Hub you can discover, share, create and use curated data products organization-wide. On Data Product Hub, data producers can publish curated data products to share with data consumers in their community. Data consumers can easily access data products for their business needs.

Data products can contain one or more data or data-related assets. They are curated, packaged, and distributed to be easily accessible and reusable. Unlike data assets in governance catalogs, data products are managed as products with lifecycle management, wide distribution, and multiple purposes to provide maximum business value.

Data Product Hub is available in the watsonx.data and Data Fabric experiences.