Data source definitions overview
Use data source definitions to monitor where your data is stored and uniquely identify it.
- Required service
- IBM watsonx.data intelligence
A data source definition is an asset that functions as a unique stable identifier for the location of a data source such as a relational database. A data source definition uses endpoints to identify a single data source instance. For most data source types, an endpoint consists of the hostname or IP address, the port number, and the database name or instance identifier. In a multinode environment, a single data source definition is defined by a list of endpoints.
When multiple connections reference the same data source, a data source definition helps the platform recognize those connections as referring to a single source, even when they use different endpoints or exist in different workspaces.
Architecture overview
A data source definition sits between physical data sources and the connections that you create to access them.
The following diagram shows how multiple connections and endpoints can be associated with a single data source definition
As shown in the diagram, a single data source can be accessed through multiple connections that use different endpoints. A data source definition groups these endpoints under a single identifier, that helps you consistently identify the same data source across projects, catalogs, and deployment spaces. The platform automatically associates matching connections with existing data source definitions, ensuring consistent source identification throughout the platform.
Capabilities
Data source definitions help you consistently identify and manage data sources across the platform, regardless of how users connect to them.
With data source definitions, you can:
- Identify multinode data sources by associating multiple endpoints with a single data source definition, so that assets from the same underlying source are treated consistently.
- Group related connections that point to the same data source, even when those connections are created in different projects, catalogs, or deployment spaces.
- Provide consistent source identification across platform workflows, helping the platform recognize identical data assets and maintain consistent asset identity across workspaces.
- With the watsonx.data intelligence entitlement, you can prepare lineage metadata imports by providing a consistent identifier for imported data sources. Data source definitions are required when you import lineage metadata.
- With the watsonx.data intelligence entitlement, you can, explore relationships between assets across connections and workspaces by using a shared data source identity. For more information, see Exploring relationships.
- With the watsonx.data intelligence entitlement, apply data protection solutions consistently for supported data sources. For more information, see Protection solutions for data source definitions.
Workflow
Data source definitions are typically used as part of the following process:
- Create a data source definition for a data source that you want to identify and manage consistently across the platform. For more information, see Creating a data source definition.
- Define or manage endpoints that identify the data source. A data source definition can include one or more endpoints that represent the same underlying data source. The platform automatically associates matching connections and connected data assets with the data source definition based on endpoint information. For more information, see Adding endpoints to a data source definition.
- Manage the data source definition throughout its lifecycle. Update endpoints, manage access, configure protection solutions, and control how connections interact with the data source.
- Use the data source definition in platform workflows that rely on consistent source identification.