Importing unstructured data

Required permissions

To import unstructured data, you must have the Admin or the Editor role in the project where you import data.

To import unstructured lineage metadata, you must have the Lineage administrator role.

Required connections

When you connect to an unstructured data source, the following connections must be created in the project where you import data:

• A connection to one of the supported data sources from which you want to import unstructured data
• A connection to IBM watsonx.data Presto to connect to the Iceberg table to store the imported metadata.

You can use the same IBM watsonx.data Presto connection for many unstructured data imports but this connection must be added to each project where the imports are run.

Required project settings

Define the storage for base document set. In the project settings, go to Unstructured Data Integration > Document set storage, select a Presto connection and the schema where you want to store the imported content.

Required configuration for lineage import

For lineage metadata import, data source definitions are required. If you have required permissions and roles, data source definitions are created automatically when you run unstructured data import, or unstructured data enrichment.

The following roles and permissions are required:

• Create data source definitions or Manage data source definitions permission. These permissions are not assigned to any predefined role, you must create a custom role. For more information, see Roles and asset privacy settings for data source definitions.

If you want to use your own names for data source definitions, create them manually for the following connections:

• The data source from which you import unstructured data
• The IBM watsonx.data Presto connection

For more information about data source definitions, see Creating a data source definition.