Generic S3 connection

To access your data from a storage service that is compatible with the Amazon S3 API, create a connection asset for it.

Create a Generic S3 connection

To create the connection asset, you need these connection details:

  • Endpoint URL: The endpoint URL to access to S3
  • Bucket(optional): The name of the bucket that contains the files
  • Region (optional): S3 region. Specify a region that matches the regional endpoint.
  • Access key: The access key (username) that authorizes access to S3
  • Secret key: The password associated with the Access key ID that authorizes access to S3
  • The SSL certificate of the trusted host. The certificate is required when the host certificate is not signed by a known certificate authority.
  • Disable chunked encoding: Select if the storage does not support chunked encoding.
  • Enable global bucket access: Consult the documentation for your S3 data source for whether to select this property.
  • Enable path style access: Consult the documentation for your S3 data source for whether to select this property.

For Credentials and Certificates, you can use secrets if a vault is configured for the platform and the service supports vaults. For information, see Using secrets from vaults in connections.

Choose the method for creating a connection based on where you are in the platform

In a project
Click Assets > New asset > Data access tools > Connection. See Adding a connection to a project.

In a catalog
Click Add to catalog > Connection. See Adding a connection asset to a catalog.

In a deployment space
Click Add to space > Connection. See Adding connections to a deployment space.

In the Platform assets catalog
Click New connection. See Adding platform connections.

Next step: Add data assets from the connection

Where you can use this connection

You can use the Generic S3 connection in the following workspaces and tools:

Projects

  • AutoAI (Watson Machine Learning)
  • Data Refinery (Watson Studio or Watson Knowledge Catalog)
  • DataStage (DataStage service). See Connecting to a data source in DataStage.
  • Metadata enrichment (Watson Knowledge Catalog)
  • Metadata import (Watson Knowledge Catalog)
  • Notebooks (Watson Studio). Click Read data on the Code snippets pane to get the connection credentials and load the data into a data structure. See Load data from data source connections.

Catalogs

  • Platform assets catalog

  • Other catalogs (Watson Knowledge Catalog)

    Note:

    Preview, profile, and masking do not work for this connection in Watson Knowledge Catalog.

Watson Query service You can connect to this data source from Watson Query.

Federal Information Processing Standards (FIPS) compliance

The Generic S3 connection cannot be created in a FIPS environment.

Generic S3 connection setup

For setup information, consult the documentation of the S3-compatible data source that you are connecting to.

Supported file types

The Generic S3 connection supports these file types:  Avro, CSV, delimited text, Excel, JSON, ORC, Parquet, SAS, SAV, SHP, and XML.

Related connection: Amazon S3 connection

Parent topic: Supported connections