Configuring endpoints for file ingestion rules

To activate data ingestion, you must select a specific storage provider to configure. Then, you can transfer data to Sterling Intelligent Promising to ingest the data from the designated storage buckets.

1. Log in to Order Hub.
2. From the menu, click Endpoints.
3. Select the Ingest tab.
   The Endpoints page opens with the following information:

   Table 1. Endpoints page

   Field name	Description
   Entity	The event type for which you configure an endpoint ingest rule. Click any Entity link to view the endpoint details. An Entity link is available only when you configure a rule. To sort the entities, hover over the Entity column and select the Sort icon.
   Description	A description of the event type.
   File type	The type of file that is supported by the endpoint, for example, a CSV file.
   Status	The status of the endpoint and whether it is configured. An entity stays in an unconfigured state until you activate or configure it. Valid options are Configured or Unconfigured.
   Actions	Edit an endpoint that is configured or configure an endpoint. Delete an endpoint that is configured.

   You can select any column heading to sort the information.
4. Complete one of the following steps:
   • To configure an endpoint, click the Edit icon for an endpoint that is in the Unconfigured status.
   • Select an entity for which you want to configure a file ingest rule.
5. On the Endpoint details page, view the following endpoint information that is provided:

   Endpoint ID

   The ID that is associated with the endpoint to configure.

   Entity

   The data type for which you configure a file ingestion rule.

   Description

   A description of the file ingestion rule.

   File type

   The type of file that is supported by the endpoint, for example, a CSV file.

   Status

   The status of the endpoint and whether it is active.

   To activate the endpoint, slide the Status toggle to Active.

   Maximum number of keys (optional)

   Enter a positive integer to forward files separately instead of forwarding the whole prefix location. By default, the 0 value indicates that all the files available in the prefix are forwarded together in one iteration. The tags that were available in the original files are not kept as part of this process.

   This step is necessary only if there is a requirement to maintain the tags set by the file uploader on the files at the forwarded location.

6. In the cloud storage provider section, to select a cloud storage provider to configure, select one of the following options:
   • Amazon S3
   • Microsoft Azure Blob Storage
   • Google Cloud Storage
   • Webhook
   • Amazon S3/ IBM Cloud Object Storage

     Table 2. Amazon S3/IBM Cloud Object Storage hash-based message authentication (HMAC) credentials

     Access credentials and connections	Description
     HMAC Access key	The access key that is used to access the bucket.
     HMAC Secret key	The secret key that is used to access the bucket.
     Region	The region that is associated with the S3 service instance.
     Endpoint URL	The endpoint URL of the S3 cloud storage service instance.

     Configure the bucket names and prefixes for the allocation file processing workflows.

     Table 3. Import location

     Import location	Description
     Bucket name	The name of the bucket.
     Prefix	The prefix value of the storage location in the bucket for import ingest data.

   • Microsoft Azure Blob Storage

     Table 4. Microsoft Azure Blob Storage details

     Authentication	Description
     Authentication type	Select one of the following options and provide the details that are required: Shared key (access keys) Account Name: The name of the Azure Storage account that is used to access the Azure blob container. Account Key: The access key for the Azure Storage account that is used to access the Azure blob container. Azure SAS Credential Signature: The shared access signature that is used to access the Azure blob container.
     Endpoint URL	The endpoint URL of the Microsoft Azure Blob storage instance.

     Configure the bucket names and prefixes for the allocation file processing workflows.

     Table 5. Microsoft Azure Blob import location details

     Import location	Description
     Container name	The Azure blob container name.
     Blob prefix	The prefix value of the storage location in the bucket for import ingest data.

   • Google Cloud Storage

     Table 6. Google Cloud Storage configuration details

     Google cloud credentials	Description
     Service account key (JSON)	The private key that is used to access the bucket.
     Endpoint URL	The endpoint URL of the google cloud storage instance.

     Configure bucket names and prefixes for allocation file processing workflows.

     Table 7. Google Cloud Storage import location details

     import location	Description
     Bucket name	The name of the bucket.
     Prefix	The prefix value of the storage location in the bucket for import ingest data.

7. Table 8. Validation failures

   Failed records publishing	Description
   Publish rule ID	The ID that is associated with the publish rule.
   Validation failure prefix	The prefix for the validation failure.
   Schema ID	The ID for the schema.

8. Click Save.
9. To view the endpoint details for an existing configuration, select the Ingest tab and click any of the entity links that have a Configured status.
   The Endpoints page displays the configuration details.
10. Edit or delete a configuration.
    • To edit the configuration information, click the Edit icon. Edit the Endpoint and Region bucket details. Then, edit the Secret key and Access key credentials and click Save.

      You can’t edit the bucket name or the prefix.

    • To delete a configuration, on the Endpoints page, click the Delete icon. In the Confirm delete dialog, click Delete.