Automating data imports

Edit online

You can upload your data on a periodic basis into a object storage bucket by using CSV or JSON files. The Envizi Supply Chain Intelligence administrator creates import rules to import data at predefined intervals from your object storage bucket.

To automate your data import, complete the following steps:
  1. Review the data dictionary and prepare your CSV files. Ensure that your data files are formatted correctly in CSV format. For more information, see Sample CSV files .
    Note: The last column (Used as identifier (default csv schema)) of the data dictionary is not applicable for the JSON file import.
  2. Analyze and define master data and orders and then prepare your files for upload. For more information, see Defining master data and Defining orders
  3. If you do not already have object storage, provision your object storage bucket where you upload data. For more information, see Managing object storage.
    Note: To automate data ingestion by using an object storage bucket, you need to create a custom bucket with full access permissions. The default bucket included with the Envizi Supply Chain Intelligence has limited functions and access, and is suitable for most of the basic needs.
  4. Create an import rule by using rule builder within Envizi Supply Chain Intelligence to process your data from the defined object storage bucket. For more information, see Creating import rules.
  5. Upload your CSV or JSON files to object storage using the appropriate API for your object storage instance. For more information, see Ingesting CSV data.
  6. Ensure that the object storage upload returned 200 status code to indicate success and that the object is uploaded to your object storage instance. Once the file has been read and processed by Envizi Supply Chain Intelligence, a JSON status file with the format of YOURFILENAME_READDATE_status.json, for example testFile_2024-05-01T220611Z_status.json is created in the importStatus directory of the object storage bucket.
  7. Optional: To determine when the data uploaded to COS has been imported and processed by Envizi Supply Chain Intelligence the status of the Envizi Supply Chain Intelligence job is required. To access the status of the job, either use the Job Manager API or read and parse the JSON status file created in the importStatus directory of the object storage bucket. The JSON status file can be accessed using the object storage API.