Accessing Azure storage with the AzCopy CLI

Install the AzCopy command line tool to access Azure storage in DataStage.

Before you begin

An Azure storage account is required to complete this task.

Procedure

  1. Install the Azure CLI tool az. Follow the instructions in How to install the Azure CLI, or if you are a root user on an RHEL sandbox, run the following command. 
    curl -L https://aka.ms/InstallAzureCli | bash
  2. Install AzCopy.
    1. Download the tool from Download AzCopy.
    2. Copy azcopy_linux_amd64_x.x.x.tar.gz to /ds-storage/ directory on ds-px-default-ibm-datastage-px-runtime-xxx-xxx pod.
    3. Extract the tar file azcopy_linux_amd64_x.x.x.tar.gz. After that, you can see the "azcopy" and "NOTICE.txt" files in /ds-storage/azcopy_linux_amd64_x.x.x.
    4. Add directory path to PATH variable by using following command: 
      export PATH=/ds-storage/azcopy_linux_amd64_x.x.x:$PATH
  3. To access Azure storage by using azcopy, you need a Shared Access Signature (SAS token).
  4. Set the required environment variables: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY, AZURE_CONTAINER_NAME, AZURE_SAS_TOKEN before you access Azure storage account.
  5. Find scripts in /opt/ibm/PXService/Azure on ds-px-default-ibm-datastage-px-runtime-xxx-xxx pod for performing operations on the Azure storage:
    • Use "list_objects.sh" to list files in the Azure storage.
    • Use "azure_upload.sh" to export a local file to the Azure storage.
    • Use "azure_download.sh" to download an object from the Azure storage to local disk.
    • Use "azure_delete.sh" to delete an object from the Azure storage.