Referencing files in Bash node (DataStage)

To reference files in the Bash node for your pipelines, you need to download the files from IBM Cloud® Object Storage into Run Bash script node using the cpdctl download asset command.

About this task

Procedure

  1. In Bash node, add the following command to reference the files in a IBM Cloud Object Storage bucket other than the project-associated bucket. You need to specify the full file path in the bucket. 
    cpdctl asset file download --path <COS_BUCKET_NAME>/<FILE_NAME> --output-file <FILE_NAME>
  2. In Bash node, add the following command to reference the files generated by the DataStage® parallel jobs. For more information on how to reference different types of data, see Sharing DataStage artifacts with all IBM Cloud Object Storage containers. 
    cpdctl asset file download --path DataStage/files/<FILE_NAME> --output-file <FILE_NAME>
  3. After downloading the file, you can use the file name referenced in the --output-file to modify it. Reference the file in the same Bash node that you downloaded the files in.
    Example: 
    cat FILE_NAME