Sharing storage volumes between Orchestration Pipelines and DataStage

You can pass data files between DataStage® and Orchestration Pipelines by using a shared storage volume.

About this task

Establish access to a storage volume by creating a connection to a persistent volume claim (PVC) and adding it to the PX runtime custom resource.

Procedure

  1. Create a storage volume of either the Existing PVC or New PVC volume type. Existing PVCs must have the read-write-many (RWX) access mode to be mounted by DataStage and Pipelines. See Managing storage volumes for more details.
  2. Create a storage volume connection for the PVC to make it visible to Pipelines in a project. See Storage volume connection. for more details.
  3. Update the PX runtime instance to mount the PVC by editing the PX runtime CR to add the PVC name and mount path of the storage volume.
    # list PXRuntime CRs
    oc -n <cp4d-project> get pxruntime 
    
    # edit PXRuntime CR
    oc -n <cp4d-project> edit pxruntime <PXRuntime CR name>
    
    # Add the new PVC under additional_storage. The sample below has two PVCs mounted to /mnts/data1 and /mnts/data2.
    
    spec:
      additional_storage:                      # mount additional persistent volumes
      - mount_path: /mnts/data1                # the path to mount the persistent volume
        pvc_name: data1-pvc                    # the name of the associated persistent volume claim
      - mount_path: /mnts/data2
        pvc_name: data2-pvc
    
    DataStage flows reference files from the storage volume by the mount path that is specified in the CR. Update the PX runtime CR with the preexisting mount path for the storage volume to keep the mount path consistent between Pipelines and DataStage. All storage volume mount paths start with /mnts.