Runtime storage instance isolation

In DataStage, you can create different runtime instances with a context specific for a particular business unit.

DataStage Runtime instance-level isolation

You can achieve this level of isolation by creating different runtime instances, where each instance mounts its own NFS export. For more information, see Setting up an NFS mount in DataStage.

In this case, all files are owned by the same user ID. If the user uses NFS client to directly access the storage system, all NFS exports have the same user ID, therefore there is isolation from Cloud Pak for Data but no logical isolation at the storage level. However, if the direct storage access outside of Cloud Pak for Data is restricted, then this level of isolation is sufficient.

DataStage Runtime instance-level isolation with Supplemental Group

In this case, each runtime instance mounts NFS file system with a supplemental group ID that is specific to that business unit, therefore each runtime instance has the logical isolation on both Cloud Pak for Data and storage level.

To achieve this level of isolation you can create different runtime instances where each instance mounts its own NFS, and as a part of NFS mount, specify the supplemental group id. For more information, see Setting up an NFS mount in DataStage.

DataStage Runtime instances number limitations

Out of box DataStage operator has enough resources to monitor up to 6DataStage runtime instances.

If you want to create more instances, you need to allocate more resources to DataStage operator. The following sample code assumes that user wants to create 40 DataStage runtime instances:

# Retrieve the csv name

operator_ns=cpd-operators

csv_name=`oc -n $operator_ns get csv | grep datastage-operator | awk '{print $1}'`

# Update the csv to set --max-concurrent-reconciles=20 and the resource limits to 4 core and 8gb

oc -n $operator_ns patch csv $csv_name --type='json' -p='[{"op": "replace", "path": "/spec/install/spec/deployments/0/spec/template/spec/containers/0/args", "value": ["--zap-log-level", "info", "--max-concurrent-reconciles", "40"]},{"op":"replace", "path":"/spec/install/spec/deployments/0/spec/template/spec/containers/0/resources/limits", "value": {"cpu": "4", "memory":"8Gi"}}]'