Changing values in the dynamic configuration file in DataStage

Edit the default dynamic configuration template file to use your own pool, resource, and scratch disk values.

If the APT_CONFIG_FILE environment variable is not set, the ds-runtime environment uses a template file to generate a dynamic configuration file. The template configuration file is named dynamic_config.apt.template and is located at /px-storage/config.

By default, the template configuration file contains the following entries:

{
node "conductor"
{
fastname "$conductor"
pools "conductor"
resource disk "/px-storage/pds_files/node1" {pool "" "export" "node1"}
resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
}
node "compute"
{
fastname "$pod"
pools ""
resource disk "/px-storage/pds_files/node#" {pool "" "export" "node#"}
resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
}
}

Edit this file to change pools, resource disk, and resource scratchdisk values for both the conductor and compute nodes.

The ds-runtime environment checks for the presence of the file /px-storage/config/aptconfig.template and if it is present, it uses this template to generate the configuration file with additional partition information that it receives from the environment. Wherever the px-runtime environment finds a "node#" in the template configuration file, the px-runtime environment replaces it with an increasing value like node2, node3, or node4, based on the partition information.

See the following example generated configuration file for four partitions:

{
node "node1"
{
fastname "10.254.12.172"
pools "conductor"
resource disk "/px-storage/pds_files/node1" {pool "" "export" "node1"}
resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
}
node "node2"
{
fastname "ds-px-default-ibm-datastage-px-compute-0.ds-px-default-ibm-datastage-px-compute"
pools ""
resource disk "/px-storage/pds_files/node2" {pool "" "export" "node2"}
resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
}
node "node3"
{
fastname "ds-px-default-ibm-datastage-px-compute-1.ds-px-default-ibm-datastage-px-compute"
pools ""
resource disk "/px-storage/pds_files/node3" {pool "" "export" "node3"}
resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
}
node "node4"
{
fastname "ds-px-default-ibm-datastage-px-compute-0.ds-px-default-ibm-datastage-px-compute"
pools ""
resource disk "/px-storage/pds_files/node4" {pool "" "export" "node4"}
resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
}
node "node5"
{
fastname "ds-px-default-ibm-datastage-px-compute-1.ds-px-default-ibm-datastage-px-compute"
pools ""
resource disk "/px-storage/pds_files/node5" {pool "" "export" "node5"}
resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
}
}

Creating and editing a new template

You can create a new template for your project and edit it to match your requirements.

Connect with the infra node.

[root@api.is-cpd-a.cp.fyre.ibm.com ~]# oc get pods | grep px
ds-px-default-ibm-datastage-px-compute-0 1/1 Running 0 6d21h
ds-px-default-ibm-datastage-px-compute-1 1/1 Running 0 6d21h
ds-px-default-ibm-datastage-px-runtime-c46dfd64d-87zv7 1/1 Running 0 6d21h

Enter the

px
pod

$oc rsh ds-px-default-ibm-datastage-px-runtime-c46dfd64d-87zv7

Copy the px-storage/config/dynamic_config.apt.template into your project, for example:

sh-5.1$ cp /px-storage/config/dynamic_config.apt.template /ds-storage/PXRuntime/Projects/a44d10c3-cbc2-48ae-9c65-09ddb0c8b4b5/

Edit the dynamic_config.apt.template file with nano editor.

sh-5.1$ nano /ds-storage/PXRuntime/Projects/a44d10c3-cbc2-48ae-9c65-09ddb0c8b4b5/dynamic_config.apt.template

If you want to change the scratch disc location, change the scratchdisk setting in the project specific dynamic_config.apt.template.

Initially copied file:

{
  node "conductor"
  {
   fastname "$conductor"
   pools "conductor"
   resource disk "/px-storage/pds_files/node1" {pool "" "export" "node1"}
   resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
  }
  node "compute"
  {
   fastname "$pod"
   pools ""
   resource disk "/px-storage/pds_files/node#" {pool "" "export" "node#"}
   resource scratchdisk "/opt/ibm/PXService/Server/scratch" {pool ""}
  }
}

Changed file:

{
  node "conductor"
  {
   fastname "$conductor"
   pools "conductor"
   resource disk "/px-storage/pds_files/node1" {pool "" "export" "node1"}
   resource scratchdisk "/new/scratch/dir" {pool ""}
  }
  node "compute"
  {
   fastname "$pod"
   pools ""
   resource disk "/px-storage/pds_files/node#" {pool "" "export" "node#"}
   resource scratchdisk "/new/scratch/dir" {pool ""}
  }
}

In this case, once the changes are saved, jobs that run withing the project will pick up the new template.