Changing the limit of allowed concurrent jobs for batch deployments

The default limit for concurrent deployment jobs is two. Any deployment job request for a batch deployment that already has two running jobs is placed in a queue for execution later. You can change this limit if required.

About this task

Prerequisites:You must be a Kubernetes admin to change the default limit of concurrent deployment job runs.

To change the default limit of concurrent deployment job runs:

Procedure

  1. Run this command to put the Watson Machine Learning operator in maintenance mode: oc patch wmlbase wml-cr --type merge --patch '{"spec": {"ignoreForMaintenance": true}}'
  2. Update the wmlruntimemanager configmap value:
    1. Run the following command: oc project <control_plane>.
    1. Take backup of the existing wmlruntimemanager configmap:oc get cm wmlruntimemanager -o yaml > wmlruntimemanager_org.yaml
    2. Update the number of parallel jobs from 2 to a larger value:oc get cm wmlruntimemanager -o yaml | sed -e 's|private = 2|private = <new limit>|' > wmlruntimemanager_new.yaml
    3. Apply the updated configmap: oc apply -f wmlruntimemanager_new.yaml
  3. Restart the wml-deployment-manager pod and then wait for wml-deployment-manager to be operational again: oc delete pod -l app=wml-deployment-manager
  4. Run this command to put the operator back in the normal operation mode: oc patch wmlbase wml-cr --type merge --patch '{"spec": {"ignoreForMaintenance": false}}'