Adding hyperparameter search plugins

Edit online

Add or remove hyperparameter search algorithms as plugins to or from WML Accelerator. Hyperparameter search algorithm plugins can also be queried by type of name.

To add and manage a hyperparameter search algorithm as a plugin in WML Accelerator, see: For an example on how to create your own hyperparameter search plugin, see Develop a hyperparameter search plugin.

Installing a search plugin

There are two methods to install hyperparameter search plugins, local and upload installation. If the script files are large, it is recommenced that the files are copied to the server before you use the local installation method.

Local install
Use this method if the plugin scripts are located on the dlpd server, or in a shared folder that the dlpd service can access.

Use the following rest command to submit a local installation request:

> curl -k -X POST \
--header 'Authorization: Bearer <jwttoken>' \
-H 'Accept: application/json' \
-H 'Content-Type: multipart/form-data' \
-F data='{"name": "${plugin_algorithm_name}", "path": "${plugin_path_in_server}", "condaHome": "${conda_home_path}", "condaEnv": "${conda_env_name}", "remoteExec": ${enable_remote_execution}, "logLevel": "${log_level}"}’ \ 
'https://${cpd-name}/platform/rest/deeplearning/v1/hypersearch/algorithm/install'
Parameters are as follows:
Table 1. Local install parameters
Parameter name Type Description Required
plugin_algorithm_name string Name of the user plugin algorithm.

The plugin algorithm name should start with a letter, and contain only alphanumeric characters and dashes, up to 100 characters.

The plugin algorithm name cannot be same with the built-in algorithm names, such as Random, TPE, Bayesian, Hyperband and ExperimentGridSearch.

 true
plugin_path_in_server string

Full path of the user plugin algorithm.

The path should exist in the dlpd service server host. Ensure that the path and can be accessed by the CLUSTERADMIN operating system user and has the read and execution permissions, as well as all the parent path.

The path must contain a optimizer.py file which implements the BasePluginOptimizer class.

 true
conda_home_path string

Full path of the user conda home used to run the plugin scripts.

The path must exist on the management hosts and must be accessed by the CLUSTERADMIN operating system user.

If not specified, DLI_CONDA_HOME is used.

If specified, the conda_env_name parameter must be specified.

 false
conda_env_name string

Name of the user conda environment where the plugin scripts are run.

If conda_home_path is specified, the conda environment that is specified in conda_home_path is used to run the plugin scripts. Otherwise, if not specified, DLI_CONDA_HOME is used.

 false
enable_remote_execution boolean
Specify whether to enable the execution mode of the plugin algorithm, valid values:
  • true: enable remote execution mode. WML Accelerator HPO starts a Spark job to run the plugin scripts remotely on a compute host.
  • false: disable remote execution mode. The plugin scripts run on the dlpd service host (management host).

The default value is false. Enabling remote execution mode can save management host resources, but can impact the time it takes to start Spark applications.

 false
log_level string Specify the log level for the plugin algorithm, such as: DEBUG, INFO, WARNING, ERROR, CRITICAL.

The default value is INFO.

 false
cpd-route string To obtain value:
oc get route -n ${cpd-namespace}
 true
Note:
  • Only Python 3 is supported for plugin tuning.
Upload install
Use this method if you want to upload the plugin scripts from a client host.
Use the following rest command to submit an upload installation request:
> curl -k -X POST \
--header 'Authorization: Bearer <jwttoken>'
-H 'Accept: application/json' \
-H 'Content-Type: multipart/form-data' \
-F file=@${plugin_tar_file_path} \
-F data='{"name": "${plugin_algorithm_name}", "condaHome": "${conda_home_path}", "condaEnv": "${conda_env_name}", "remoteExec": ${enable_remote_execution}, "logLevel": "${log_level}"}' \ 
'https://${cpd-route}/platform/rest/deeplearning/v1/hypersearch/algorithm/install'
To obtain cpd-route:
oc get route -n ${cpd-namespace}
Parameters are as follows:
Table 2. Upload install parameters
Parameter name Type Description Required
plugin_algorithm_name string

Name of the user plugin.

The plugin name should start with a letter, and only contains letters, numbers and dashes only, up to 100 characters

 true
plugin_tar_file_path string The full path of the user plugin algorithm tar file. The tar file should end with suffix “.tar”. true
conda_home_path string

The full path of the user conda home used to run the plugin scripts.

The path should exist in management hosts and can be access by the CLUSTERADMIN operating system user.

If not specified, will use the default DLI_CONDA_HOME. If specified, the conda_env_name should also be specified.

 false
conda_env_name string

The user conda env name used to run the plugin scripts

If conda_home_path is specified, it will use the conda environment specified in conda_home_path to run the plugin scripts. Otherwise, DLI_CONDA_HOME is used.

 false
enable_remote_execution boolean
Specify whether to enable the execution mode of the plugin algorithm, valid values:
  • true: enable remote execution mode. WML Accelerator HPO starts a Spark job to run the plugin scripts remotely on a compute host.
  • false: disable remote execution mode. The plugin scripts will run in the dlpd service host (management host).

The default value is false. Enable remote execution mode can save management host resources, but it can impact the time it takes to start Spark applications.

false
log_level string Specify the log level for the plugin algorithm, such as: DEBUG, INFO, WARNING, ERROR, CRITICAL.

The default value is INFO.

 false
cpd-route string The route is determined by: 
oc get route -n ${cpd-namespace}
 true
Note:
  • Only Python 3 is supported for the plugin tuning.
  • Make sure dependent python modules are installed in the conda environment:
    > source ${conda_home_path}/etc/profile.d/conda.sh
> conda activate ${conda_env_name}
> pip install --yes dill== 0.3.1.1 glog==0.3.5 protobuf==3.7.1
> conda deactivate
  • If plugin_tar_file_path is specified and the path is also set in the body of the data, the path value will not take effect. The uploaded files found in the location specified by plugin_tar_file_path will be used.
  • How to tar your plugin scripts into a plugin tar file:
    1. Put all your scripts under a plugin root folder, for example: /tmp/your_plugin_root_folder, the folder must contain a optimizer.py file which implements the BasePluginOptimizer class, together with other modules and files for the hyperparameter search plugin.
    2. Navigate to the plugin root folder, /tmp/your_plugin_root_folder:
      cd /tmp/your_plugin_root_folder/
    3. Tar the plugin scripts:
      tar cf /tmp/test.tar *
    4. Replace ${plugin_tar_file_path} in the request command with /tmp/test.tar.

Querying a search plugin

There are two ways to query search plugins, either by name or type.
Query plugins by type
Use the following rest command to query all supported tuning algorithms:
> curl -k -X GET \
--header 'Authorization: Bearer <jwttoken>' \
-H 'Accept: application/json' \
'https://${cpd-route}/platform/rest/deeplearning/v1/hypersearch/algorithm?type=${algo_type}'
Parameters are as follows:
Parameter name Type Description Required
algo_type string

The type of tuning algorithm.

Specify type=USER_PLUGIN to query all the user installed plugin algorithm.

Specify type=BUILD_IN to query all the build-in algorithms.

If the parameter is not specified, all algorithms (including build-in and user installed) will be returned.

 false
cpd-route string The route is determined by: 
oc get route -n ${cpd-namespace}
 true
Query plugins by name
Use the following rest command to query all supported tuning algorithms:
> curl -k -X GET \
--header 'Authorization: Bearer <jwttoken>' \
-H 'Accept: application/json' \
'https://${cpd-route}/platform/rest/deeplearning/v1/hypersearch/algorithm/${plugin_algorithm_name}'
Parameters are as follows:
Parameter name Type Description Required
plugin_algorithm_name string name of the user plugin algorithm. true
cpd-route string The route is determined by: 
oc get route -n ${cpd-namespace}
 true

Deleting a search plugin

Use the following rest command to delete a user plugin algorithm:
> curl -k -X DELETE \
--header 'Authorization: Bearer <jwttoken>' \
'https://${cpd-route}/platform/rest/deeplearning/v1/hypersearch/algorithm/${plugin_algorithm_name}
Parameters are as follows:
Parameter name Type Description Required
plugin_algorithm_name string name of the user plugin algorithm. true
cpd-route string The route is determined by: 
oc get route -n ${cpd-namespace}
 true