Creating a custom software specification in a project

Learn how to create a custom software specification in a project and then promote it to a deployment space.

If your model requires custom components such as user-defined transformers, estimators, or user-defined tensors, you can create a custom software specification that is derived from a base, or a predefined specification. Python functions and Python scripts also support custom software specifications.

You can use custom software specification to reference any third-party libraries, user-created Python packages, or both. Third-party libraries or user-created Python packages must be specified as package extensions that can then be referenced in a custom software specification.

Custom software specifications are promoted with the assets that use them. Python functions and scripts, and models that are created with the Scikit-learn, XGBoost, and Tensorflow frameworks can use the software specifications for deployments.

Software specifications are also included when you import a project or space that includes one.

For details on creating environments with software specifications, see Environments. For details on adding custom components, see Custom components.

Creating a custom software specification

The high-level steps to create a custom software specification that uses third-party libraries and user-created Python packages.

Create a custom software specification
Save a conda YAML file that contains a list of third-party libraries and create a package extension. Note: This step is not required if the model does not have any dependency on a third-party library.
Add a reference of the package extensions to the custom software specification that you created.

Creating a custom software specification in a notebook

You can use the Watson Machine Learning APIs or Python client to define a custom software specification that is derived from a base specification.

These notes are applicable when you are specifying software specification for Scikit-Learn, XGBoost, Tensorflow, Keras or PyTorch models trained in Watson Studio notebooks, or Python functions developed in a notebook.

If you created a model or Python function in a Watson Studio notebook with "Default Python 3.9" environment, then save it with a reference to the software specification by using the name "default_py3.9". To get the corresponding software specification ID, use:
```
 softwareSpecId = wml_client.software_specifications.get_id_by_name('default_py3.9')
```
You can create custom environments in Watson Studio to install third-party libraries that are required to run your Python scripts. If you trained a model or developed a Python function in a Watson Studio notebook with a customized environment, when you save it, specify the reference to the custom software specification. The name of the custom software specification is the same as the name of the custom environment. For example, if the custom environment is called "my_cust_model1_env":
```
  softwareSpecId = wml_client.software_specifications.get_id_by_name('my_cust_model1_env')
```

Creating a custom software specification by using the Python client

Authenticate and create the client.

Refer to Authentication.

Create and set the default deployment space, then list available software specifications.

 metadata = {
     wml_client.spaces.ConfigurationMetaNames.NAME:
         'examples-create-software-spec',
     wml_client.spaces.ConfigurationMetaNames.DESCRIPTION:
         'For my models'
 }
 space_details = wml_client.spaces.store(meta_props=metadata)
 space_uid = wml_client.spaces.get_id(space_details)

 # set the default space
 wml_client.set.default_space(space_uid)

 # see available meta names for software specs
 print('Available software specs configuration:', wml_client.software_specifications.ConfigurationMetaNames.get())
 wml_client.software_specifications.list()

 asset_id = 'undefined'
 pe_asset_id = 'undefined'

Create the metadata for package extensions to add to the base specification.

 pe_metadata = {
     wml_client.package_extensions.ConfigurationMetaNames.NAME:
         'My custom library',
     # optional:
     # wml_client.software_specifications.ConfigurationMetaNames.DESCRIPTION:
     wml_client.package_extensions.ConfigurationMetaNames.TYPE:
         'conda_yml'
 }

Create a yaml file that contains the list of additional packages and then save it as customlibrary.yaml.

Example yaml file:
```
 name: add-regex-package
 dependencies:
     - regex
```
For information on the proper yaml file structure, refer to Examples of customizations.

Store package extension information.

 pe_asset_details = wml_client.package_extensions.store(
     meta_props=pe_metadata,
     file_path='customlibrary.yaml'
 )
 pe_asset_id = wml_client.package_extensions.get_id(pe_asset_details)

Create the metadata for the software specification and store the software specification.

 # Get the id of the base software specification
 base_id = wml_client.software_specifications.get_id_by_name('default_py3.9')

 # create the metadata for software specs
 ss_metadata = {
     wml_client.software_specifications.ConfigurationMetaNames.NAME:
         'Python 3.9 with pre-installed ML package',
     wml_client.software_specifications.ConfigurationMetaNames.DESCRIPTION:
         'Adding some custom libraries like regex', # optional
     wml_client.software_specifications.ConfigurationMetaNames.BASE_SOFTWARE_SPECIFICATION:
         {'guid': base_id},
     wml_client.software_specifications.ConfigurationMetaNames.PACKAGE_EXTENSIONS:
         [{'guid': pe_asset_id}]
 }

 # store the software spec
 ss_asset_details = wml_client.software_specifications.store(meta_props=ss_metadata)

 # get the id of the new asset
 asset_id = wml_client.software_specifications.get_id(ss_asset_details)

 # view new software specification details
 import pprint as pp

 ss_asset_details = wml_client.software_specifications.get_details(asset_id)
 print('Package extensions', pp.pformat(
     ss_asset_details['entity']['software_specification']['package_extensions']
 ))

Propagating software specifications and package extensions from projects to deployment spaces

You can export custom software specifications and package extensions that you created in a Watson Studio project to a deployment space by using the "Promote" option in the Watson Studio interface. If you want to update software specifications and package extensions after they were promoted to the deployment space, follow these steps:

In the deployment space, delete the software specifications, package extensions, and associated models (optional) by using the Watson Machine Learning Python client.
In a Watson Studio project, promote the model, function, or script that is associated with the changed custom software specification and package extension to the space.

Parent topic: Managing frameworks and software specifications