Supported machine learning tools, libraries, frameworks, and software specifications
In IBM Watson Machine Learning, you can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions. The environment for these models and functions is made up of specific hardware and software specifications.
Software specifications define the language and version that you use for a model or function. You can use software specifications to configure the software that is used for running your models and functions. By using software specifications, you can precisely define the software version to be used and include your own extensions (for example, by using conda .yml files or custom libraries).
You can get a list of available software and hardware specifications and then use their names and IDs for use with your deployment. For more information, see Python client or REST API.
Predefined software specifications
You can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions.
The following table lists the predefined (base) model types and software specifications. If you see (F) next to the name of a software specification, it means that you can use this software specification that is FIPS 140-2 compliant.
For more information, see Services that support FIPS. The s390x platform does not support FIPS, even if you use FIPS-compliant software specifications.
| Framework | Versions | Model Type | Default Software specification |
Supported platforms |
|---|---|---|---|---|
| AutoAI | 0.1 | NA | hybrid_0.1 autoai-kb_rt23.1-py3.10 (F) autoai-ts_rt23.1-py3.10 (F) autoai-tsad_rt23.1-py3.10 (F) |
x86, s390x, PPC |
| AutoAI | 0.1 | NA | hybrid_0.1 (deprecated) autoai-kb_rt22.2-py3.10 (F)(deprecated) autoai-ts_rt22.2-py3.10 (F)(deprecated) |
x86 (deprecated), PPC(deprecated), s390x(deprecated) |
| Decision Optimization | 20.1 | do-docplex_20.1 do-opl_20.1 do-cplex_20.1 do-cpo_20.1 |
do_20.1 (F) | x86, PPC |
| Decision Optimization | 22.1 | do-docplex_22.1 do-opl_22.1 do-cplex_22.1 do-cpo_22.1 |
do_22.1 (F) | x86, PPC |
| Hybrid/AutoML | 0.1 | wml-hybrid_0.1 | hybrid_0.1 (F) | x86, PPC, s390x |
| PMML | 3.0 to 4.3 | pmml_. (or) pmml_..*3.0 - 4.3 | pmml-3.0_4.3 (F) | x86, PPC |
| PyTorch | 1.12(deprecated) | pytorch-onnx_1.12(deprecated) pytorch-onnx_rt22.2(deprecated) |
runtime-22.2-py3.10 (F)(deprecated) pytorch-onnx_rt22.2-py3.10 (F)(deprecated) pytorch-onnx_rt22.2-py3.10-edt (F)(deprecated) |
x86(deprecated), PPC(deprecated), s390x(deprecated) |
| PyTorch | 1.12(deprecated) | pytorch-onnx_1.12(deprecated) pytorch-onnx_rt22.2(deprecated) |
pytorch-onnx_rt22.2-py3.10-dist(x86) (F)(deprecated) | x86(deprecated) |
| PyTorch | 2.0 | pytorch-onnx_2.0 pytorch-onnx_rt23.1 |
runtime-23.1-py3.10 (F) pytorch-onnx_rt23.1-py3.10 (F) pytorch-onnx_rt23.1-py3.10-edt (F) pytorch-onnx_rt23.1-py3.10-dist (F) |
x86, s390x, PPC |
| Python Functions | NA | NA | runtime-22.2-py3.10 (F)(deprecated) | x86(deprecated), PPC(deprecated), s390x(deprecated) |
| Python Functions | NA | NA | runtime-23.1-py3.10 (F) | x86, PPC, s390x |
| Python Functions | NA | NA | runtime-23.1-py3.10-cuda | x86 |
| Python Scripts | NA | NA | runtime-22.2-py3.10 (F)(deprecated) | x86(deprecated), PPC(deprecated), s390x(deprecated) |
| Python Scripts | NA | NA | runtime-23.1-py3.10 (F) | x86, s390x, PPC |
| R Scripts | NA | NA | runtime-22.2-r4.2 (F)(deprecated) runtime-23.1-r4.2 (F) |
x86 |
| R Shiny applications | NA | NA | rstudio_r4.2 (deprecated) rstudio-23.1-r4.2 |
x86 |
| Scikit-learn | 1.1 | scikit-learn_1.1 | runtime-22.2-py3.10 (F)(deprecated) | x86(deprecated), PPC(deprecated), s390x(deprecated) |
| Scikit-learn | 1.1 | scikit-learn_1.1 | runtime-23.1-py3.10 (F) | x86, s390x, PPC |
| Spark | 3.3 | mllib_3.3 | spark-mllib_3.3 (F) | x86, PPC |
| Spark | 3.4 | mllib_3.4 | spark-mllib_3.4 (F) | x86, PPC |
| SPSS | 17.1 | spss-modeler_17.1 (see notes) | spss-modeler_17.1 (F) | x86 |
| SPSS | 18.1 | spss-modeler_18.1 (see notes) | spss-modeler_18.1 (F) | x86 |
| SPSS | 18.2 | spss-modeler_18.2 (see notes) | spss-modeler_18.2 (F) | x86 |
| Tensorflow | 2.9(deprecated) | tensorflow_2.9(deprecated) tensorflow_rt22.2(deprecated) |
runtime-22.2-py3.10 (F)(deprecated) tensorflow_rt22.2-py3.10 (F)(deprecated) |
x86(deprecated), PPC(deprecated), s390x(deprecated) |
| Tensorflow | 2.9(deprecated) | tensorflow_2.9(deprecated) tensorflow_rt22.2(deprecated) |
tensorflow_rt22.2-py3.10-dist(x86) (F)(deprecated) tensorflow_rt22.2-py3.10-edt(x86) (F)(deprecated) |
x86(deprecated), s390x(deprecated), PPC(deprecated) |
| Tensorflow | 2.12 | tensorflow_2.12 tensorflow_rt23.1 |
runtime-23.1-py3.10 (F) tensorflow_rt23.1-py3.10-dist (F) tensorflow_rt23.1-py3.10-edt (F) tensorflow_rt23.1-py3.10 (F) |
x86, s390x, PPC |
| Tensorflow | 2.12 | tensorflow_2.12 tensorflow_rt23.1 |
runtime-23.1-py3.10-cuda | x86 |
| XGBoost | 1.6 | xgboost_1.6 or scikit-learn_1.1 (see notes) | runtime-22.2-py3.10 (F)(deprecated) | x86(deprecated), PPC(deprecated) |
| XGBoost | 1.6 | xgboost_1.6 or scikit-learn_1.1 (see notes) | runtime-23.1-py3.10 (F) | x86, s390x, PPC |
Important:
-
If a framework version is marked as deprecated, then support for this framework will be removed in a future release.
-
For XGBoost, if model is trained with sklearn wrapper (XGBClassifier or XGBRegressor), use the
scikit-learn_1.1model type in Python 3.10. -
You can also deploy R Shiny apps (version 0.1). Software specifications available for Shiny apps:
rstudio_r4.2andrstudio-23.1-r4.2(for x86) andshiny-r3.6(deprecated, used for x86 and PPC). -
None of the R Shiny software specifications are FIPS-compliant.
-
For SPSS Modeler 17.1, 18.1 and 18.2, FIPS compliance is supported only for online deployments. Batch deployments for these model types are not FIPS compliant.
-
Beginning Cloud Pak for Data version 4.8.4, you can create deployments that use GPU inferencing with Watson Machine Learning API for the
runtime-23.1-py3.10-cudasoftware specification. This feature is not available in Cloud Pak for Data versions 4.8.3 and earlier.
Constricted model types and software specifications
Constricted software specifications are only supported in an upgraded instance. They are not supported in new installations, imported space assets, and patch operations.
Support for the following software specifications is constricted:
| Framework | Versions | Model Type | Default software specification | Supported platforms |
|---|---|---|---|---|
| AutoAI | 0.1 | NA | hybrid_0.1autoai-kb_rt22.1-py3.9 autoai-ts_rt22.1-py3.9 |
x86 |
| PyTorch | 1.10 | pytorch-onnx_1.10 pytorch-onnx_rt22.1 |
runtime-22.1-py3.9 pytorch-onnx_rt22.1-py3.9 pytorch-onnx_rt22.1-py3.9-edt |
x86 |
| Python Functions | NA | NA | runtime-22.1-py3.9 | x86 |
| Python Scripts | NA | NA | runtime-22.1-py3.9 | x86 |
| R Scripts | NA | NA | default_r3.6 runtime-22.1-r3.6 |
x86 |
| R Shiny applications | NA | NA | shiny-r3.6 | x86, PPC |
| Scikit-learn | 1.0 | scikit-learn_1.0 | runtime-22.1-py3.9 | x86 |
| Tensorflow | 2.7 | tensorflow_2.7 tensorflow_rt22.1 |
runtime-22.1-py3.9 tensorflow_rt22.1-py3.9 |
x86 |
| XGBoost | 1.5 | xgboost_1.5 or scikit-learn_1.0 (see notes) | runtime-22.1-py3.9 | x86 |
For XGBoost, if model is trained with sklearn wrapper (XGBClassifier or XGBRegressor), use the scikit-learn_1.1 model type in Python 3.10.
Discontinued model types and software specifications
Support for the following model types is discontinued:
| Model types | End of support |
|---|---|
| do-docplex_12.10 do-opl_12.10 do-cplex_12.10 do-cpo_12.10 |
4.0.9 |
| do-docplex_12.9 do-opl_12.9 do-cplex_12.19 do-cpo_12.9 |
4.0.7 |
| mllib_2.4 | 4.0.7 |
| mllib_2.4 (for PMML deployments) |
4.0.8 |
| mllib_3.0 | 4.6 |
| pytorch-onnx_1.3 | 4.0.6 |
| pytorch-onnx_1.7 | 4.0.8 |
| scikit-learn_0.23 | 4.0.8 |
| tensorflow_2.1 | 4.0.6 |
| tensorflow_2.4 | 4.0.8 |
| xgboost_0.90 | 4.0.6 |
| xgboost_1.3 | 4.0.8 |
Support for the following software specifications is discontinued:
| Software specification | End of support |
|---|---|
| autoai-kb_3.3-py3.7 | 4.0.8 |
| autoai-kb_3.4-py3.8 | 4.0.8 |
| autoai-obm_3.0 | 4.6 |
| autoai-obm_3.2 | 4.6 |
| autoai-ts_3.9-py3.8 | 4.0.8 |
| default_py3.7 | 4.0.6 |
| default_py3.7_opence | 4.0.8 |
| default_py3.8 | 4.0.8 |
| do_12.10 | 4.0.9 |
| do_12.9 | 4.0.7 |
| pytorch-onnx_1.3-py3.7 | 4.0.6 |
| pytorch-onnx_1.3-py3.7-edt | 4.0.6 |
| runtime_22.1-py3.9-nnpa | 4.8.0 |
| spark-mllib_2.4 | 4.0.7 |
| spark-mllib_2.4 (for PMML deployments) |
4.0.8 |
| spark-mllib_3.0 | 4.5 (PMML model type only) |
| spark-mllib_3.0 | 4.6 |
| spark-mllib_3.2 | 4.7 |
| tensorflow_2.4-py3.7 | 4.0.8 |
| tensorflow_2.4-py3.8 | 4.0.8 |
| tensorflow_rt22.1-py3.9-nnpa | 4.8.0 |
When you have assets that rely on discontinued software specifications or frameworks, in some cases the migration is seamless. In other cases, your action is required to retrain or redeploy assets.
-
Existing deployments of models that are built with discontinued framework versions or software specifications are removed on the date of discontinuation.
-
No new deployments of models that are built with discontinued framework versions or software specifications are allowed.
-
If you upgrade from a previous version of Cloud Pak for Data, deployments of models, functions, or scripts that are based on unsupported frameworks are removed. You must re-create the deployments with supported frameworks.
-
If you upgrade from a previous version of Cloud Pak for Data and you have models that use unsupported frameworks, you can still access the models. However, you cannot train or score them until you upgrade the model type and software specification, as described in Managing outdated software specifications or frameworks.
Runtime differences
For various reasons, package versions that are installed in Watson Machine Learning runtimes and Watson Studio Notebook runtimes can be different, even if they are based on the same software specification.
Differences in release 4.8.0:
| Package | Watson Machine Learning version | Watson Studio Notebook version |
|---|---|---|
freetds |
1.3.3 | 1.3.18 |
ipython |
8.10.0 | 8.4.0 |
krb5 |
1.19.4 | 1.20.1 |
libcurl |
7.88.1 | 8.1.1 |
libnghttp2 |
1.46.0 | 1.52.0 |
libpq |
12.9 | 12.15 |
prompt-toolkit |
3.0.36 | 3.0.20 |
readline |
8.1.2 | 8.2 |
| Package | Watson Machine Learning version | Watson Studio Notebook version |
|---|---|---|
freetds |
1.3.3 | 1.3.18 |
krb5 |
1.19.4 | 1.20.1 |
libcurl |
7.88.1 | 8.2.1 |
libnghttp2 |
1.46.0 | 1.52.0 |
libpq |
12.9 | 12.15 |
urllib3 |
1.26.15 | 1.26.18 |
Beginning Cloud Pak for Data version 4.8.4, you can reuse Watson Studio runtimes for Watson Machine Learning as the base image for x86 platform to ensure that you are using the same runtime environment to build and deploy the model. This feature is not available in Cloud Pak for Data versions 4.8.3 and earlier. For more information, see Customizing Python deployment images in Watson Machine Learning.
Learn more
-
To learn more about how to customize software specifications, see Customizing with third-party and private Python libraries.
-
To learn more about how to use and customize environments, see Environments.
-
To learn more about how to use software specifications for deployments, see the following Jupyter notebooks:
Parent topic: Frameworks and software specifications