Supported machine learning tools, libraries, frameworks, and software specifications
In IBM Watson Machine Learning, you can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions. The environment for these models and functions is made up of specific hardware and software specifications.
Software specifications define the language and version that you use for a model or function. They enable you to better configure the software that is used for running your models and functions. By using software specifications, you can precisely define the software version to be used and include your own extensions (for example, by using conda .yml files or custom libraries).
You can get a list of available software and hardware specifications and then use their names and IDs for use with your deployment. For details on how to do that, refer to the documentation for Python client or REST API.
Important: The tables in this topic document the supported frameworks and software specifications for the current release of Cloud Pak for Data. To see the list of supported frameworks and software specifications for a specific refresh version of Cloud Pak for Data, open the PDF file for "Deploying and managing models and functions" for that refresh version in Documentation for previous 4.6.x refreshes.
Predefined software specifications
you can use popular tools, libraries, and frameworks to train and deploy machine learning models and functions
This table lists the predefined (base) model types and software specifications.
Framework | Versions | Model Type | Default Software specification |
Supported platforms |
---|---|---|---|---|
AutoAI | 0.1 | NA | hybrid_0.1 autoai-kb_rt22.1-py3.9 (deprecated) Deprecated since release 4.6.4 autoai-kb_rt22.2-py3.10 autoai-ts_rt22.1-py3.9 (deprecated) Deprecated since release 4.6.4 autoai-ts_rt22.2-py3.10 |
x86, PPC |
Decision Optimization | 20.1 | do-docplex_20.1 do-opl_20.1 do-cplex_20.1 do-cpo_20.1 |
do_20.1 | x86, PPC |
Decision Optimization | 22.1 | do-docplex_22.1 do-opl_22.1 do-cplex_22.1 do-cpo_22.1 |
do_22.1 | x86, PPC |
Hybrid/AutoML | 0.1 | wml-hybrid_0.1 | hybrid_0.1 | x86, PPC |
PMML | 3.0 to 4.3 | pmml_. (or) pmml_..*3.0 - 4.3 | pmml-3.0_4.3 | x86, PPC |
PyTorch | 1.10 | pytorch-onnx_1.10 pytorch-onnx_rt22.1 |
runtime-22.1-py3.9 (deprecated) Deprecated since release 4.6.4 pytorch-onnx_rt22.1-py3.9 (deprecated) Deprecated since release 4.6.4 pytorch-onnx_rt22.1-py3.9-edt (deprecated) Deprecated since release 4.6.4 |
x86, PPC, s390x |
PyTorch | 1.10 | pytorch-onnx_1.10 pytorch-onnx_rt22.1 |
runtime_22.1-py3.9-nnpa | s390x |
PyTorch | 1.12 | pytorch-onnx_1.12 pytorch-onnx_rt22.2 |
runtime-22.2-py3.10 pytorch-onnx_rt22.2-py3.10 pytorch-onnx_rt22.2-py3.10-edt |
x86, PPC Added in release 4.6.5: s390x |
PyTorch | 1.12 | pytorch-onnx_1.12 pytorch-onnx_rt22.2 |
Added in release 4.6.1: pytorch-onnx_rt22.2-py3.10-dist(x86) |
x86 |
Python Functions | 0.1 | NA | runtime-22.1-py3.9 (deprecated) Deprecated since release 4.6.4 |
x86, PPC, s390x |
Python Functions | 0.1 | NA | runtime_22.1-py3.9-nnpa | s390x |
Python Functions | 0.1 | NA | runtime-22.2-py3.10 | x86, PPC |
Python Functions | 0.1 | NA | runtime-22.2-py3.10 | s390x Added in release 4.6.3 |
Python Scripts | 1.0 | NA | runtime-22.1-py3.9 (deprecated) Deprecated since release 4.6.4 |
x86, PPC, s390x |
Python Scripts | 1.0 | NA | runtime_22.1-py3.9-nnpa | s390x |
Python Scripts | 1.0 | NA | runtime-22.2-py3.10 | x86, PPC |
Python Scripts | 1.0 | NA | runtime-22.2-py3.10 | s390x Added in release 4.6.3 |
R Scripts | 1.0 | NA | default_r3.6 (deprecated) | x86, PPC |
R Scripts | 1.0 | NA | runtime-22.1-r3.6 (deprecated) runtime-22.2-r4.2 |
x86 |
Scikit-learn | 1.0 | scikit-learn_1.0 | runtime-22.1-py3.9 (deprecated) Deprecated since release 4.6.4 |
x86, PPC, s390x |
Scikit-learn | 1.0 | scikit-learn_1.0 | runtime_22.1-py3.9-nnpa | s390x |
Scikit-learn | 1.1 | scikit-learn_1.1 | runtime-22.2-py3.10 | x86, PPC |
Scikit-learn | 1.1 | scikit-learn_1.1 | runtime-22.2-py3.10 | s390x Added in release 4.6.3 |
Spark | 3.2 | mllib_3.2 | spark-mllib_3.2 (deprecated) | x86, PPC |
Spark | 3.3 | mllib_3.3 | spark-mllib_3.3 | x86, PPC |
SPSS | 17.1 | spss-modeler_17.1 | spss-modeler_17.1 | x86, PPC |
SPSS | 18.1 | spss-modeler_18.1 | spss-modeler_18.1 | x86, PPC |
SPSS | 18.2 | spss-modeler_18.2 | spss-modeler_18.2 | x86, PPC |
Tensorflow | 2.7 | tensorflow_2.7 tensorflow_rt22.1 |
runtime-22.1-py3.9 (deprecated) Deprecated since release 4.6.4 tensorflow_rt22.1-py3.9 (deprecated) Deprecated since release 4.6.4 |
x86, PPC, s390x |
Tensorflow | 2.7 | tensorflow_2.7 tensorflow_rt22.1 |
tensorflow_rt22.1-py3.9-nnpa (deprecated) Deprecated since release 4.6.4 |
s390x |
Tensorflow | 2.9 | tensorflow_2.9 tensorflow_rt22.2 |
runtime-22.2-py3.10 tensorflow_rt22.2-py3.10 |
x86, PPC |
Tensorflow | 2.9 | tensorflow_2.9 tensorflow_rt22.2 |
runtime-22.2-py3.10 tensorflow_rt22.2-py3.10 |
s390x Added in release 4.6.3 |
Tensorflow | 2.9 | tensorflow_2.9 tensorflow_rt22.2 |
Added in release 4.6.1: tensorflow_rt22.2-py3.10-dist(x86) tensorflow_rt22.2-py3.10-edt(x86) |
x86 |
XGBoost | 1.5 | xgboost_1.5 or scikit-learn_1.0 (see notes) | runtime-22.1-py3.9 (deprecated) Deprecated since release 4.6.4 |
x86, PPC, s390x |
XGBoost | 1.5 | xgboost_1.5 or scikit-learn_1.0 (see notes) | runtime_22.1-py3.9-nnpa | s390x |
XGBoost | 1.6 | xgboost_1.6 or scikit-learn_1.1 (see notes) | runtime-22.2-py3.10 | x86, PPC |
XGBoost | 1.6 | xgboost_1.6 or scikit-learn_1.1 (see notes) | runtime-22.2-py3.10 | s390x Added in release 4.6.3 |
Important:
- For XGBoost, if model is trained with sklearn wrapper (XGBClassifier or XGBRegressor):
- in Python 3.9, use the
scikit-learn_1.0
model type. - in Python 3.10, use the
scikit-learn_1.1
model type.
- in Python 3.9, use the
- You can also deploy R Shiny apps (version 0.1). Software specifications:
rstudio_r4.2
(for x86) andshiny-r3.6
(deprecated, used for x86 and PPC). - If a framework version is marked as deprecated, then support for this framework will be removed in a future release.
- Training a model based on Tensorflow and PyTorch requires using Watson Studio or Watson Machine Learning Accelerator.
Discontinued model types and software specifications
Support for the following model types was discontinued:
Model types | End of support |
---|---|
do-docplex_12.10 do-opl_12.10 do-cplex_12.10 do-cpo_12.10 |
4.0.9 |
do-docplex_12.9 do-opl_12.9 do-cplex_12.19 do-cpo_12.9 |
4.0.7 |
mllib_2.4 | 4.0.7 |
mllib_2.4 (for PMML deployments) |
4.0.8 |
mllib_3.0 | 4.6 |
pytorch-onnx_1.3 | 4.0.6 |
pytorch-onnx_1.7 | 4.0.8 |
scikit-learn_0.23 | 4.0.8 |
tensorflow_2.1 | 4.0.6 |
tensorflow_2.4 | 4.0.8 |
xgboost_0.90 | 4.0.6 |
xgboost_1.3 | 4.0.8 |
Support for the following software specifications was discontinued:
Software specification | End of support |
---|---|
autoai-kb_3.3-py3.7 | 4.0.8 |
autoai-kb_3.4-py3.8 | 4.0.8 |
autoai-obm_3.0 | 4.6 |
autoai-obm_3.2 | 4.6 |
autoai-ts_3.9-py3.8 | 4.0.8 |
default_py3.7 | 4.0.6 |
default_py3.7_opence | 4.0.8 |
default_py3.8 | 4.0.8 |
do_12.10 | 4.0.9 |
do_12.9 | 4.0.7 |
pytorch-onnx_1.3-py3.7 | 4.0.6 |
pytorch-onnx_1.3-py3.7-edt | 4.0.6 |
spark-mllib_3.0 | 4.5 (PMML model type only) |
spark-mllib_2.4 | 4.0.7 |
spark-mllib_2.4 (for PMML deployments) |
4.0.8 |
spark-mllib_3.0 | 4.6 |
tensorflow_2.4-py3.7 | 4.0.8 |
tensorflow_2.4-py3.8 | 4.0.8 |
When you have assets that rely on discontinued software specifications or frameworks, in some cases the migration is seamless. In other cases, your action is required to retrain or redeploy assets.
- Existing deployments of models that are built with discontinued framework versions or software specifications will be removed on the date of discontinuation.
- No new deployments of models that are built with discontinued framework versions or software specifications will be allowed.
- If you upgrade from a previous version of Cloud Pak for Data, deployments of models, functions, or scripts that are based on unsupported frameworks are removed. You must re-create the deployments with supported frameworks.
- If you upgrade from a previous version of Cloud Pak for Data and you have models that use unsupported frameworks, you can still access the models. However, you cannot train or score them until you upgrade the model type and software specification, as described in Managing outdated software specifications or frameworks.
Runtime differences
For various reasons, package versions installed in Watson Machine Learning runtimes and Watson Studio Notebook runtimes can be different, even if they are based on the same software specification.
Differences in release 4.6.0:
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
cloudpickle |
1.6.0 | 2.0.0 |
fsspec |
2021.10.1 | 2022.2.0 |
werkzeug |
2.1.1 | 2.0.3 |
Differences in release 4.6.1:
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
cloudpickle |
1.6.0 | 2.0.0 |
fsspec |
2021.10.1 | 2022.2.0 |
werkzeug |
2.1.1 | 2.0.3 |
huggingface_hub |
Not available | 0.2.1 |
importlib-metadata |
Not available | 4.11.3 |
docplex |
2.23.222 | 2.22.213 |
ibm-watson-openscale |
3.0.26 | 3.0.27 |
dsx_core_utils |
4.6.0.12 | 4.6.1-5 |
hadoop_lib_utils |
4.6.0.12 | 4.6.1-5 |
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
libcurl |
7.85.0 | 7.84.0 |
ibm-watson-openscale |
3.0.26 | 3.0.27 |
docplex |
2.23.222 | 2.22.213 |
dsx_core_utils |
4.6.0.12 | 4.6.1-5 |
hadoop_lib_utils |
4.6.0.12 | 4.6.1-5 |
Differences in release 4.6.3:
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
py4j |
0.10.9.3 | 0.10.9.2 |
werkzeug |
2.2.2 | 2.0.3 |
zlib |
1.2.13 | 1.2.12 |
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
libcurl |
7.85.0 | 7.84.0 |
werkzeug |
2.2.2 | 2.0.3 |
mysql-connector-python |
8.0.31 | 8.0.30 |
Differences in release 4.6.4:
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
pillow |
9.3.0 | 9.0.1 |
werkzeug |
2.1.1 | 2.0.3 |
zlib |
1.2.13 | 1.2.12 |
py4j |
0.10.9.3 | 0.10.9.2 |
ibm-watson-machine-learning |
1.0.283 | 1.0.288 |
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
libcurl |
7.87.0 | 7.84.0 |
mysql-connector-python |
8.0.32 | 8.0.30 |
ibm-watson-machine-learning |
1.0.283 | 1.0.288 |
Differences in release 4.6.5:
Package | Watson Machine Learning version | Watson Studio Notebook version |
---|---|---|
pillow | 9.3.0 | 9.0.1 |
werkzeug | 2.1.1 | 2.2.3 |
zlib | 1.2.13 | 1.2.12 |
Note:
Deployment images based on the runtime-22.2-py3.10
software specification for Watson Machine Learning and Watson Studio do not differ.
Learn more
-
For details on customizing software specifications, refer to Customizing with third-party and private Python libraries.
-
For details on using and customizing environments, refer to Environments.
-
For specific deployment examples, refer to sample Jupyter notebooks:
Parent topic: Managing frameworks and software specifications