Installing the MLDL frameworks
Find instructions for installing the machine learning and deep learning (MLDL) frameworks.
- Setting up the software repository
- Creating conda environments (recommended)
- Installing all frameworks at the same time
- Installing frameworks individually
- Accepting the WML CE license agreement
- Helping the conda installer: strict channel priority and the powerai-release package
- Additional conda channels
Setting up the software repository
The WML CE MLDL packages are distributed as conda packages in an online conda repository. conda must be configured to give priority to installing packages from this channel.
Add the WML CE channel to the conda configuration by running the following command:
conda config --prepend channels \ https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/
Creating conda environments (recommended)
With conda, you can create environments that have different versions of Python or packages installed in them. Conda environments are optional but recommended. If not used, packages are installed in the default environment called base, which often has a higher risk of containing conflicting packages or dependencies. Switching between environments is called activating the environment.
The syntax to create and activate a conda environment is:
conda create --name <environment name> python=<python version> conda activate <environment name>
The only valid Python versions with WML CE are Python 3.6 and 3.7.
For example, to create an environment named wmlce_env with Python 3.6:
conda create --name wmlce_env python=3.6 conda activate wmlce_env
For more information on what you can do with conda environment see https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html.
Installing all frameworks at the same time
All the MLDL frameworks except RAPIDS packages can be installed at the same time by
powerai meta-package. All the RAPIDS packages can be installed using the
- To install
poweraiGPU packages in the conda environment, run:
conda install powerai
- To install
poweraiCPU packages in the conda environment, run:
conda install powerai-cpu
- To install RAPIDS packages, run:
conda install powerai-rapids
Installing frameworks individually
You can install the MLDL frameworks individually. The framework packages include the following versions.
|Package||Description||Version||Available on ppc64le||Available on x86_64|
||IBM-optimized version of Berkeley Vision and Learning Center Caffe||1.0.0||X||X|
||IBM-optimized Caffe CPU-only package||1.0.0||X||X|
||Distributed Deep Learning||1.5.0||X||X|
||WML CE Snap ML||1.5.0||X|
||WML CE Snap ML Spark||1.4.0||X|
||TensorFlow CPU-only package||1.15||X||X|
||TensorFlow with GPU support||1.15||X||X|
||xgboost with GPU support||0.90||X||X|
||xgboost CPU-only package||0.90||X||X|
With the conda environment activated, run the following command:
conda install <package name>
Accepting the WML CE license agreement
During the conda install, the packages are downloaded from the internet and after downloading, the license agreement is presented. Read the license agreement and accept the terms and conditions to complete the install. If you decline the license agreement the packages are not installed.
After you finish reading the license agreement, future installations can be automated to silently accept the license agreement by running the following command before running the conda install command:
The license accept has to be done only once on a per user basis.
Helping the conda installer: strict channel priority and the powerai-release package
The conda installer uses a set of rules to determine which packages to install. Channel priorities and package versions are weighted heavily, but the installer also considers factors such as the number of packages that would need to be installed, whether any packages would need to be upgraded or removed, and so on.
- Packages from Anaconda channels over the WML CE channel in spite of channel priorities.
- Packages from an older release of WML CE in spite of newer versions being available.
You can guide the conda installer to ensure that it chooses the desired WML CE package using the strict channel priority option and the powerai-release meta-package.
Strict channel priority
The strict channel priority option forces the conda installer to give additional weight to the
priority of channels defined in the configuration. It is useful in cases where the conda installer
is preferring packages from lower-priority channels. The simplest use is just to add
--strict-channel-priority to the install command:
(my-wmlce-env) $ conda install --strict-channel-priority tensorflow
(my-wmlce-env) $ conda config --show ... channel_priority: flexible channels: - https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/ - defaults ...
You could permanently change the channel priority setting to strict:
(my-wmlce-env) $ conda config --set channel_priority strict
The powerai-release meta-package can be used to specify the WML CE release you want to install from. It is useful when the installer prefers packages from an earlier release, or if you intentionally want to install packages from an older release.
(my-wmlce-env) $ conda install pytorch powerai-release=1.6.2 (my-wmlce-env) $ conda install pytorch powerai-release=1.6.1
--strict-channel-priority option can be used with
powerai-release for greater control:
(my-wmlce-env) $ conda install --strict-channel-priority pytorch powerai-release=1.6.2
For more information on the conda installer's rules, see
Running the solver at: Understanding and Improving Conda’s performance.
Additional conda channels
The main WML CE conda channel is described above. That channel includes the formal, supported WML CE releases.
Additional conda channels are available to complement the main channel. Packages in these channels are not formally supported. Both of these channels are optional. WML CE will install and run fine without either. They can also be used independently of each other (Supplementary does not need Early Access or vice versa). Use them if you want the packages they provide and do not need formal support.
- The WML CE Supplementary channel is
available at: https://anaconda.org/powerai/.
This channel includes packages that are not part of WML CE, but which may be useful to WML CE users. The packages are built from recipes in the WML CE GitHub repository: https://github.com/ibm/powerai.
Problem reports and recipe contributions from the community are welcome. More information about the Supplementary channel can be found in the PowerAI Supplementary Channel README.
- The WML CE Early Access channel is available at: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access/.
This channel is updated occasionally with latest versions of various packages included in WML CE. The purpose of the channel is to make new versions of frameworks available in advance of formal WML CE releases. Packages published in the Early Access channel may not exactly match a later WML CE release. For example, package and prerequisite versions may differ.Packages in the Early Access channel might depend on packages in the main channel, so both channels might be needed in the conda config.Note: Easy upgrade between packages in the main and Early Access channels is not guaranteed. Using a separate conda environment for Early Access packages is recommended.
- Early Access channel (optional, higher priority)
- WML CE main channel
- Supplementary channel (optional, lower priority)