Configuring the ONNX compiler service for your ML for IBM z/OS

If you plan to import, deploy, and manage your ONNX models with ML for IBM z/OS®, you must first configure the required ONNX compiler service. With the service, your ONNX models are compiled into an executable format during import, uploaded into the MLz repository, and readied for deployment and scoring.

Before you begin

Only one ONNX compiler service is required for each installation of the MLz. Before you start, make sure that you have completed the following tasks:

Make sure that the MLz core service that will be used to import the ONNX model must be installed and configured on hardware that is equivalent to or higher than the hardware where the MLz scoring will be deployed.
The core service performs ONNX model compilation using the z Deep Learning Compiler (zDLC) and generates model versions compatible only up to the hardware level on which it is installed. For example:
- If you plan to use the IBM® z17™ on-chip AI accelerator for scoring ONNX models, both the core service and the scoring service must be configured on an IBM z17™.
- If you plan to use the IBM z16™ on-chip AI accelerator for scoring ONNX models, you can install the core service on either an IBM z16 or an IBM z17™, and the scoring service must run on IBM z16.
Review the Installation roadmap and complete all preceding tasks in the sequence.
Provision a z/OS Container Extensions (zCX) instance or a Linux® on Z server with Docker installed as required in Installing prerequisite hardware and software for ML for IBM z/OS.
Make sure xlc utility is configured for <mlz_setup_userid> as described in Configuring user ID for setting up Machine Learning for IBM z/OS Enterprise.
Create user <docker_user> with all required privileges as described in Additional configuration steps for ML for z/OS Enterprise.

Procedure

Start an USS session as <mlz_setup_userid>.
Locate the onnx-compiler.sh script in the $IML_INSTALL_ENT_DIR/iml-onnx/onnx-compiler directory on the system where your MLz runs.

You can use this script to create, configure, and manage the ONNX compiler service within a zCX instance or a Linux on Z server.
Create the compiler service by issuing the following command:
```
./onnx-compiler.sh create
```
When prompted, respond by entering the requested information or accepting the default value for each question:
- Enter the IP address or hostname of the zCX instance or the Linux on Z server.
- Enter the SSH port number of the zCX instance or the Linux on Z server.
- Enter the SSH username <docker_user> of the zCX instance or the Linux on Z server.
  The <docker_user> will be used to connect to the zCX instance or the Linux on Z server, and start the docker container of the ONNX compiler service.
- Select the authentication type for zCX or Linux on Z server SSH login by the <docker_user>:
  - If you select password, enter the SSH password.
  - If you select passphrase, enter the full path of the private ssh key file and optionally the SSH passphrase.
- Enter the port number of the ONNX compiler service on zCX or Linux on Z server or press Enter to use default port 18080.
After collecting all required information, the script loads the ONNX compiler service image and related assets into the zCX instance or the Linux on Z server. It returns a message similar to the following example when the service is successfully created:
```
ONNX compiler service created successfully.
```
Start the ONNX compiler service by entering Y when prompted.
If you prefer, you can start the service at a later time by running the following command by using SSH:
```
./onnx-compiler.sh start
```
Verify that the ONNX compiler service is successfully started.
The onnx-compiler.sh script returns a message similar to the following example when the service is successfully started:
```
onnx-compiler-service container <onnx-compiler-service-containerID> started successfully on zCX.
Onnx compiler service started successfully. See details in the log /media/azd_shared_volume/iml-home/
           onnx-compiler/logs/onnx-compiler-service.log on zCX.
```
If needed, log in the zCX instance or the Linux on Z server via SSH and review onnx-compiler-service.log for details.
If necessary, update the compiler service by issuing the following command:
```
./onnx-compiler.sh update
```
If you changed the port number of the ONNX compiler service and the IP address (or hostname) of the zCX instance or the Linux on Z server where the compiler service runs, you must restart your MLz UI service to reactivate the ONNX support. See Stopping and starting services running on ML for IBM z/OS for instructions.
If necessary, stop the compiler service by issuing the following command:
```
./onnx-compiler.sh stop
```