Deploying a trained model

Deploy a trained model to get it ready to use within PowerAI Vision or a different program, such as IBM® PowerAI. Deploying a model creates a unique API endpoint based on that model for inference operations.

1. Click Models from the menu.
2. Select the model you want to deploy and click Deploy.
3. Specify a name for the model, and for models that were trained with the Optimized for speed (tiny YOLO v2) model, choose the accelerator to deploy to. You can choose GPU, CPU, or Xilinx FPGA - 16 bit (technology preview).
   Note: Deploying a model to a Xilinx FPGA requires the Xilinx Alveo U200 Accelerator card.
   GPUs are used as follows:
   • Each Tiny YOLO V2, Detectron, Single Shot Detector (SSD), Structured segment network (SSN), or custom deployed model takes one GPU. The GPU group is listed as '-', which indicates that this model uses a full GPU and does not share the resource with any other deployed models.
   • Multiple Faster R-CNN and GoogLeNet models are deployed to a single GPU. PowerAI Vision uses packing to deploy the models. That is, the model is deployed to the GPU that has the most models deployed on it, if there is sufficient memory available on the GPU. The GPU group can be used to determine which deployed models share a GPU resource. To free up a GPU, all deployed models in a GPU group must be deleted (undeployed).
     Note: PowerAI Vision leaves a 500MB buffer on the GPU.
4. Click Deploy. The Deployed Models page is displayed. When the model has been deployed, the status column displays Ready.
5. Click the deployed model to get the API endpoint, to view details about the model, such as the owner and the accuracy, and to test other videos or images against the model. For information about using the API see Vision Service API documentation.
   Note: When using the API, the smaller confidence threshold you specify, the more results are returned. If you specify 0, many, many results will be returned because there is no filter based on the confidence level of the model.
6. If necessary, you can delete a deployed model. To delete a deployed model, click Deployed Models. Next, select the model that you want to delete and click Delete. The trained model is not deleted from PowerAI Vision.