Running the containers

IMPORTANT: Before running a container, review the Prerequisites topic.

There are different options for deploying the Watson NLP Runtime image. Deployment modes have different advantages and disadvantages, depending on your organization's needs.

Deployment modes

Standalone deployment

The standalone deployment option allows you to build a docker volume out of pretrained model images and mount it into a running Watson NLP Runtime container. When the container runs it will expose REST and gRPC endpoints that client programs can use to make inference requests. This is the quickstart, local deployment option and is good for demonstrating the capabilities of Watson NLP.

See Run with Docker run to deploy this way.

Single image serverless deployment

Single image serverless deployments provide a custom image build with a minimal set of library dependencies and pretrained models that can be deployed to a runtime offering such as IBM Cloud Code Engine, AWS Fargate, or Microsoft Azure Serverless.

Advantages of this type of deployment include:

Disadvantages to this type of deployment include:

See Run with a serverless container runtime offering to deploy this way.

Static model multi-pod deployment

Static model multi-pod deployments on Kubernetes are similar to Serverless deployments, serving a static set of pretrained models with no extra dependencies other than a vanilla Kubernetes cluster.

Advantages of this type of deployment include:

Disadvantages to this type of deployment include:

See Run with Kubernetes to deploy this way.

Dynamic model multi-pod deployment

Deployments using KServe ModelMesh Serving provide a cloud-native API for dynamic model management, and can be configured to handle internet-scale load. Advanced load-balancing and scaling techniques manage replicas of individual models across a serving cluster and automatically adjust to changing demand. This is IBM's recommended deployment method for production use of Watson NLP models.

Advantages of this type of deployment include:

Disadvantages to this type of deployment include:

See Run with Kubernetes and KServe ModelMesh Serving to deploy this way.