Creating online deployments in Watson Machine Learning

Create an online (also called Web service) deployment to load a model or Python code when the deployment is created to generate predictions online, in real time. For example, if you create a classification model to test whether a new customer is likely to participate in a sales promotion, you can create an online deployment for the model. Then, you can enter the new customer data to get an immediate prediction.

Supported frameworks

Online deployment is supported for these frameworks:

  • PMML
  • Python Function
  • PyTorch-Onnx
  • Tensorflow
  • Scikit-Learn
  • Spark MLlib
  • SPSS
  • XGBoost

You can create an online deployment from the user interface or programmatically.

To send payload data to an asset that has been deployed online (for example, to classify the data, or make a prediction from the data) you must know the endpoint URL of the deployment. For details, refer to Retrieving the deployment endpoint.

Additionally, you can:

Creating an online deployment from the User Interface

  1. From the deployment space, click the name of the asset that you want to deploy. The details page opens.

  2. Click Create deployment.

  3. Choose Online as the deployment type.

  4. Provide a name and description for the deployment.

  5. If you want to specify a name to be used instead of deployment ID, use the Serving name field.

    • The name is validated to be unique within the namespace.
    • The name must contain only these characters: [a-z,0-9,_] and must be maximum 36 characters long.
    • Serving name works only as part of the prediction URL. In other cases you must still use the deployment ID.

  6. Optional: Select a hardware specification:

    Restriction:

    You cannot create or select custom hardware specifications from the user interface in a deployment space. To learn more about ways to create and select a hardware specification, see Managing hardware specifications for deployments.

  7. Click Create to create the deployment.

Creating an online deployment programmatically

Refer to Machine learning samples and examples for links to sample notebooks. These notebooks demonstrate creating online deployments that use the Watson Machine Learning REST API and Watson Machine Learning Python client library.

Retrieving the online deployment endpoint

You can find the endpoint URL of a deployment in these ways:

  • From the Deployments tab of your space, click your deployment name. A page with deployment details opens. You can find the endpoint there.
  • Using the Watson Machine Learning Python client:
    1. List the deployments by calling the Python client method client.deployments.list()
    2. Find the row with your deployment. The deployment endpoint URL is listed in the url column.

Notes:

  • If you added Serving name to the deployment, two alternative endpoint URLs show on screen; one containing the deployment ID, and the other containing your serving name. You can use either one of these URLs with your deployment.
  • The API Reference tab also shows code snippets in various programming languages that illustrate how to access the deployment.

For more information on endpoints, refer to Endpoint URLs.

Best practice: using an external API gateway to provide a stable endpoint

For any REST endpoints provided by Watson Machine Learning that are being used by other applications or processes in the enterprise, consider using an API gateway. An API gateway provides a stable URL that can be tied to the Watson Machine Learning REST endpoint. When you use as API gateway, you ensure the highest availability by using deployments across two or more Cloud Pak for Data instances. When even a single Cloud Pak for Data instance is used, the API gateway ensures that the consuming applications can continue to use the same URL in all cases, even if a deployment is accidentally deleted and needs to be re-created. Watson Machine Learning does generally provide stable URLs (that is, you can update the version of a model for an online deployment without any downtime or change to the URL). As the result, API gateway reconfiguration is rarely required (for example, if you are moving to a new Cloud Pak for Data instance or adding one for increased availability).

IBM Cloud Pak for Integration includes an API gateway as part of its core functionality. This gateway is generally useful for managing API dependencies, but you can use any other available API gateway.

Testing your online deployment

From the Deployments tab of your space, click your deployment name. A page with deployment details opens. The Test tab provides a place where you can enter data and get a prediction back from the deployed model. If your model has a defined schema, a form shows on screen. In the form, you can enter data in one of these ways: - Enter data directly in the form - Change to the JSON tab and enter your input data as JSON code Regardless of method, the input data must match the schema of the model. Submit the input data and get a score, or prediction, back.

Sample deployment code

When you submit JSON code as the payload, or input data, for a deployment, your input data must match the schema of the model. The 'fields' must match the column headers for the data, and the 'values' must contain the data, in the same order. Use this format:

{"input_data":[{
        "fields": [<field1>, <field2>, ...],
        "values": [[<value1>, <value2>, ...]]
}]}

Refer to this example: 

{"input_data":[{
        "fields": ["PassengerId","Pclass","Name","Sex","Age","SibSp","Parch","Ticket","Fare","Cabin","Embarked"],
        "values": [[1,3,"Braund, Mr. Owen Harris",0,22,1,0,"A/5 21171",7.25,null,"S"]]
}]}

Notes:

  • All strings are enclosed in double quotation marks. The Python notation for dictionaries looks similar, but Python strings in single quotation marks are not accepted in the JSON data.
  • Missing values can be indicated with null.
  • You can specify a hardware specification for an online deployment, for example if you are scaling a deployment.

Preparing payload that matches the schema of an existing model

Refer to this sample code:

model_details = client.repository.get_details("<model_id>")  # retrieves details and includes schema
columns_in_schema = []
for i in range(0, len(model_details['entity']['schemas']['input'][0].get('fields'))):
    columns_in_schema.append(model_details['entity']['schemas']['input'][0].get('fields')[i]['name'])

X = X[columns_in_schema] # where X is a pandas dataframe that contains values to be scored
#(...)
scoring_values = X.values.tolist()
array_of_input_fields = X.columns.tolist()
payload_scoring = {"input_data": [{"fields": [array_of_input_fields],"values": scoring_values}]}

Accessing the online deployment details

To access your online deployment details: From the Deployments tab of your space, click your deployment name and then click the Deployment details tab. The Deployment details tab contains specific information related to the currently opened online deployment and allows for adding a model to the model inventory, to enable activity tracking and model comparison.

Additional information

Refer to Assets in deployment spaces for details on managing deployment jobs, and updating, scaling, or deleting an online deployment.

Parent topic: Managing predictive deployments