Creating Online Deployments

Create an online (also called Web service) deployment to load a model or Python code when the deployment is created to generate predictions online, in real time. For example, if you create a classification model to test whether a new customer is likely to participate in a sales promotion, you can create an online deployment for the model. Then, you can enter the new customer data to get an immediate prediction.

Note: Models based on the Decision Optimization framework, Python scripts, and R scripts don't support online deployment.

Creating an online deployment

1. From the deployment space, click the name of the saved model that you want to deploy. The model detail page opens.
2. Click Create deployment.
3. Choose Online as the deployment type.
4. Provide a name and description for the deployment.
5. If you want to specify a name to be used instead of deployment ID, use the Serving name field.
   • The name is validated to be unique within the namespace.
   • The name must contain only these characters: [a-z,0-9,_] and must be maximum 36 characters long.
   • Serving name works only as part of the prediction URL. In other cases you must still use the deployment ID.
6. Click Create to create the deployment.
7. When the status in the Deployment tab shows as Deployed, click the name of the deployment to view the deployment details.

Assigning compute resources with a hardware specification

A hardware specification controls the resources that are allotted for the deployment. The specifications available depend on the deployment type. You can assign a hardware specification when you create the deployment, or edit the specification from the model details.

Hardware specifications for Python functions, Tensorflow models, and models that use custom software specifications

Hardware specifications for SPSS deployments

Working with an online deployment

Online deployment has three tabs:

• API Reference displays the API endpoint that you need to access the deployment programmatically. You need the endpoint to use the deployment in an application. Additionally, the API Reference tab shows code snippets in various programming languages that illustrate how to access the deployment. Note: If you added Serving name to the deployment, two alternative endpoint URLs show on screen; one containing the deployment ID, and the other containing your serving name. You can use either one of these URLs with your deployment.
• Test provides a place where you can enter data and get a prediction back from the deployed model. If your model has a defined schema, a form shows on screen. In the form, you can enter data in one of these ways:
  • Enter data directly in the form
  • Change to the JSON tab and enter your input data as JSON code Regardless of method, the input data must match the schema of the model. Submit the input data and get a score, or prediction, back.
• Deployment details contains specific information related to the currently opened online deployment and allows for adding a model to the model inventory, to enable activity tracking and model comparison.

Sample deployment code

When you submit JSON code as the payload, or input data, for a deployment, your input data must match the schema of the model. The 'fields' must match the column headers for the data, and the 'values' must contain the data, in the same order. Use this format:

{"input_data":[{
        "fields": [<field1>, <field2>, ...],
        "values": [[<value1>, <value2>, ...]]
}]}

Refer to this example:

{"input_data":[{
        "fields": ["PassengerId","Pclass","Name","Sex","Age","SibSp","Parch","Ticket","Fare","Cabin","Embarked"],
        "values": [[1,3,"Braund, Mr. Owen Harris",0,22,1,0,"A/5 21171",7.25,null,"S"]]
}]}

Notes:

• All strings are enclosed in double quotation marks. The Python notation for dictionaries looks similar, but Python strings in single quotation marks are not accepted in the JSON data.
• Missing values can be indicated with null.
• You can specify a hardware specification for an online deployment, for example if you are scaling a deployment.

Preparing payload that matches the schema of an existing model

Refer to this sample code:

model_details = client.repository.get_details("<model_id>")  # retrieves details and includes schema
columns_in_schema = []
for i in range(0, len(model_details['entity']['schemas']['input'][0].get('fields'))):
    columns_in_schema.append(model_details['entity']['schemas']['input'][0].get('fields')[i]['name'])

X = X[columns_in_schema] # where X is a pandas dataframe that contains values to be scored
#(...)
scoring_values = X.values.tolist()
array_of_input_fields = X.columns.tolist()
payload_scoring = {"input_data": [{"fields": [array_of_input_fields],"values": scoring_values}]}

Creating online deployments programmatically

Refer to Machine learning samples and examples for links to sample notebooks. These notebooks demonstrate creating online deployments that use the Watson Machine Learning REST API and Watson Machine Learning Python client library.

Additional information

Refer to Managing deployed assets for details on scaling, updating or deleting an online deployment.

Parent topic: Deploying assets