Prompting a custom foundation model
After a custom foundation model is installed, registered, and deployed, use the capabilities of watsonx.ai to prompt the model.
When your deployed custom foundation model is available, you can prompt it by using one of these methods:
Prompt a custom model by using Prompt Lab
Open the custom model from the list of available foundation models. You can then work with the model as you do with foundation models that are provided with watsonx.ai.
- Use the Prompt Lab to create prompts and review prompts for your custom foundation model
- Build and save reusable prompts as prompt templates with variables
- Deploy and test prompt templates
Prompt a custom model by using the API
Refer to these examples to code a prompt for the custom foundation model:
Generating text response
The following code sample shows how to generate a text response by using the /ml/v1/deployments/<deployment ID>/text/generation API endpoint:
curl -X POST "https://<cluster_url>/ml/v1/deployments/<your deployment ID>/text/generation?version=2024-01-29" \
-H "Authorization: Bearer $TOKEN" \
-H "content-type: application/json" \
--data '{
"input": "Hello, what is your name",
"parameters": {
"max_new_tokens": 200,
"min_new_tokens": 20
}
}'
Generating stream response
The following code sample shows how to generate a stream response by using the /ml/v1/deployments/<deployment ID>/text/generation_stream API endpoint:
curl -X POST "https://<cluster_url>/ml/v1/deployments/<your deployment ID>/text/generation_stream?version=2024-01-29" \
-H "Authorization: Bearer $TOKEN" \
-H "content-type: application/json" \
--data '{
"input": "Hello, what is your name",
"parameters": {
"max_new_tokens": 200,
"min_new_tokens": 20
}
}'
For details about prompting custom foundation models in a watsonx.ai lightweight engine installation, see Working with watsonx.ai lightweight engine.
Parent topic: Deploying custom foundation models