Choosing an AI model

Choosing the right AI model helps your agent understand user intent, reason through tasks, and generate reliable responses. Your choice directly affects accuracy, performance, and cost.

Before you begin

Before you select a model:

  • Identify the use case that your agent must support, including accuracy, latency, and cost expectations.

  • Review Considerations for choosing an AI model to understand how model characteristics affect performance.

  • Check Available AI models to confirm which models are supported in your region and cloud provider.

  • Make sure that you have an existing agent or are ready to create one.

Model availability and status

The AI model drop-down list includes models that are:

  • Provided by watsonx Orchestrate by default

  • Recommended for your region

  • Added through ADK

  • Connected through AI gateway

Select View all foundation models to see other supported models.

Important: When you use a third-party model, the model is provided under its own license and is not governed by IBM. Review the license terms before use. Read the terms.

Each model displays one of the following status tags:

Default: The model that watsonx Orchestrate uses automatically when no model is selected for an agent.
Recommended: Model suggested by watsonx Orchestrate
Deprecated: The model is supported now but might be withdrawn in a future release. This tag is displayed with a warning icon.
Not available: The model is no longer available for use.

Order of models in the list

The drop-down list displays models in this order:

  1. Selected model is displayed at the top of the drop-down
  2. The default model is displayed as the second if not already selected
  3. The models that are listed as recommended in the ADK (if it is available in the data-center)
  4. The external models that are added by using AI gateway are displayed in alphabetical order

If a model is not available, an error appears in the Pre-deployment summary. You cannot deploy your agent until you select a supported model.

Select a model

To assign an AI model to your agent:

  1. Create a new agent or open an existing one.

  2. Locate the AI model drop-down list in the agent settings.

  3. Review the available models and select the model that best fits your agent’s task complexity, input type, and performance requirements.

After selecting a model, continue configuring your agent by customizing prompts or adding knowledge.

What to do next