Choosing a foundation model in watsonx.ai
There are many factors to consider when you choose a foundation model to use for inferencing from a generative AI project.
For example, for a solution that summarizes call center problem reports, you might want a foundation model with these characteristics:
- Scores well on benchmarks for summarization tasks
- Handles large amounts of text, which means a large context window length
- Can interpret images of damaged items, so accepts inputs in both text and image modalities
Determine which factors are most important for you and your organization.
- Tasks the model can do
- Multimodal foundation models
- Languages supported
- Tuning options for customizing the model
- License and IP indemnity terms
- Model attributes, such as size, architecture, and context window length
After you have a short list of models that best fit your needs, you can test the models to see which ones consistently return the results you want.
Foundation models that support your use case
To get started, find foundation models that can do the type of task that you want to complete.
The following table shows the types of tasks that the foundation models in IBM watsonx.ai support. A checkmark (✓) indicates that the task that is named in the column header is supported by the foundation model. For some of the tasks, you can click a link to go to a sample prompt for the task.
| Model | Conversation from Chat API |
Tool interaction from Chat API |
Retrieval-augmented generation (RAG) | Samples |
|---|---|---|---|---|
| ibm-defense-4-0-small | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Tool-calling sample • Chat API |
| ibm-defense-4-0-micro | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Tool-calling sample • Chat API |
| ibm-defense-3-3-8b-instruct | ✓ | ✓ | ✓ • RAG from Prompt Lab |
• Chat API |
| granite-4-h-tiny | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Tool-calling sample • Chat API |
| granite-4-h-small | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Chat API |
| granite-docling-258M | ✓ | ✓ RAG from Prompt Lab |
• Chat with image example • Chat API |
|
| granite-3-3-8b-instruct | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Q&A | |
| granite-3-2-8b-instruct | ✓ | ✓ | ✓ RAG from Prompt Lab |
• Chat API |
| granite-3-2b-instruct | ✓ | • Code • Chat API |
||
| granite-3-8b-instruct | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Code • Chat API • Tool-calling |
| granite-3b-code-instruct | • Code | |||
| granite-4-1b-speech | ✓ | ✓ | ||
| granite-8b-code-instruct | • Code | |||
| granite-20b-code-instruct | ✓ | • Code • Chat API |
||
| granite-20b-code-base-schema-linking | • Code | |||
| granite-20b-code-base-sql-gen | • Code | |||
| granite-34b-code-instruct | ✓ | • Code • Chat API |
||
| granite-guardian-3-2b | ✓ RAG from Prompt Lab |
• Classification | ||
| granite-guardian-3-8b | ✓ RAG from Prompt Lab |
• Classification | ||
| granite-guardian-3-2-5b | ✓ | ✓ RAG from Prompt Lab |
• Classification • Chat API |
|
| granite-vision-3-3-2b | ✓ | • Chat with image example • Chat API |
||
| allam-1-13b-instruct | • Classification • Translation |
|||
| codestral-2501 | • Code | |||
| codestral-2508 | • Code | |||
| devstral-small-2512 | ✓ | ✓ | • Classification • Q&A • Summarization |
|
| devstral-medium-2507 | ✓ | ✓ | • Classification • Q&A • Summarization |
|
| devstral-medium-2512 | ✓ | ✓ | • Classification • Q&A • Summarization |
|
| gpt-oss-20b | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Chat API |
| gpt-oss-120b | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Chat API |
| llama-4-maverick-17b-128e-instruct-fp8 | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Dialog • Chat • Chat API |
|
| llama-4-scout-17b-16e-instruct-int4 | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Dialog • Chat • Chat API |
|
| llama-3-3-70b-instruct | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Sample chat • Tool-calling sample • Chat API |
| llama-3-2-1b-instruct | ✓ | ✓ RAG from Prompt Lab |
• Code • Dialog • Tool-calling |
|
| llama-3-2-3b-instruct | ✓ | ✓ RAG from Prompt Lab |
• Code • Dialog • Tool-calling |
|
| llama-3-2-11b-vision-instruct | ✓ | ✓ | ✓ RAG from Prompt Lab |
• Chat with image example • Chat API • Tool-calling sample |
| llama-3-2-90b-vision-instruct | ✓ | ✓ | ✓ RAG from Prompt Lab |
• Chat with image example • Chat API • Tool-calling sample |
| llama-3-1-8b | ✓ RAG from Prompt Lab |
• Dialog | ||
| llama-guard-3-11b-vision | ✓ | ✓ RAG from Prompt Lab |
• Classification • Chat with image example • Chat API |
|
| llama-3-1-8b-instruct | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Dialog • Chat API • Tool-calling |
| llama-3-1-70b-instruct | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Dialog • Chat API • Tool-calling |
| magistral-small-2509 | ✓ | ✓ | ✓ | |
| magistral-medium-2509 | ✓ | ✓ | ✓ | |
| ministral-3-8B-instruct-2512 | ✓ | ✓ | ✓ | |
| ministral-3B-instruct-2512 | ✓ | ✓ | ✓ • RAG from Prompt Lab |
|
| ministral-8b-instruct | • Classification • Extraction • Summarization • Translation |
|||
| ministral-14b-instruct-2512 | ✓ | ✓ | ✓ RAG from Prompt Lab |
• Chat API • Chat with image example |
| mistral-large-2512 | ✓ | ✓ | ✓ • RAG from Prompt Lab |
• Classification • Extraction • Summarization • Code • Translation • Chat API • Tool-calling • Chat with image example |
| mistral-large-instruct-2411 | ✓ | ✓ RAG from Prompt Lab |
• Classification • Extraction • Summarization • Code • Translation |
|
| mistral-small-3-2-24b-instruct-2506 | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Chat API • Chat with image example |
| mistral-small-3-1-24b-instruct-2503 | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Chat API | |
| nvidia-nemotron-nano-12b-v2-vl-fp8 | ✓ | ✓ | ✓ | |
| nvidia-nemotron-3-nano-30b-a3b-fp8 | ✓ | ✓ | ✓ | |
| pixtral-12b | ✓ | ✓ RAG from Prompt Lab |
• Classification • Extraction •Summarization • Chat with image example |
|
| pixtral-large-instruct-2411 | ✓ | • Chat with image example | ||
| voxtral-small-24b-2507 | ✓ | ✓ | ✓ • RAG from Prompt Lab • RAG from AutoAI |
• Chat API • Classification • Extraction • Generation • Summarization • Translation |
| voxtral-mini-2507 | ✓ | ✓ |
- To review various prompt samples that are grouped by task type, see Sample prompts.
- To determine how well a foundation model can perform certain tasks, see Foundation model benchmarks.
Multimodal foundation models
Multimodal foundation models are capable of processing and integrating information from many modalities or types of data. These modalities can include text, images, audio, video, and other forms of sensory input.
The multimodal foundation models that are available from watsonx.ai can do the following types of tasks:
- Image-to-text generation
- Useful for visual question answering, interpretation of charts and graphs, captioning of images, and more.
- Audio-to-text generation
- Useful for speech recognition, transcription of spoken content, voice command understanding, meeting note generation, accessibility support, and more.
The following table lists the available foundation models that support modalities other than text-in and text-out.
| Model | Input modalities | Output modalities |
|---|---|---|
| granite-vision-3-2-2b | image, text | text |
| llama-4-maverick-17b-128e-instruct-fp8 | image, text | text |
| llama-3-2-11b-vision-instruct | image, text | text |
| llama-3-2-90b-vision-instruct | image, text | text |
| llama-guard-3-11b-vision | image, text | text |
| ministral-8b-instruct-2512 | image, text | text |
| ministral-14b-instruct-2512 | image, text | text |
| mistral-large-2512 | image, text | text |
| mistral-small-3-2-24b-instruct-2506 | image, text | text |
| pixtral-12b | image, text | text |
| voxtral-small-24b-2507 | audio, text | text |
Foundation models that support your language
Many foundation models work well only in English. But some model creators include multiple languages in the pretraining data sets to fine-tune their model on tasks in different languages, and to test their model's performance in multiple languages. If you plan to build a solution for a global audience or a solution that does translation tasks, look for models that were created with multilingual support in mind.
The following table lists natural languages that are supported in addition to English by foundation models in watsonx.ai. For more information about the languages that are supported for multilingual foundation models, see the model card for the foundation model.
| Model | Languages other than English |
|---|---|
| Granite 4.0 (granite-4-h-small, granite-4-h-micro, granite-4-h-tiny ) | German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. You can fine tune these Granite models for languages beyond these 12 languages |
| Granite Instruct 3.3 (granite-3-3-2b-instruct, granite-3-3-8b-instruct) | German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. You can fine tune these Granite models for languages beyond these 12 languages. |
| Granite Vision (granite-vision-3-3-2b, granite-vision-3-2-2b) | German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese |
| IBM Defense 4.0 | German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. |
| allam-1-13b-instruct | Arabic |
| gpt-oss-120b | Multilingual |
| Llama 4 (llama-4-maverick-17b-128e-instruct-fp8) | Arabic, French, German, Hindi, Indonesian, Italian, Portuguese, Spanish, Tagalog, Thai, and Vietnamese. |
| Llama 3.3 (llama-3-3-70b-instruct) | German, French, Italian, Portuguese, Hindi, Spanish, and Thai |
| Llama 3.2 (llama-3-2-1b-instruct, llama-3-2-3b-instruct. Also llama-3-2-11b-vision-instruct, llama-3-2-90b-vision-instruct, and llama-guard-3-11b-vision with text-only inputs) | English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai |
| Llama 3.1 (llama-3-1-8b-instruct, llama-3-1-70b-instruct) | English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai |
| Ministral 3 (ministral-3b-instruct-2512, ministral-8b-instruct-2512, ministral-14b-instruct-2512) | French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic, and dozens of other languages. |
| ministral-8b-instruct | Multilingual (See model card) |
| mistral-large-2512 | French, German, Italian, Spanish, Chinese, Japanese, Korean, Portuguese, Dutch, Polish, and dozens of other languages. |
| Mistral Medium (mistral-medium-2505 , mistral-medium-2508) | Multilingual (See model card) |
| mistral-small-3-2-24b-instruct-2506 | French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Swedish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, Farsi. |
| voxtral-small-24b-2507 | Spanish, French, Portuguese, Hindi, German, Dutch, Italian. |
Foundation models that you can tune
You can run tuning experiments that change the parameter weights of the underlying foundation model to guide the model to generate output that is optimized for a task.
The following table shows foundation models that you can tune by using various fine tuning methods in IBM watsonx.ai. A checkmark (✓) indicates that fine tuning is supported by the foundation model.
| Model name | Full fine tuning | LoRA fine tuning | QLoRA fine tuning |
|---|---|---|---|
| allam-1-13b-instruct | ✓ | ||
| granite-3b-code-instruct | ✓ | ||
| granite-8b-code-instruct | ✓ | ||
| granite-20b-code-instruct | ✓ | ||
| granite-3-1-8b-base | ✓ | ✓ | |
| llama-3-1-8b | ✓ | ✓ | |
| llama-3-1-8b-instruct | ✓ | ||
| llama-3-1-70b | ✓ | ||
| llama-3-1-70b-gptq | ✓ |
For more information, see Choosing a model to tune.
Model types and IP indemnification
Review the intellectual property indemnification policy for the foundation model that you want to use. Some third-party foundation model providers require you to exempt them from liability for any IP infringement that might result from the use of their AI models.
IBM-developed foundation models that are available from watsonx.ai have standard intellectual property protection, similar to what IBM provides for hardware and software products.
IBM extends its standard intellectual property indemnification to the output that is generated by covered models. Covered Models include IBM-developed and some third-party foundation models that are available from watsonx.ai. Third-Party Covered Models are identified in table 4.
The following table describes the different foundation model types and their indemnification policies. See the reference materials for full details.
| Foundation model type | Indemnification policy | Foundation models | Details | Reference materials |
|---|---|---|---|---|
| IBM Covered Model | Uncapped IBM indemnification | • IBM Granite • IBM Slate |
IBM-developed foundation models that are available from watsonx.ai. | License information |
| Third-Party Covered Model | Capped IBM indemnification | Mistral Commercial Models | Third-party covered models that are available from watsonx.ai. | License information |
| Non-IBM Product | No IBM indemnification | Various | Third-party models that are available from watsonx.ai and are subject to their respective license terms, including associated obligations and restrictions. | See model information. |
| Custom Model | No IBM indemnification | Various | Foundation models that you import to use in watsonx.ai are Client content. | Client is solely responsible for the selection and use of the model and output and compliance with third-party license terms, obligations, and restrictions. |
For more information about third-party model license terms, see Third-party foundation models.
More considerations for choosing a model
| Model attribute | Considerations |
|---|---|
| Context length | Sometimes called context window length, context window, or maximum sequence length, context length is the maximum allowed value for the number of tokens in the input prompt plus the number of tokens in the generated output. When you generate output with models in watsonx.ai, the number of tokens in the generated output is limited by the Max tokens parameter. |
| Fine-tuned | After a foundation model is pretrained, many foundation models are fine tuned for specific tasks, such as classification, information extraction, summarization, responding to instructions, answering questions, or participating in a back-and-forth dialog chat. A model that undergoes fine tuning on tasks similar to your planned use typically do better with zero-shot prompts than models that are not fine tuned in a way that fits your use case. One way to improve results for a fine-tuned model is to structure your prompt in the same format as prompts in the data sets that were used to fine tune that model. |
| Instruction-tuned | Instruction-tuned means that the model was fine tuned with prompts that include an instruction. When a model is instruction tuned, it typically responds well to prompts that have an instruction even if those prompts don't have examples. |
| IP indemnity | In addition to license terms, review the intellectual property indemnification policy for the model. For more information, see Model types and IP indemnification. |
| License | In general, each foundation model comes with a different license that limits how the model can be used. Review model licenses to make sure that you can use a model for your planned solution. |
| Model architecture | The architecture of the model influences how the model behaves. A transformer-based model typically has one of the following architectures: • Encoder-only: Understands input text at the sentence level by transforming input sequences into representational vectors called embeddings. Common tasks for encoder-only models include classification and entity extraction. • Decoder-only: Generates output text word-by-word by inference from the input sequence. Common tasks for decoder-only models include generating text and answering questions. • Encoder-decoder: Both understands input text and generates output text based on the input text. Common tasks for encoder-decoder models include translation and summarization. |
| Supported programming languages | Not all foundation models work well for programming use cases. If you are planning to create a solution that summarizes, converts, generates, or otherwise processes code, review which programming languages were included in a model's pretraining data sets and fine-tuning activities to determine whether that model is a fit for your use case. |