What's new and changed in watsonx.ai

You can now use the following foundation models for inferencing from the Prompt Lab and the API:

• ibm-defense-4-0-small
• devstral-small-2512
• devstral-medium-2512
• ministral-14b-instruct-2512
• mistral-large-2512

For details, see Supported foundation models.

For details, see Inferencing deployed custom foundation models.

For details, see Adding generative chat function to your applications with the chat API.

For details, see Saving a RAG pattern as a deployable AI service.

For details, see Automating a RAG pattern with AutoAI.

For details, see Foundation models supported for AutoAI experiments.

You can now use the following foundation models for inferencing from the Prompt Lab and the API:

• granite-4-h-tiny
• granite-docling-258M
• ibm-defense-4-0-micro

For details, see Supported foundation models.

You can now securely configure and interact with foundation models from multiple providers with the model gateway by using the API. In addition, you can manage the model gateway with integrated load-balancing, access policies, and rate limits.

For details, see Model gateway.

You can now add custom embedding and reranking models to watsonx.ai and use them to capture semantic meaning and refine retrieved results.

All custom foundation models now use the vLLM inferencing server. If your deployed models use the TGIS inferencing server, you might have to migrate them.

For details, see Requirements for deploying custom foundation models.

You can classify your document with the classification API into one of several supported common document types without running a longer extraction task. By pre-processing the document, you can then customize your text extraction request to efficiently extract relevant details from your document.

For details about the text classification API and the document understanding library, see Text classification [Text classification] and Understanding documents.

For details, see Create a vector index programmatically.

For details, see Creating jobs to generate unstructured synthetic data.

• Use semantic chunking in AutoAI for RAG experiments

  You can now use the semantic chunking method to break down documents in an AutoAI for RAG experiment. Semantic chunking splits documents based on meaning, making it well-suited for complex or unstructured data.

  For details, see Customizing chunking settings.

• Use chat API models in AutoAI for RAG experiments

  You can now use chat API models in AutoAI for RAG experiments, instead of prompt template models. These models must have chat capabilities to work in AutoAI for RAG experiments.

  For details, see Supported foundation models.

• Auto-deploy top pattern in AutoAI for RAG experiments

  You can now enable automatic deployment of the top-performing pattern after an AutoAI for RAG experiment completes. You can turn on auto-deployment when you set up the experiment. Auto-deployment helps reduce manual steps and further automates the experiment workflow.

  For details, see Creating the AutoAI RAG experiment.

• Use multiple vector indexes in AutoAI for RAG experiments

  You can now select up to 20 vector indexes for your document collection in an AutoAI for RAG experiment. During experiment setup, when you add document and evaluation sources, choose Knowledge bases and then select up to 20 connections. You can then define details for each connection, such as index name and embedding models. Using multiple indexes gives you more flexibility and can improve the quality and performance of your experiments.

  For details, see Using vector store knowledge bases in AutoAI RAG experiments.

• Use SQL database schemas in AutoAI for RAG experiments

  You can now choose an SQL database schema as a knowledge base in an AutoAI for RAG experiment. You can use SQL connections such as Db2, PostgreSQL, and MySQL. When using SQL sources, chunking settings are disabled, and only answer correctness metrics are available for optimization. With an SQL RAG, structured data can be retrieved directly from the relational database, which can improve answer accuracy and relevance when compared with document-based sources.

  For details, see Using SQL knowledge bases in AutoAI RAG experiments.