Milvus search integration setup

Edit online

Milvus is a vector database that you can use for handling large-scale datasets. For applications that require real-time search capabilities and numerous concurrent users, you can use Milvus, which has a distributed architecture, high performance, and flexible data model.

You can have only one search integration per environment. When you change the existing search integration to another integration types such as Elasticsearch or Custom service, the settings of the existing search integration are overwritten.

Prerequisites for connecting Milvus to an assistant

You must have an active account on watsonx.data and watsonx.ai.

To create an account on watsonx.data see, watsonx.data home page.

To create an account on watsonx.ai, see watsonx.ai home page.

You must provision a Milvus instance in watsonx.data.

For more information on creating a Milvus instance in watsonx.data, see Adding Milvus service in watsonx.data.

For more information on updating your Milvus authentication credentials, see Updating your Milvus authentication credentials.

For more information on creating an API key, see API keys.

You must configure watsonx.data Milvus in IBM watsonx.ai.

To configure watsonx.data in IBM watsonx.ai, see Configuring watsonx.data in IBM watsonx.ai.

For more details on creating a vector index, see Creating vector index.

For more information about creating collections, see Creating collections.

Connecting Milvus to an assistant

Integrating Milvus in assistant involves three platforms.

AI assistant, where we build the integration.
watsonx.data, where we are provisioning the Milvus instance.
watsonx.ai, where we are building the data source called collections.

Selecting Milvus as a search integration in assistant

After you create a AI assistant instance, you can select Milvus as the search integration by using one of the following procedures:

Selecting Milvus search integration from the Integrations page
1. Go to Home > Integrations.
2. Scroll down to the Extensions section. In the Search tile, click Add to open the Set up a new search integration window.
3. Select Milvus to open the Connect your search provider window.
Selecting Milvus search integration from the Environments page
1. Go to Home > Environments.
2. In the Resolution methods section, go to the Extensions section.
3. Click Add inside the Search tile to open the Set up a new search integration window.
4. Select Milvus to open the Connect your search provider window.
Selecting Milvus search integration through Conversational search
1. In the Home page, scroll down to the Assistant architecture section.
2. In the Conversational search node, click Add to open the Set up a new search integration window.

If Milvus is already added, the Open button is displayed. Else, Add is displayed. To set up custom service on the Draft or Live assistant environment, click Open.

Connecting to watsonx.data to set up Milvus

After you select Milvus as the search integration, use data on your Milvus instance in watsonx.data to set up Milvus on your assistant.

For more information on creating a Milvus instance in watsonx.data, see Prerequisites for Milvus integration.

You can use the Connect Milvus to connect to the Milvus service inside watsonx.data. In the Connect your search provider section, provide the following fields from watsonx.data to enable your assistant to connect to your Milvus instance.

GRPC host and GRPC port

To get the details of GRPC host and GRPC port, see Connection details.

Choose an authentication type
- If you select Basic authentication, you must provide a username and password.
- If you select watsonx.data API key, you must provide the corresponding API key.
- if you select None, you cannot provide any other authentication details.

For more information about credentials, see Getting credentials.

Updating your Milvus authentication credentials

The 2.3 release of IBM watsonx.data impacts existing Milvus authentication types. You must review and update your connection by 10 December 2025 to avoid critical service interruptions.

Step 1: Find your new Milvus username

Your new Milvus username uses this format:

ibmlhapikey_<your watsonx.data username>

Examples


watsonx.data username	Milvus username
`abc@ibm.com`	`ibmlhapikey_abc@ibm.com`
`serviceid-abcdef-123456`	`ibmlhapikey_serviceid-abcdef-123456`

If you do not know your watsonx.data username:

Go to Infrastructure manager in watsonx.data console.
Select your Milvus service to open the Details page.
Click the Access control tab.

Your watsonx.data usernames appear in the first column of the table.

Step 2: Update your Milvus connection in assistant

Update your Milvus credentials for each assistant that uses a Milvus as a source.

Go to Home > Integrations.
Go to the Extensions section and click Open on the search tile.

If prompted, select the environment to update. Repeat these steps for each environment where Milvus search is integrated. If you use both Draft and Live, update each one separately.

Select Milvus.
Record your existing connection information. You need to have a complete record of all your current Milvus settings so that you can enter them again after you update your Milvus credentials.
Go to the Instance tab and click Update details.
Specify your connection details:
- GRPC host and GRPC port.
- Username: Use your new Milvus username from Step 1: Find your new Milvus username.
- Password: Use the same password or API key that you used earlier.

If you are authenticated with a watsonx.data API key, that same key acts as your password.

Click Next.
Click Save and then click Close.
Return to the search tile and click Open.
Select the same environment you chose in step 2.
Click Milvus.
Go to the Conversational Search section.
Reenter the settings you recorded in step 4.
Click Save and then click Close.
Use your assistant chat window to verify that Milvus search results appear correctly.

Ingesting data into the Milvus vector database through watsonx.ai

After you collect information from watsonx.data, you must ingest data into the Milvus database to use in your AI assistant.

In the Milvus window of your assistant, click Next to go to Select data source and provide the following details:

In Database, select your preferred database.
In Choose collection, select your collection. For more information, see Prerequisites for Milvus integration.

In Choose index, select the index.
In Choose embedding_model_id, select your model.

For more information on the supported embedding models, see Embedding models.

Your selected model must align with the model that you used to create your index.

Configuring the result content

After you connect Milvus by selecting the data source, you can configure how the search response displays in the Milvus window of your assistant. In the Configure result content section of Select data source, provide the following fields to map the title, body, and URL from Milvus to the search response in the assistant window:

Title

Search result title. Use the title, name, or similar type of field from the collection as the search result title. Select something for the title or no search result response is displayed in the Facebook and Slack integrations.

Body

Search result description. Use an abstract, summary, or highlight field from the collection as the search result body. Select something for the body or no search result response is displayed in the Facebook and Slack integrations.

URL

This field can be populated with any footer content that you want to include at the end of the search result.

Enabling conversational search in Milvus

After you configure the result content, click Next to go to Conversational search (optional).

Conversational search is available only in the Plus and Enterprise plans of watsonx Assistant.

To activate conversational search, switch the Conversational Search toggle to on. For more information, see Conversational search.

Defining filter expression

Expand the Advanced Milvus settings section to define Filter.

You can define the filter as a string to filter the Milvus search results. For more information, see Milvus-filter-search. For more information on filter expression examples, see Filter expression examples.

Tuning your conversational search

You can tune your conversational search’s tendency to say “I don’t know" and the generated response length.

Use the Message, No results found, and Connectivity issue tabs to customize different messages to share with users based on the successfulness of the search.

Table 1. Search result messages
Tab	Scenario	Example message
Message	Search results are returned	`I found this information that might be helpful:`
No results found	No search results are found	`I searched my knowledge base for information that might address your query, but did not find anything useful to share.`
Connectivity issue	I was unable to complete the search for some reason	`I might have information that could help address your query, but am unable to search my knowledge base at the moment.`

You can skip this step if you do not want to change the Milvus instance details. If you want to change the Milvus instance credentials, click the Instance tab, change the authentication type, or edit the credentials, and then click Save. If you want to change the Milvus instance URL, click the Update button next to the URL that takes you to Step 1 to start the initial setup.

Click Save and then Close to finish the Milvus setup.

Filtering expression examples

The following examples help you to define a filter expression.

Contains

title like "%action%"

This filter expression allows only the search results with title that contains the word action.

Doesn't contain

not (title like "%action%")

This filter expression allows only the search results with title not containing the word action.

Equal

"Understand your most and least successful actions"

This filter expression allows only the search results with title equal to the phrase Understand your most and least successful actions.

Doesn't Equal

"Understand your most and least successful actions"

This filter expression allows only the search results with title not equal to the phrase Understand your most and least successful actions.

Nested filter expression

(title like "%action%") and (url in ["www.url1.com", "www.url2.com"])

This filter expression allows only the search results with the title that contains the word "action" and the url in one of ["www.url1.com", "www.url2.com"].

Configuring your assistant to use Milvus

After you configure the Milvus search integration, you must configure your assistant to use Milvus when the response matches no action. For more information about updating No matches to use search, see Use search when no action matches.

Testing Milvus

You can test search integration with Milvus in actions preview, the preview page, or by using the preview link.

You cannot see proper results in your preview when conversational search is off.

In this example, the user asks, What is an action?.

A text-based reply from the best results in your knowledge base displays when conversational search is on.