Adding vectorized documents for grounding foundation model prompts

Capabilities

Use a vector index asset that is associated with a vector data store to retrieve text embeddings of your grounding documents that are stored in the vector database.

The vector index asset capabilties are based on the following settings you can configure:

Vector store

Select a vector data store that is compatible with document set, such as an in-memory vector store for temporary storage or a third-party vector store for larger, persistent collections.

Grounding document file types and file size limits

The file format and size of your grounding documents depend on the type of vector data store you select.

Embedding model

Select an embedding model that is compatible with the selected vector data store to generate text embeddings for your grounding documents.

Chunking strategy

Select a chunking strategy to determine how to split your grounding documents into smaller segments. Settings include chunk size, chunk overlap for context continuity, and optional PDF page splitting.

For details about how to customize a vector index asset, see Vector index settings.

Ways to work

You can use various methods to create a vector index asset and add grounding documents to the asset in watsonx.ai including:

• From the UI, in the Assets tab in your project or the Prompt Lab. For details, see Creating a vector index from the UI.
• Programmatically by using the vector index transactional APIs. For details, see Creating a vector index with code.

Architecture overview

To provide contextual information to a prompt, first add grounding documents to a vector index asset, and then associate the vector index with a foundation model prompt.

The task of adding grounding documents to an index is depicted in the retrieval-augmented generation diagram by the preprocessing step, where company documents are vectorized.

Workflow

You can use the following high-level steps to create vector index assets and associate them with a prompt,:

1. Select a vector data store in which to store your documents. If you choose to use a third-party vector store, you must set up a connection to the data store before you create the vector index.

2. Create a vector index and add your grounding documents to the vector data store.

   Optional: Choose an embedding model and vectorization settings that you want to use to vectorize your documents when you create the vector index asset.

3. Review how well your vectorized grounding documents answer questions by submitting test queries to the vector index asset. Adjust query settings to improve responses, as applicable.

4. Associate the vector index asset with a foundation model prompt in one of the following ways:

   • From the vector index asset page, click the View vector index info icon at the start of the page to open the About this asset panel, and then click Open in Prompt Lab.
   • From the Prompt Lab in chat mode, click the Grounding with documents icon at the start of the page, and then click Select or create vector index.

5. Save the prompt logic in a notebook to use in your generative AI application.

Learn more

• Supported encoder models
• Chatting with documents and images
• Generate text embeddings programmatically