Choosing a vector store for a RAG experiment
Before you run a retrieval-augmented generation experiment in AutoAI, you can choose a vector database to store and retrieve data.
For AutoAI RAG experiments, you can choose from two vector store options:
- In-memory Chroma database
- Milvus database
Chroma vector database
An in-memory Chroma vector store stores data on your computer's RAM. The index of vectorized is fast for querying and retrieving data. Due to memory constraints, the in-memory database is built for small to medium-sized datasets.
If you don't specify a connection to a vector store, the vectorized content is saved to the default, in-memory Chroma database. The content does not persist beyond the experiment, so the Chroma option is not a viable production method for deploying a RAG pattern.
Milvus vector database
A Milvus vector store database has a scalable and distributed architecture. It offers high performance in retrieving data when it has optimized indexing and supports many indexing algorithms and distance metrics. It's built for large datasets and is more durable than a Chroma database.
The vectorized content persists for future patterns. For details, see Working with Milvus.
If you are using Milvus in a coded solution, familiarize yourself with the rules for naming conventions and database schema:
- Collection names use this format:
autoai_rag_a0b1c2d3_ymdHMS>where y-year, m-month, d-day, H-hours, M-minute, S-second. - A Milvus database uses this schema:
| Field | Type |
|---|---|
| document_id | VarChar |
| start_index | Int64 |
| sequence_number | Int64 |
| text | VarChar |
| pk | Int64 |
| vector | FloatVector |
Next steps
Follow the steps to code an AutoAI RAG experiment for your use case:
Parent topic: Building RAG experiments with AutoAI