Setting up RAG Chatbot

This section explains how to set up a RAG Chatbot using the existing templates provided in AI Services.

Tip: Ensure all hardware pre-requisites are fully met before deploying the RAG Chatbot. Refer to RAG Chatbot Prerequisites.

1. Create an App Using the RAG Template

Initialize a new application using the built-in template - rag. It generates all essential resources required to configure and run a complete RAG workflow. You can also specify ports for the chatbot using the --params flag (for example: --params "ui.port=3000"). If not provided, the system automatically assigns a free port for the chatbot UI.

Provide a unique name for the application to ensure smooth deployment (replace with <app-name>).

ai-services application create <app-name> -t rag

After the create command completes successfully, the next steps will appear in the output. Alternatively, you can follow the instructions below. Make sure to replace <app-name> with your application name accordingly.

Note: When creating the application in air-gapped environment, use --skip-model-download and --skip-image-download flags. Refer to Configuring Air-gapped Environment.

ai-services application create <app-name> --template rag --skip-model-download --skip-image-download

2. Place the Documents for Ingestion

Add your source documents to the designated ingestion directory path -> /var/lib/ai-services/applications/<app-name>/docs/. These files will be processed and indexed for retrieval by the RAG pipeline.

Verify the pods are in a healthy operational state before starting document ingestion.

ai-services application ps <app-name>

Refer to troubleshooting unhealthy pods, if needed.

3. Start Document Ingestion

Trigger the ingestion process to parse and upload the documents into DB. Once complete, the documents become searchable and ready for retrieval during chat interactions.

ai-services application start <app-name> --pod=<app-name>--ingest-docs

In the event of incomplete/partial document ingestion or an error causing the ingestion process to exit, re-run the command to process the remaining documents.

If you observe a failure in loading the documents into database during ingestion, refer to Troubleshooting Failing Milvus Pod.

4. Access the Chatbot

The chatbot URL is rendered on the terminal once the application is created and can also be viewed by using the ai-service application info <app-name> command. For example,

$ ai-services application info rag-test
Application Name: rag-test
Application Template: RAG
Version: 0.0.1
Info:
-------
Day N:
- Chatbot is available to use at http://10.20.177.252:3000
- Chatbot Backend is available to use at http://10.20.177.252:5000
- If you want to serve any more new documents via this RAG application, add them inside "/var/lib/ai-services/rag-test/docs" directory
- If you want to do the ingestion again, execute below command and wait for the ingestion to be completed before accessing the chatbot to query the new data.
`ai-services application start rag-test --pod=rag-test--ingest-docs`
- In case if you want to clean the documents added to the db, execute below command
`ai-services application start rag-test --pod=rag-test--clean-docs`

Once the chatbot is successfully deployed and configured, it can be accessed and utilized through the designated interface.

Note: For the best experience, please use the latest stable version of Google Chrome, Mozilla Firefox, or Safari to access the Chatbot.