Generative AI (gen AI) is revolutionizing the ability to quickly access knowledge. Organizations aiming to improve operations are taking note. According to IDC FutureScape: Worldwide Generative Artificial Intelligence 2024 Predictions, IDC, October 2023, “By 2025, two-thirds of businesses will use a combination of gen AI and retrieval augmented generation (RAG) to power domain-specific self-service knowledge discovery, improving decision efficacy by 50%.”
To actualize this, organizations need gen AI capabilities, such as natural language question-answering systems and enterprise search, to support self-service knowledge discovery for employees, customers and more. Ventana Research also noted in their recent 2024 Buyers Guide for GenAI Platforms that documents have become the new data flow for gen AI use cases.
Through the prompting process, large language models (LLMs) can provide more accurate and specific responses. With IBM® watsonx.ai™, companies are starting to do just that. AI developers and model builders are augmenting foundation models with the help of RAG frameworks to give users access to the documents and data they need for informed, data-driven decisions and insights.
As mentioned in a previous blog, watsonx.ai recently expanded the capabilities of the Prompt Lab interface with a new Chat mode, which adds to the existing Freeform and Structured modes. The Chat mode within the Prompt Lab enables you to converse with the foundation model of your choice.
This conversational style interaction is compelling but limited by the data that the foundation model has been trained on. For instance, when asked a question the foundation model does not know the answer to, the model might either admit its ignorance or worse, try to ‘invent’ an answer (a phenomenon known as ‘hallucination’).
One technique to reduce hallucination is ‘model grounding,’ where documents are provided to ‘ground’ the conversation and complete missing knowledge. Because documents are often larger than the model’s context window, a RAG pattern is used to work around this limitation.
We’re happy to introduce the new ‘Chat with Documents’ feature in watsonx.ai, where AI developers can augment an LLM’s knowledge base by grounding it with documents.
To help AI developers overcome the possibility of an AI model hallucinating, start by opening the menu from the Prompt Lab input section using the chat mode interface. Then upload a document into the project or select existing files you have previously uploaded.
Upon returning to the Chat mode, the interface reconfigures to indicate that you are now chatting with the foundation model, augmented by relevant content from the uploaded files. This means you can ask questions and receive answers based on the document content.
You are not limited to just one document. Uploading files is a shortcut for creating a vector index that captures information about the documents and the way grounding is implemented. You can extend this behavior to a vector store such as Milvus (from IBM® watsonx.data™) or Elasticsearch. Vector stores can hold thousands of documents, greatly expanding grounding fidelity and capability.
In the video, we demonstrate how to use the new Chat with Documents feature in watsonx.ai to ground foundation models such as the IBM® Granite™ model to answer specific business-related questions. For instance, to teach the model about IBM watsonx™ projects, a collaboration feature in the product, we would augment our chat with the entire watsonx documentation to provide more precise answers.
In addition to concise responses, the foundation model also delivers citations, enabling users to quickly explore source material. When the model meets expectations, you can deploy REST endpoints capturing this behavior, embedding the models into your AI assistants or agents for further testing and scaling of AI applications.
To get started chatting with models and see what you can build in watsonx.ai, try the new watsonx chat demo.
Curious about how IBM ranks among gen AI software providers in Ventana Research’s 2024 Buyers Guide for GenAI Platforms?