Generative AI (gen AI) is revolutionizing the ability to quickly access knowledge. Organizations aiming to improve operations are taking note. According to IDC FutureScape: Worldwide Generative Artificial Intelligence 2024 Predictions, IDC, October 2023, “By 2025, two-thirds of businesses will use a combination of gen AI and retrieval augmented generation (RAG) to power domain-specific self-service knowledge discovery, improving decision efficacy by 50%.”
To actualize this, organizations need gen AI capabilities, such as natural language question-answering systems and enterprise search, to support self-service knowledge discovery for employees, customers and more. Ventana Research also noted in their recent 2024 Buyers Guide for GenAI Platforms that documents have become the new data flow for gen AI use cases.   

Through the prompting process, large language models (LLMs) can provide more accurate and specific responses. With IBM®™, companies are starting to do just that. AI developers and model builders are augmenting foundation models with the help of RAG frameworks to give users access to the documents and data they need for informed, data-driven decisions and insights.   

Expanding the capabilities of Prompt Lab  

As mentioned in a previous blog, recently expanded the capabilities of the Prompt Lab interface with a new Chat mode, which adds to the existing Freeform and Structured modes. The Chat mode within the Prompt Lab enables you to converse with the foundation model of your choice.  

This conversational style interaction is compelling but limited by the data that the foundation model has been trained on. For instance, when asked a question the foundation model does not know the answer to, the model might either admit its ignorance or worse, try to ‘invent’ an answer (a phenomenon known as ‘hallucination’).  

One technique to reduce hallucination is ‘model grounding,’ where documents are provided to ‘ground’ the conversation and complete missing knowledge. Because documents are often larger than the model’s context window, a RAG pattern is used to work around this limitation. 

Introducing Chat with Documents  

We’re happy to introduce the new ‘Chat with Documents’ feature in, where AI developers can augment an LLM’s knowledge base by grounding it with documents.  

To help AI developers overcome the possibility of an AI model hallucinating, start by opening the menu from the Prompt Lab input section using the chat mode interface. Then upload a document into the project or select existing files you have previously uploaded.  

Upon returning to the Chat mode, the interface reconfigures to indicate that you are now chatting with the foundation model, augmented by relevant content from the uploaded files. This means you can ask questions and receive answers based on the document content. 

You are not limited to just one document. Uploading files is a shortcut for creating a vector index that captures information about the documents and the way grounding is implemented. You can extend this behavior to a vector store such as Milvus (from IBM®™) or Elasticsearch. Vector stores can hold thousands of documents, greatly expanding grounding fidelity and capability. 

In the video, we demonstrate how to use the new Chat with Documents feature in to ground foundation models such as the IBM® Granite™ model to answer specific business-related questions. For instance, to teach the model about IBM watsonx™ projects, a collaboration feature in the product, we would augment our chat with the entire watsonx documentation to provide more precise answers. 

In addition to concise responses, the foundation model also delivers citations, enabling users to quickly explore source material. When the model meets expectations, you can deploy REST endpoints capturing this behavior, embedding the models into your AI assistants or agents for further testing and scaling of AI applications.  

Ready to learn more?  

To get started chatting with models and see what you can build in, try the new watsonx chat demo

Curious about how IBM ranks among gen AI software providers in Ventana Research’s 2024 Buyers Guide for GenAI Platforms?

Read this report to learn more

More from AI for the Enterprise

Success and recognition of IBM offerings in G2 Summer Reports  

2 min read - IBM offerings were featured in over 1,365 unique G2 reports, earning over 230 Leader badges across various categories.   This recognition is important to showcase our leading products and also to provide the unbiased validation our buyers seek. According to the 2024 G2 Software Buyer Behavior Report, “When researching software, buyers are most likely to trust information from people with similar roles and challenges, and they value transparency above other factors.”  With over 90 million visitors each year and hosting more than 2.6…

Unify and share data across Netezza and for new generative AI applications

3 min read - In today's data and AI-driven world, organizations are generating vast amounts of data from various sources. The ability to extract value from AI initiatives relies heavily on the availability and quality of an enterprise's underlying data. In order to unlock the full potential of data for AI, organizations must be able to effectively navigate their complex IT landscapes across the hybrid cloud.   At this year’s IBM Think conference in Boston, we announced the new capabilities of IBM, an open…

Speed, scale and trustworthy AI on IBM Z with Machine Learning for IBM z/OS v3.2 

4 min read - Recent years have seen a remarkable surge in AI adoption, with businesses doubling down. According to the IBM® Global AI Adoption Index, about 42% of enterprise-scale companies surveyed (> 1,000 employees) report having actively deployed AI in their business. 59% of those companies surveyed that are already exploring or deploying AI say they have accelerated their rollout or investments in the technology. Yet, amidst this surge, navigating the complexities of AI implementation, scalability issues and validating the trustworthiness of AI…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters