Quick start: Prompt a foundation model with the retrieval-augmented generation pattern

Take this tutorial to learn how to use foundation models in IBM watsonx.ai to generate factually accurate output grounded in information in a knowledge base by applying the retrieval-augmented generation pattern. Foundation models can generate output that is factually inaccurate for a variety of reasons. One way to improve the accuracy of generated output is to provide the needed facts as context in your prompt text. This tutorial uses a sample notebook using the retrieval-augmented generation pattern method to improve the accuracy of the generated output.

Required services
Watson Studio
Watson Machine Learning
watsonx.ai

Your basic workflow includes these tasks:

  1. Open a project. Projects are where you can collaborate with others to work with data.
  2. Add a notebook to your project. You can create your own notebook, or add a sample notebook to your project.
  3. Add and edit code, then run the notebook.
  4. Review the notebook output.

Read about retrieval-augmented generation pattern

You can scale out the technique of including context in your prompts by leveraging information in a knowledge base. The retrieval-augmented generation pattern involves three basic steps:

  • Search for relevant content in your knowledge base
  • Pull the most relevant content into your prompt as context
  • Send the combined prompt text to the model to generate output

Read more about the retrieval-augmented generation pattern

Watch a video about using the retrieval-augmented generation pattern

Watch Video Watch this video to preview the steps in this tutorial. There might be slight differences in the user interface shown in the video. The video is intended to be a companion to the written tutorial.

This video provides a visual method to learn the concepts and tasks in this documentation.


Try a tutorial to prompt a foundation model with the retrieval-augmented generation pattern

In this tutorial, you will complete these tasks:




Tips for completing this tutorial

Here are some tips for successfully completing this tutorial.

Get help in the community

If you need help with this tutorial, you can ask a question or find an answer in the Cloud Pak for Data Community discussion forum.

Set up your browser windows

For the optimal experience completing this tutorial, open Cloud Pak for Data in one browser window, and keep this tutorial page open in another browser window to switch easily between the two applications. Consider arranging the two browser windows side-by-side to make it easier to follow along.

Side-by-side tutorial and UI

Tip: If you encounter a guided tour while completing this tutorial in the user interface, click Maybe later.



Task 1: Open a project

You need a project to store the sample notebook.

Follow these steps to open an existing project or create a new project.

  1. From the Quick navigation, click All projects.
  2. Open an existing project, or create a new project:
    1. Click New project on the Projects page.
    2. Select Create an empty project.
    3. On the Create a project screen, type a name and optional description for the project.
    4. Click Create.

For more information or to watch a video, see Creating a project.

Checkpoint icon Check your progress

The following image shows the empty project. You are now ready to open the Prompt Lab.

Project overview page




Task 2: Add the sample notebook to your project

The sample notebook uses a small knowledge base and a simple search component to demonstrate the basic pattern. The scenario used in this notebook is for a company that sells seeds for planting in a garden. The website for an online seed catalog has many articles to help customers plan their garden and ultimately select which seeds to purchase. The new widget is being added to the website to answer customer questions on the contents of the articles.

  1. From your project, click the Assets tab.

  2. Click New asset > Jupyter notebook editor.

  3. Select the URL page.

  4. For the Name, type Simple introduction to retrieval-augmented generation with watsonx.ai.

  5. For the Notebook URL, copy and paste the following URL:

    
    https://raw.githubusercontent.com/IBMDataScience/sample-notebooks/master/Cloud/IPYNB/Simple-Introduction-to-retrieval-augmented-generation.ipynb
    
  6. Click Create. Wait for the notebook editor to load.

  7. From the menu, click Kernel > Restart & Clear Output, then confirm by clicking Restart and Clear All Outputs to clear the output from the last saved run.

Checkpoint icon Check your progress

The following image shows the notebook open in Edit mode. Now you are ready to set up the prerequisites for running the notebook.

Notebook open in Edit mode




Task 3: Edit the notebook

Before you can run the notebook, you need to set up the environment. Follow these steps to verify the notebook prerequisites:

  1. Scroll to the For IBM watsonx.ai on Cloud Pak for Data section in the notebook to see the prerequisite to run the notebook.

  2. Create a platform API key. You need to pass your credentials using a Cloud Pak for Data platform API key. If you don't already have a saved API key, then follow these steps to create an API key.

    1. Click your profile icon.

    2. Click Profile and settings.

    3. Click API key > Generate new key.

    4. Click Generate.

    5. Click Copy.

    6. Save the API key for future use.

    7. Close the dialog box.

  3. Scroll to the Run the cell to provide the platform API key, url (hostname), and username section:

    1. Click the Run icon Run icon to run the cell.

    2. Paste the API key, and press Enter.

  4. Under Run the cell to set the credentials for IBM watsonx.ai on Cloud Pak for Data, click the Run icon Run icon to run the cell and set the credentials.

Checkpoint icon Check your progress

The following images shows the notebook with the prerequisites completed. Now you are ready to run the notebook and review the output.

Notebook with the prerequisites completed




Task 4: Run the notebook and review the output

The sample notebook includes information about the retrieval-augmented generation and how you can adapt the notebook for your specific use case. Follow these steps to run the notebook and review the output:

  1. Scroll to the Step 2: Create a Knowledge Base section in the notebook:

    1. Click the Run icon Run icon for each of the three cells in that section.

    2. Review the output for the three cells in the section. The code in these cells sets up the knowledge base as a collection of two articles. These articles were written as samples for watsonx.ai, they are not real articles published anywhere else. The authors and publication dates are fictional.

  2. Scroll to the Step 3: Build a simple search component section in the notebook:

    1. Click the Run icon Run icon for each of the two cells in that section.

    2. Review the output for the two cells in the section. The code in these cells builds a simple search component. Many articles that discuss retrieval-augmented generation assume the retrieval component uses a vector database. However, to perform the general retrieval-augmented generation pattern, any search-and-retrieve method that can reliably return relevant content from the knowledge base will do. In this notebook, the search component is a trivial search function that returns the index of one or the other of the two articles in the knowledge base, based on a simple regular expression match.

  3. Scroll to the Step 4: Craft prompt text section in the notebook:

    1. Click the Run icon Run icon for each of the two cells in that section.

    2. Review the output for the two cells in the section. The code in these cells crafts the prompt text. There is no one, best prompt for any given task. However, models that have been instruction-tuned, such as bigscience/mt0-xxl-13b, google/flan-t5-xxl-11b, or google/flan-ul2-20b, can generally perform this task with a sample prompt. Conservative decoding methods tend towards succinct answers. In the prompt, notice two string placeholders (marked with %s) that will be replaced at generation time:

      • The first placeholder will be replaced with the text of the relevant article from the knowledge base
      • The second placeholder will be replaced with the question to be answered
  4. Scroll to the Step 5: Generate output using the foundation models Python library section in the notebook:

    1. Click the Run icon Run icon for each of the three cells in that section.

    2. Review the output for the three cells in the section. The code in these cells generates output by using the Python library. You can prompt foundation models in watsonx.ai programmatically using the Python library. For more information about the library, see the following topics:

  5. Scroll to the Step 6: Pull everything together to perform retrieval-augmented generation section in the notebook:

    1. Click the Run icon Run icon for each of the two cells in that section. This code pulls everything together to perform retrieval-augmented generation.

    2. Review the output for the first cell in the section. The code in this cell sets up the user input elements.

    3. For the second cell in the section, type a question related to tomatoes or cucumbers to see the answer and the source. For example, Do I use mulch with tomatoes?.

    4. Review the answer to your question.

Checkpoint icon Check your progress

The following image shows the completed notebook.

The completed notebook



Next steps

Additional resources

Parent topic: Quick start tutorials