Coding an AutoAI RAG experiment with text extraction

Step 1: Prepare the prerequisites for preparing data and set up the experiment

Prepare the prerequisites for the experiment.

Before you use the sample code, you must perform the following setup task:

• Contact your Cloud Pak for Data administrator and ask them for your account credentials

1. Install and import the required modules and dependencies. For example:

   pip install 'ibm-watsonx-ai[rag]>=1.1.11'
   pip install "langchain_community>=0.3,<0.4"
   

2. Connect to WML.

   Authenticate the Watson Machine Learning service on IBM Cloud Pak for Data. You need to provide the platform url, your username, and your api_key.

   username = 'PASTE YOUR USERNAME HERE'
   api_key = 'PASTE YOUR API_KEY HERE'
   url = 'PASTE THE PLATFORM URL HERE'
   

3. Use these to initialize the client. For example:

   from ibm_watsonx_ai import APIClient, Credentials
   
   credentials = Credentials(
       username = "username",
       api_key = "***********",
       url = "url",
       instance_id = "openshift",
       version = "5.1"
   )
   
   client = APIClient(credentials)
   

   Alternatively, you can use your username and password to authenticate WML services.

   credentials = Credentials(
       username=***,
       password=***,
       url=***,
       instance_id="openshift",
       version="5.1"
   )
   
   client = APIClient(credentials)
   

4. Create a space for your work. See Creating a space.

5. Set a default space. For example:

   client.set.default_space("<Space GUID>")
   

6. Prepare the grounding documents.

7. Prepare the evaluation data.

benchmarking_data = [
     {
        "question": "What are the two main variants of Granite Code models?",
        "correct_answer": "The two main variants are Granite Code Base and Granite Code Instruct.",
        "correct_answer_document_ids": <TEXT EXTRACTION OUTPUT FILENAME>
     },
     {
        "question": "What is the purpose of Granite Code Instruct models?",
        "correct_answer": "Granite Code Instruct models are finetuned for instruction-following tasks using datasets like CommitPack, OASST, HelpSteer, and synthetic code instruction datasets, aiming to improve reasoning and instruction-following capabilities.",
        "correct_answer_document_ids": <TEXT EXTRACTION OUTPUT FILENAME>
     },
     {
        "question": "What is the licensing model for Granite Code models?",
        "correct_answer": "Granite Code models are released under the Apache 2.0 license, ensuring permissive and enterprise-friendly usage.",
        "correct_answer_document_ids": <TEXT EXTRACTION OUTPUT FILENAME>
     },
]

import os

test_filename = "benchmark.json"

if not os.path.isfile(test_filename):
    with open(test_filename, "w") as json_file:
        json.dump(benchmarking_data, json_file, indent=4)

cos_client.upload_file(test_filename, cos_bucket_name, test_filename)

test_data_reference = DataConnection(
    connection_asset_id=cos_connection_id,
    location=S3Location(bucket=cos_bucket_name, path=test_filename),
)
test_data_reference.set_client(client)

test_data_references = [test_data_reference] 

Step 2: Process input documents with Text Extraction

1. Initialize the text extraction service.

   from ibm_watsonx_ai.foundation_models.extractions import TextExtractions
   
   extraction = TextExtractions(
       credentials=credentials,
       space_id=<Space GUID>,
   )
   

2. Run the text extraction job.

   from ibm_watsonx_ai.metanames import TextExtractionsMetaNames
   
   response = extraction.run_job(
       document_reference=input_data_reference,
       results_reference=result_data_reference,
       steps={
           TextExtractionsMetaNames.OCR: {
               "process_image": True,
               "languages_list": ["en"],
           },
           TextExtractionsMetaNames.TABLE_PROCESSING: {"enabled": True},
       },
       results_format="markdown",
   )
   
   job_id = response['metadata']['id']
   

3. Get job details.

   extraction.get_job_details(job_id)
   

4. When the status is completed, move to the next step.

Step 3: Configure the RAG optimizer

The rag_optimizer object provides a set of methods for working with the AutoAI RAG experiment. In this step, enter the details to define the experiment. These are the available configuration options:

This sample code shows the configuration options for running the experiment with the ibm-watsonx-ai SDK documentation:

from ibm_watsonx_ai.experiment import AutoAI

experiment = AutoAI(credentials, project_id=project_id)

rag_optimizer = experiment.rag_optimizer(
    name='DEMO - AutoAI RAG ibm-watsonx-ai SDK documentation',
    description="AutoAI RAG experiment grounded with the ibm-watsonx-ai SDK documentation",
    max_number_of_rag_patterns=5,
    optimization_metrics=[AutoAI.RAGMetrics.ANSWER_CORRECTNESS]
)

rag_optimizer = experiment.rag_optimizer(
    name='DEMO - AutoAI RAG ibm-watsonx-ai SDK documentation',
    description="AutoAI RAG experiment grounded with the ibm-watsonx-ai SDK documentation",
    embedding_models=["ibm/slate-125m-english-rtrvr"],
    foundation_models=["ibm/granite-13b-chat-v2","mistralai/mixtral-8x7b-instruct-v01"],
    max_number_of_rag_patterns=5,
    optimization_metrics=[AutoAI.RAGMetrics.ANSWER_CORRECTNESS]
)

Step 4: Run the experiment

Run the optimizer to create the RAG patterns by using the specified configuration options. Use the output from text extraction as input in your AutoAI RAG experiment.

In this code sample for running a Chroma experiment, the task is run in interactive mode. You can run the task in the background by changing the background_mode to True.

input_data_references = [result_data_reference]

rag_optimizer.run(
    input_data_references=input_data_references,
    test_data_references=test_data_references,
    background_mode=False
)

Step 5: Review the patterns and select the best one

After the AutoAI RAG experiment completes successfully, you can review the patterns. Use the summary method to list completed patterns and evaluation metrics information in the form of a Pandas DataFrame so you can review the patterns, ranked according to performance against the optimized metric.

summary = rag_optimizer.summary()
summary

For example, pattern results display like this:

best_pattern = rag_optimizer.get_pattern()

payload = {
    client.deployments.ScoringMetaNames.INPUT_DATA: [
        {
            "values": ["How to use new approach of providing credentials to APIClient?"],
        }
    ]
}

best_pattern.query(payload)

According to the document, the new approach to provide credentials to APIClient is by using the Credentials class. Here's an example:


from ibm_watsonx_ai import APIClient
from ibm_watsonx_ai import Credentials

credentials = Credentials(
                   url = "https://us-south.ml.cloud.ibm.com",
                   token = "***********",
                  )

client = APIClient(credentials)


This replaces the old approach of passing a dictionary with credentials to the APIClient constructor.

rag_optimizer.get_inference_notebook(pattern_name='Pattern3')