Coding an AutoAI RAG experiment with text extraction

Grounding documents

Prepare and connect to the grounding documents that you will use to run the AutoAI RAG experiment with the text extraction service.

1. Create a connection to Cloud Object Storage and fetch the ID.

   datasource_name = 'bluemixcloudobjectstorage'
   
   conn_meta_props= {
       client.connections.ConfigurationMetaNames.NAME: f"Connection to input data - {datasource_name} ",
       client.connections.ConfigurationMetaNames.DATASOURCE_TYPE: client.connections.get_datasource_type_id_by_name(datasource_name),
       client.connections.ConfigurationMetaNames.DESCRIPTION: "ibm-watsonx-ai SDK documentation",
       client.connections.ConfigurationMetaNames.PROPERTIES: {
           'bucket': <BUCKET_NAME>,
           'access_key': <ACCESS_KEY>,
           'secret_key': <SECRET_ACCESS_KEY>,
           'iam_url': 'https://iam.cloud.ibm.com/identity/token',
           'url': <ENDPOINT_URL>
       }
   }
   
   conn_details = client.connections.create(meta_props=conn_meta_props)
   cos_connection_id = client.connections.get_id(conn_details)
   

2. Prepare two connection assets, one for input and one for the text extraction service output.

   text_extraction_input_filename = <TEXT EXTRACTION INPUT FILENAME>
   text_extraction_result_filename = <TEXT EXTRACTION RESULT FILENAME>
   

   from ibm_watsonx_ai.helpers import DataConnection, S3Location
   
   input_data_reference = DataConnection(
       connection_asset_id=cos_connection_id,
       location=S3Location(
           bucket=<BUCKET_NAME>,
           path=text_extraction_input_filename
       ),
   )
   input_data_reference.set_client(client)
   
   result_data_reference = DataConnection(
       connection_asset_id=cos_connection_id,
       location=S3Location(
           bucket=<BUCKET_NAME>,
           path=text_extraction_result_filename
       )
   )
   result_data_reference.set_client(client)
   

Evaluation data

For evalutation data input:

• Data must be in the JSON format with a fixed schema with these fields: question, correct_answer, correct_answer_document_ids
• correct_answer_document_ids must refer to the text extraction service output file

benchmarking_data = [
    {
        "question": "What are the two main variants of Granite Code models?",
        "correct_answer": "The two main variants are Granite Code Base and Granite Code Instruct.",
        "correct_answer_document_ids": [text_extraction_result_filename],
    },
    {
        "question": "What is the purpose of Granite Code Instruct models?",
        "correct_answer": "Granite Code Instruct models are finetuned for instruction-following tasks using datasets like CommitPack, OASST, HelpSteer, and synthetic code instruction datasets, aiming to improve reasoning and instruction-following capabilities.",
        "correct_answer_document_ids": [text_extraction_result_filename],
    },
    {
        "question": "What is the licensing model for Granite Code models?",
        "correct_answer": "Granite Code models are released under the Apache 2.0 license, ensuring permissive and enterprise-friendly usage.",
        "correct_answer_document_ids": [text_extraction_result_filename],
    },
]

import os, json

test_filename = "benchmark.json"

if not os.path.isfile(test_filename):
    with open(test_filename, "w") as json_file:
        json.dump(benchmarking_data, json_file, indent=4)

test_data_reference = DataConnection(
    connection_asset_id=cos_connection_id,
    location=S3Location(bucket=<BUCKET_NAME>, path=test_filename),
)
test_data_reference.set_client(client)

# Upload file to the COS bucket
test_data_reference.write(test_filename)

test_data_references = [test_data_reference]

Select a pattern to test locally

The next step is to select a pattern and test it locally.

best_pattern = rag_optimizer.get_pattern()

Query the RAG pattern locally.

from ibm_watsonx_ai.deployments import RuntimeContext

runtime_context = RuntimeContext(api_client=client)
inference_service_function = best_pattern.inference_service(runtime_context)[0]

question = "How to use new approach of providing credentials to APIClient?"

context = RuntimeContext(
    api_client=client,
    request_payload_json={"messages": [{"role": "user", "content": question}]},
)
print(inference_service_function(context)["body"]["choices"][0]["message"]["content"])

The model's response looks like this:

According to the document, the new approach to provide credentials to APIClient is by using the Credentials class. Here's an example:


from ibm_watsonx_ai import APIClient
from ibm_watsonx_ai import Credentials

credentials = Credentials(
                   url = "https://us-south.ml.cloud.ibm.com",
                   token = "***********",
                  )

client = APIClient(credentials)


This replaces the old approach of passing a dictionary with credentials to the APIClient constructor.

Get an inference and indexing notebook

To download a specified inference notebook, use the get_inference_notebook(). If you leave pattern_name empty, the method downloads the notebook of the best computed pattern.

rag_optimizer.get_inference_notebook(pattern_name='Pattern3')

For more information and code samples, refer to the AutoAI RAG with watsonx Text Extraction service notebook.