Monitoring IBM watsonx models

Prerequisites

Make sure that the following prerequisites are met:

• Python 3.10 or later. Python 3.13 is not supported due to dependency compatibility issues. Use Python 3.12 or
• IBM watsonx credentials (project ID and API key from IBM watsonx)
• An Instana account
• Review of Getting Started with Generative AI Observability on basic setup, agent, and agnetless modes

Instrumenting your IBM watsonx application

1. Install the required packages.

   pip install traceloop-sdk langchain-ibm ibm_watsonx_ai ibm-watson-machine-learning

   Note: If you encounter a compilation error with pandas (such as error: too few arguments to function '_PyLong_AsByteArray'), this indicates you're using Python 3.13, which is not yet supported. For more information, see Python version is not compatible.

2. Set environment variables.

   Finding your IBM watsonx credentials

   You can find your credentials in the IBM watsonx.ai console:

   Quick method (Discover section):

   • On the IBM watsonx.ai home page, check if there's a Discover section with developer information
   • If available, this section displays your Project ID, watsonx.ai URL, and API key all in one place
   • Copy these values directly

   Alternative method:

   • API Key: Navigate to IAM > API Keys section in IBM Cloud console
     • Create a new API key if you don't have one
     • Copy the API key value (you won't be able to see it again after creation)
   • Project ID:
     • Go to your Projects in watsonx.ai
     • Select or create a project
     • Navigate to the Manage tab
     • Find and copy the Project ID
   • watsonx.ai URL: Typically something like https://us-south.ml.cloud.ibm.com (but varies for your specific region)

   Setting the environment variables

   Export your IBM watsonx credentials:

   export WATSONX_URL="<your-watsonx-url>"
   export WATSONX_PROJECT_ID="<your-project-id>"
   export WATSONX_API_KEY="<your-api-key>"

   Alternatively, if you prefer to use a .env file to store your credentials, install python-dotenv.

   pip install python-dotenv

   Create a .env file in your project directory:

   WATSONX_URL=<your-watsonx-url>
   WATSONX_PROJECT_ID=<your-project-id>
   WATSONX_API_KEY=<your-api-key>

   Then add the following import at the top of your Python file:

   from dotenv import load_dotenv
   load_dotenv()

3. Create your IBM watsonx application. Create a Python file with the following code:

   import os
   import random
   from langchain_ibm import WatsonxLLM
   from pydantic import SecretStr
   from traceloop.sdk import Traceloop
   from traceloop.sdk.decorators import workflow
   
   Traceloop.init(app_name="watsonx_chat_service", disable_batch=True)
   
   @workflow(name="watsonx_llm_query")  # type: ignore[arg-type]
   def generate_response(model_id: str, question: str) -> str:
       print(f"  → Initializing {model_id}...")
       llm = WatsonxLLM(
           model_id=model_id,
           url=SecretStr(os.getenv("WATSONX_URL", "")),
           apikey=SecretStr(os.getenv("WATSONX_API_KEY", "")),
           project_id=os.getenv("WATSONX_PROJECT_ID", ""),
           params={
               "max_new_tokens": 100,
               "temperature": 0.5,
           }
       )
       print(f"  → Sending request...")
       response = llm.invoke(question)
       print(f"  ✓ Response received")
       return response
   
   # Multiple models and prompts
   models = ['ibm/granite-8b-code-instruct', 'ibm/granite-4-h-small']
   questions = [
       "Tell me a fun fact about space exploration",
       "What would happen if you could travel at the speed of light?",
       "Explain quantum entanglement in simple terms",
       "What are the most interesting unsolved mysteries in science?",
       "If you could have any superpower, what would be most useful and why?"
   ]
   
   print("\n=== Starting WatsonX LLM Test ===\n")
   
   # Run multiple queries with different models and prompts
   for i in range(5):
       model = random.choice(models)
       question = random.choice(questions)
   
       print(f"\n[Query {i+1}/5]")
       print(f"Question: {question}")
   
       try:
           response = generate_response(model, question)
           print(f"Response: {response}\n")
       except Exception as e:
           print(f"ERROR: {str(e)}\n")
   
   print("=== Test Complete ===")

4. Run your application.

   python3 watsonx_app.py

   The application will send questions to IBM watsonx and display the responses. OpenLLMetry automatically captures traces for each API call and sends them to Instana.

5. View data on Instana.

   After running your application, the following items are displayed on the Instana Gen AI observability dashboard:

   • Model used
   • Token usage (input and output tokens)
   • Response latency
   • Request and response content

Troubleshooting

For common issues such as traces not appearing or connection errors, see the Troubleshooting guide.

Next steps

• Explore LLM providers supported by Instana
• Learn about cost calculation for your LLM usage
• Set up alerts for your IBM watsonx API usage
• Review IBM watsonx documentation for model capabilities