IBM watsonx

IBM watsonx™ models are designed for the enterprise and optimized for targeted business domains and use cases. These models are a suite of foundation models that are designed for various natural language processing (NLP) and code generation tasks. These models, including the Granite series, are built to be enterprise-grade, emphasizing accuracy, reliability, and security.

Instrumenting watsonx application

To instrument the watsonx application, complete the following steps:

Make sure that your environment meets all the prerequisites. For more information, see Prerequisites.

To install dependencies for watsonx, run the following command:

pip3 install ibm-watsonx-ai==1.1.20 ibm-watson-machine-learning==1.0.366 langchain-ibm==0.3.1

Export the following credentials to access the watsonx models used in the sample application:
```
export WATSONX_URL="<watsonx-url>"
export WATSONX_PROJECT_ID="<watsonx-project-id>"
export WATSONX_API_KEY="<watsonx-api-key>"
```
To create or use an existing WATSONX_PROJECT_ID and WATSONX_API_KEY for accessing IBM watsonx, see IBM watsonx and IBM Cloud.

Confirm the installation and configuration by running the following Python code:

import os, types, time, random
from ibm_watsonx_ai.metanames import GenTextParamsMetaNames
from ibm_watsonx_ai.foundation_models import ModelInference
from pprint import pprint
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow
from langchain_ibm import WatsonxLLM

Traceloop.init(app_name="watsonx_chat_service")

def watsonx_llm_init() -> ModelInference:
    watsonx_llm_parameters = {
        GenTextParamsMetaNames.DECODING_METHOD: "sample",
        GenTextParamsMetaNames.MAX_NEW_TOKENS: 100,
        GenTextParamsMetaNames.MIN_NEW_TOKENS: 1,
        GenTextParamsMetaNames.TEMPERATURE: 0.5,
        GenTextParamsMetaNames.TOP_K: 50,
        GenTextParamsMetaNames.TOP_P: 1,
    }
    models = ['ibm/granite-13b-chat-v2', 'ibm/granite-13b-instruct-v2']
    model = random.choice(models)
    watsonx_llm = WatsonxLLM(
        model_id=model,
        url=os.getenv("WATSONX_URL"),
        apikey=os.getenv("WATSONX_API_KEY"),
        project_id=os.getenv("WATSONX_PROJECT_ID"),
        params=watsonx_llm_parameters,
    )
    return watsonx_llm

@workflow(name="watsonx_llm_langchain_question")
def watsonx_llm_generate(question):
    watsonx_llm = watsonx_llm_init()
    return watsonx_llm.invoke(question)

for i in range(10):
    question_multiple_responses = [ "What is AIOps?", "What is GitOps?"]
    question = random.choice(question_multiple_responses)
    response = watsonx_llm_generate(question)
    if isinstance(response, types.GeneratorType):
        for chunk in response:
            print(chunk, end='')
    pprint(response)
    time.sleep(3)

Run the application by running the following command:
```
python3 ./<watsonx-sample-application>.py
```

After you configure monitoring, Instana collects the following traces and metrics from the sample application:

To view the traces collected from LLM, see View traces.

To view the metrics collected from LLM, see View metrics.

Adding LLM Security

Revealing Personally Identifiable Information (PII) to LLMs can lead to serious security and privacy risks. These risks include violating contractual obligations and increased chances of data leakage or a data breach. For more information, see LLM security.