Monitoring DeepSeek models

Edit online

DeepSeek provides open-weight AI models that can be integrated into diverse workflows, from conversational AI to software development. This guide shows you how to instrument an application using DeepSeek models with OpenLLMetry to send telemetry data to Instana.

Prerequisites

Make sure that the following prerequisites are met:

Instrumenting your DeepSeek application

  1. Install the required packages. 
    pip install groq traceloop-sdk
  2. Export your Groq API key. 
    export GROQ_API_KEY="<your-groq-api-key>"
  3. Create your DeepSeek application. Create a Python file with the following code: 
    from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow
from groq import Groq
# Initialize OpenLLMetry
Traceloop.init(app_name="deepseek_app", disable_batch=True)
# Initialize Groq client
client = Groq()
@workflow(name="deepseek_conversation")
def ask_deepseek(question: str):
    """Send a question to DeepSeek and get a response."""

    response = client.chat.completions.create(
        messages=[{"role": "user", "content": question}],
        model="deepseek-r1-distill-llama-70b", 
    )

    # Remove thinking tags from response
    result = (
        response.choices[0] 
        .message.content.replace("<think>", "")
        .replace("</think>", "") 
    ) 

    return result.strip()

# Example usage
if __name__ == "__main__":
    questions = [
        "How does transfer learning improve AI model performance?", 
        "What are the challenges in scaling LLMs for enterprise use?"    
    ]

    for question in questions:
        print(f"\nQuestion: {question}")
        answer = ask_deepseek(question)
        print(f"Answer: {answer}\n")
        print("-" * 80)
    Note:

    DeepSeek models are accessed through the Groq API. The `deepseek-r1-distill-llama-70b` model includes reasoning tokens wrapped in `<think>` tags, which are removed in the example.

  4. Run your DeepSeek application. 
    python3 deepseek_app.py

    The application will send questions to DeepSeek and display the responses. OpenLLMetry automatically captures traces for each API call and sends them to Instana.

  5. View data on Instana.

    After running your application, the following items are displayed on the Instana Gen AI observability dashboard:

    • Model used
    • Token usage (input and output tokens)
    • Response latency
    • Request and response content

Troubleshooting

For common issues such as traces not appearing or connection errors, see Troubleshooting.

Authentication errors

If you encounter authentication errors:

  1. Verify your GROQ_API_KEY is set correctly
  2. Check whether your API key is valid on Groq Console
  3. Make sure that your API key is not expired or revoked

Rate limiting errors

If you encounter rate limit errors:

  1. Check your Groq account's rate limits
  2. Add delays between requests if making multiple calls
  3. Consider upgrading your Groq plan for higher limits
  4. Implement exponential backoff for retries

Model not found errors

If you encounter model not found errors:

  1. Verify the model name is correct (for example, deepseek-r1-distill-llama-70b)
  2. Check whether the model is available through Groq3.
  3. For DeepSeek models, see groq

    Supported Models

Next steps