Migrating from OTel Data Collector

Edit online

The OTel Data Collector for Generative AI (ODCG) is deprecated. Instana has simplified the setup for generative AI observability by removing the need for a separate data collector component. You can migrate your existing setup.

Note: This procedure is applicable only if you are using ODCG. New Instana users, see Getting started.

Why ODCG is deprecated

Instana now supports sending generative AI application telemetry data directly to Instana without requiring a separate data collector component. This simplification provides reduced complexity, easier configuration, pricing management directly within the Instana UI, and lower infrastructure costs as no resources need to be allocated to ODCG.

Procedure

If you're currently using ODCG, complete the following steps to migrate to the new direct integration approach:

  1. Compare the new and old architecture:
    • Old approach (deprecated): 
      Generative AI App → OTel Data Collector (ODCG) → Instana Agent/Backend → Instana
    • New approach: 
      Generative AI App → Instana Agent/Backend → Instana
  2. Update your application configuration

    The key change is adding the INSTANA_PLUGIN=genai resource attribute and removing the separate metrics endpoint.

    • For agent mode (sending data through Instana agent)
      The old configuration is shown in the following example. 
      export TRACELOOP_BASE_URL=<instana-agent-host>:4317
export TRACELOOP_HEADERS="x-instana-key=<agent-key>,x-instana-host=<host>"
export TRACELOOP_METRICS_ENDPOINT=<odcg-host>:8000
export TRACELOOP_METRICS_ENABLED=trueexport TRACELOOP_LOGGING_ENABLED=true
export OTEL_EXPORTER_OTLP_INSECURE=true
      The new configuration is shown in the following example. 
      export OTEL_RESOURCE_ATTRIBUTES="INSTANA_PLUGIN=genai"
export TRACELOOP_BASE_URL=<instana-agent-host>:4317
export TRACELOOP_HEADERS="x-instana-key=<agent-key>,x-instana-host=<host>"
export TRACELOOP_METRICS_ENABLED=trueexport TRACELOOP_LOGGING_ENABLED=true
export OTEL_EXPORTER_OTLP_INSECURE=true
      The following items are changed.

      • OTEL_RESOURCE_ATTRIBUTES="INSTANA_PLUGIN=genai" is added to process your data as generative AI telemetry

      • TRACELOOP_METRICS_ENDPOINT is removed for the metrics to flow through the same endpoint as traces

    • For agentless mode (sending data directly to Instana backend)

      The old configuration is shown in the following example. 
      export TRACELOOP_BASE_URL=<instana-otlp-endpoint>:4317 
export TRACELOOP_HEADERS="x-instana-key=<agent-key>,x-instana-host=<host>" 
export TRACELOOP_METRICS_ENDPOINT=<odcg-host>:8000 
export TRACELOOP_METRICS_ENABLED=true export TRACELOOP_LOGGING_ENABLED=true 
export OTEL_EXPORTER_OTLP_INSECURE=false
      The new configuration is shown in the following example. 
      export OTEL_RESOURCE_ATTRIBUTES="INSTANA_PLUGIN=genai" 
export TRACELOOP_BASE_URL=<instana-otlp-endpoint>:4317 
export TRACELOOP_HEADERS="x-instana-key=<agent-key>,x-instana-host=<host>" 
export TRACELOOP_METRICS_ENABLED=true export TRACELOOP_LOGGING_ENABLED=true 
export OTEL_EXPORTER_OTLP_INSECURE=false
      The following items are changed.

      • OTEL_RESOURCE_ATTRIBUTES="INSTANA_PLUGIN=genai" is added to process your data as generative AI telemetry.

      • TRACELOOP_METRICS_ENDPOINT is removed for the metrics to flow through the same endpoint as traces

  3. Migrate your pricing configuration. Model pricing is managed through the Instana UI instead of a configuration file.

    1. Locate your current pricing configuration.

      If OTel Data Collector is used, a prices.properties file similar to the following example is displayed.

      openai.gpt-4.input=0.03 
openai.gpt-4.output=0.06 
openai.gpt-3.5-turbo.input=0.0015 
openai.gpt-3.5-turbo.output=0.002 
anthropic.claude-2.input=0.008 
anthropic.claude-2.output=0.024

    2. Configure pricing on the Instana UI.

      1. Log on to the Instana UI.
      2. Navigate to the GenAI observability dashboard.
      3. Click the Configuration tab. A predefined list of common LLM models with default pricing is displayed.
      4. Update pricing for an existing model:
        1. Find the model in the list.
        2. Click on the model name and click Edit
        3. Update the input and output token prices.
        4. Click Save.
      5. Add a new model:
        1. Click Add model pricing.
        2. Enter the provider (for example, "openai", "anthropic").
        3. Enter the model ID (for example, "gpt-4", "claude-2").
        4. Enter platform to set platform specific pricing.
        5. Enter input and output token prices (for example, "bedrock", "langchain").
        6. Click Add.
        Figure 1. Generative AI-LLM-pricing-configuration
        Generative AI-LLM-pricing-configuration

        Benefits of dashboard-based pricing:

        • Changes take effect immediately - no redeployment needed
        • Easy to update as pricing changes
        • Centralized management across all your generative AI applications
        • Audit trail of pricing changes
        Note:

        Cost metrics will appear in your dashboards only after you configure pricing. Other metrics (latency, token counts, error rates) are visible regardless of pricing configuration.

  4. After updating the configuration, restart your generative AI application to apply the changes.

    • For Kubernetes deployments: 
      kubectl rollout restart deployment/<your-app-deployment> -n <your-namespace>
    • For Red Hat OpenShift deployments: 
      oc rollout restart deployment/<your-app-deployment> -n <your-namespace>

    • For standalone applications:

      Verify that data is flowing correctly. After restarting your application, verify that Instana is receiving data:

      1. Check traces:
        1. Go to the GenAI observability dashboard in Instana.
        2. Verify that new traces are appearing for your application.
        3. Verify that traces show LLM calls, token counts, and latency information.
      2. Check metrics:

        1. In the GenAI observability dashboard, check the metrics view.

          Check whether metrics appear for Token usage (input and output), Latency, and Cost metrics (if pricing is configured).

      3. Check logs:

        • If logging is enabled, verify that logs are ingested.
        • Look for any error messages related to OpenTelemetry or Traceloop.

      4. Review application logs:

        • Check your application logs for any OpenTelemetry errors.
        • Look for successful connection messages to the Instana endpoint.

      5. Optional: Remove the OTel Data Collector.

After you verify that data is flowing correctly with the new configuration, you can safely remove the OTel Data Collector deployment.

Troubleshooting

For common migration issues such as data not appearing, cost metrics not showing, or connection problems, see Troubleshooting.

Still connecting to old ODCG endpoint

Issue: Application is still trying to connect to the OTel Data Collector after migration.

To troubleshoot this issue, try the following steps:

  1. Check whether you removed TRACELOOP_METRICS_ENDPOINT from your configuration.
  2. Check for environment variables set at different levels (container, deployment, ConfigMap).
  3. Restart your application after removing the variable.

Next steps

After successfully migrating, you can complete the following tasks:

  • Explore the GenAI observability dashboard:

    • View traces to understand LLM interaction patterns
    • Analyze metrics to identify performance bottlenecks
    • Monitor costs to optimize your LLM usage

  • Set up alerts:

    • Configure alerts for high latency
    • Set up cost threshold notifications
    • Monitor error rates

  • Optimize your setup:

    • Review and adjust pricing as needed
    • Fine-tune your Traceloop instrumentation
    • Explore advanced features like custom attributes

  • Clean up:

    • Remove the OTel Data Collector deployment if you haven't already
    • Update your documentation to reflect the new setup
    • Share the simplified setup with your team