Install OTel Data Collector for LLM (ODCL)

To collect OpenTelemetry metrics for various LLMs and LLM applications, you need to install ODCL. All implementations are based on predefined OpenTelemetry Semantic Conventions.

1. Verify that the following prerequisites are met:

   • Java SDK 11+ is installed. Run the following command to check the installed version:

     java -version
     

2. Install the collector:

   a. Download the latest version of otel-dc-llm package available:

   wget https://github.com/instana/otel-dc/releases/download/v1.0.7/otel-dc-llm-1.0.7.tar
   

   b. Extract the package to the preferred deployment location:

   tar xf otel-dc-llm-1.0.7.tar
   

3. Modify the configuration file:

   a. Open the config.yaml file:

   cd otel-dc-llm-1.0.7
   vi config/config.yaml
   

   b. Update the following fields in the config.yaml file:

   • otel.agentless.mode: Set the connection mode for the OTel data connector. The default mode is agentless.
   • otel.backend.url: Specify the gRPC endpoint for the Instana backend or Instana agent, depending on whether the agentless mode is used or not.
   • callback.interval: Define the time interval (in seconds) for posting data to the backend or agent.
   • otel.service.name: Assign a name to the Data Collector, which can be any string that you choose.
   • otel.service.port: Set the listening port for the Data Collector to receive metrics data from the instrumented applications. The default port is 8000.

   If otel.agentless.mode is set to true, metrics data is sent directly to the Instana backend. The otel.backend.url is the gRPC endpoint of the backend otlp-acceptor component with http/https scheme, for example http://<instana-otlp-endpoint>:4317. Use https:// scheme if the endpoint of the Instana backend otlp-acceptor in SaaS environments is TLS-enabled. For more information about Instana SaaS environment, see Endpoints of the Instana backend otlp-acceptor.

   If otel.agentless.mode is set to false, metrics data is sent to the Instana agent. The otel.backend.url is the gRPC endpoint of the Instana agent, for example http://<instana-agent-host>:4317.

   c. Open the prices.properties file:

   vi config/prices.properties
   

   d. Customize pricing for models in use by using the following format in the prices.properties file:

   <aiSystem>.<modelId>.input=0.0
   <aiSystem>.<modelId>.output=0.0
   

   • The <aiSystem> is the LLM provider in use.
   • The <modelId> can be set to * to match all modelIds within the <aiSystem>. The modelIds with * have less priority than items with a specific modelId.

   e. Open the logging.properties file by using the following command:

   vi config/logging.properties
   

   Configure the Java logging settings in the logging.properties file according to your needs.

4. Run the Data Collector with the following command according to your current system:

   nohup ./bin/otel-dc-llm >/dev/null 2>&1 &
   

   You can also use tools like tmux or screen to run this program in the background.