Amazon SageMaker AI

Amazon SageMaker is a fully managed machine learning (ML) service which offers a seamless experience for developing AI and analytics solutions. You can use Instana to monitor Amazon SageMaker AI by collecting traces to provide real-time insights into their performance, efficiency, and cost.

Instrumenting Amazon SageMaker AI Application

Before proceeding, make sure that your environment meets all the prerequisites. For more information, see Prerequisites. To instrument the Amazon SageMaker application, complete the following steps:

Run the following command, to install the dependencies for Amazon SageMaker:
```
pip3 install boto3==1.35.81
```
Export the following credentials to access the Amazon SageMaker models used in the sample application.
```
export AWS_ACCESS_KEY_ID=<access-key-id>
export AWS_SECRET_ACCESS_KEY=<access-key>
```
To create an API key to access the Amazon SageMaker API or use the existing one, see Amazon IAM.

Run the following code to generate a SageMaker sample application:


  import boto3
  from traceloop.sdk import Traceloop
  from traceloop.sdk.decorators import task, workflow
  import numpy as np
  import pandas as pd

  # This code is written expecting a sample model that is expecting a 12 field payload.

  # The details will be available with the app_name in Instana UI
  Traceloop.init(app_name="sagemaker_example", disable_batch=True) # initializing the traceloop

  # Creates a SageMaker Runtime client using Boto3. Allows you to interact with deployed SageMaker endpoints.
  runtime = boto3.Session().client('sagemaker-runtime', region_name="us-east-1")


  @workflow(name="sagemaker_example")
  def invoke_sagemaker_endpoint(endpoint):
      # 1 row with 12 features
      X_test = ([[1, 8.2, 10.5, 1.5, 9.6, 1.3, 11.2, 2.9, 100, 11, 12.0, 13]]) # payload being passed

      # Converting the data into CSV format
      df = pd.DataFrame(X_test)
      csv_input = df.to_csv(index=False, header=False)
      csv_bytes = csv_input.encode('utf-8') # Converting to byte as expected by sagemaker

      # Sends a POST request to the SageMaker endpoint using the invoke_endpoint() method
      response = runtime.invoke_endpoint(
          EndpointName=endpoint,
          ContentType='text/csv',
          Body=csv_bytes
      )

      # Reads the model’s prediction result from the response and decodes it from bytes to string.
      return response['Body'].read().decode('utf-8')

  invoke_sagemaker_endpoint('<sagemaker-endpoint>') # Endpoint passed

Run the application by using the following command:
```
python3 ./<sagemaker-sample-application>.py
```

After you configure monitoring, Instana collects the traces from the sample application. To view the traces that are collected from the application see Create an application perspective for viewing traces.

Adding LLM Security

When the Personally Identifiable Information (PII) is exposed to LLMs that can lead to serious security and privacy risks such as violating contractual obligations, increased chances of data leakage, or data breach. For more information, see LLM security.