Data plane logging configuration - AWS

Enable Amazon CloudWatch for logs and metrics

Amazon CloudWatch provides centralized observability for logs, metrics, and alarms.

  1. Logging for compute resources
    Ensure your applications and services emit logs to local files or standard output (stdout/stderr). These logs can be forwarded to CloudWatch using one of the following methods:
    • CloudWatch Agent
    • AWS Distro for OpenTelemetry (ADOT)
    • Fluent Bit (for ECS or EKS)
  2. VPC flow logs
    Capture network traffic at the VPC level to monitor ingress and egress flows. Use the following command to enable:
    aws ec2 create-flow-logs \
  --resource-type VPC \
  --resource-id vpc-xxxxxxxx \
  --traffic-type ALL \
  --log-destination-type cloud-watch-logs \
  --log-group-name "/byoc/network/flowlogs"
  3. Enable Amazon GuardDuty
    GuardDuty provides intelligent threat detection for your AWS environment, identifying suspicious activity such as anomalous API calls or port scanning.
    aws guardduty create-detector --enable

Enable AWS CloudTrail for data plane logging

AWS CloudTrail captures all data plane activity, including API calls and console access.
  1. Create a multi-region trail.
  2. Store logs in an encrypted S3 bucket.
  3. Forward logs to CloudWatch for alerting and SIEM integration.
aws cloudtrail create-trail \
  --name byoc-data-trail \
  --s3-bucket-name byoc-trail-logs \
  --is-multi-region-trail

Accessing and querying logs

CloudWatch console: Navigate to log groups and streams.
  • AWS CLI:
    aws logs get-log-events \
  --log-group-name "/byoc/compute/logs" \
  --log-stream-name "instance-id" \
  --limit 100
  • CloudWatch logs insights:
    fields @timestamp, @message
| filter @message like /error/i
| sort @timestamp desc