Data plane logging configuration

Configure Azure-native logging and monitoring for your Data Plane resources.

Azure provides a native observability suite with Azure Monitor, Log Analytics, and Microsoft Sentinel for collecting, querying, and alerting on telemetry and security data across the Data Plane.

Enable Diagnostic Settings for Azure Resources

Diagnostic settings must be enabled for each Azure resource to emit logs and metrics.

Common Resource Logging Targets:

  • Virtual Machines: Performance counters, syslog, audit logs
  • Network Interfaces: NSG Flow Logs, metrics
  • Azure Storage: Read/Write/Delete logs
  • Key Vault: Audit logs (e.g., Secret access)

Configuration Example (Azure CLI):


az monitor diagnostic-settings create \
--name "byoc-logs" \
--resource "/subscriptions/xxxx/resourceGroups/byoc-resources/providers/Microsoft.Compute/virtualMachines/my-vm" \
--workspace "/subscriptions/xxxx/resourceGroups/logs/providers/Microsoft.OperationalInsights/workspaces/byoc-logs" \
--logs '[{"category": "AuditLogs", "enabled": true}]'

You can also route logs to:

  • Log Analytics Workspace (for querying)
  • Storage Account (for archiving)
  • Event Hub (for third-party SIEM integration)

Monitor Logs with Log Analytics

Log Analytics Workspaces serve as the backend for log storage and query operations.

Sample KQL Queries:

Failed Login Attempts (Windows/Linux VMs):


SecurityEvent
| where EventID == 4625
| summarize Count = count() by Account, Computer, bin(TimeGenerated, 1h)

Unauthorized Key Vault Access:


AzureDiagnostics
| where ResourceType == "KEYVAULTS"
| where OperationName contains "Secret" and ResultType != "Success"

Enable Microsoft Sentinel (Optional but Recommended)

Microsoft Sentinel is a SIEM solution built on Log Analytics that offers threat detection, investigation, and automated response capabilities.

Enable Sentinel:


az sentinel workspace create \
--resource-group byoc-rg \
--workspace-name byoc-logs \
--location eastus

Examples:

  • Unusual login locations
  • Suspicious traffic from external IPs
  • Changes to role assignments

Use Playbooks for Automation:

  • Notifying via Microsoft Teams
  • Blocking IPs via NSGs
  • Opening tickets in ITSM systems

Best Practices for Logging and Security Monitoring

Category AWS Recommendations Azure Recommendations
Centralized Logging Route all logs to CloudWatch log groups organized by workload or environment. Use a single Log Analytics Workspace for all Data Plane resources.
Retention Policies Apply CloudWatch retention settings or archive to S3 with lifecycle policies. Set data retention for Log Analytics, and archive to Azure Storage if needed.
Access Control Restrict log access using IAM policies and enable CloudTrail log integrity. Use RBAC for Log Analytics and Sentinel, and enable immutable storage.
Encryption Enable encryption at rest for CloudWatch, CloudTrail, and S3 (using KMS). Use customer-managed keys (CMKs) for log data in Log Analytics and Storage.
Alerting Create CloudWatch Alarms or use Amazon EventBridge for security events. Use Azure Monitor Alerts and Microsoft Sentinel rules to trigger notifications.
Threat Detection Enable GuardDuty and integrate findings with CloudWatch or external SIEM. Enable Microsoft Defender for Cloud and Sentinel for built-in threat detection.
Network Visibility Enable VPC Flow Logs, DNS logs, and use NACL/SG logging if applicable. Use NSG Flow Logs, Firewall logs, and Private Link monitoring.
Auditability Use CloudTrail to track API activity across services and regions. Ensure Activity Logs and Resource Logs are routed to your analytics workspace.
Automation & Response Automate remediation using Lambda or Step Functions. Use Logic Apps with Sentinel Playbooks to auto-respond to threats.
Log Querying Use CloudWatch Logs Insights or export logs to S3 and analyze via Athena. Use KQL in Log Analytics for advanced filtering and security investigations.