Logging overview
Logging captures and stores system events and application logs for analysis, troubleshooting, and compliance purposes.
What is logging
Logging is the process of recording system events, application activities, and user actions. Logs provide a historical record that helps you understand system behavior, diagnose problems, meet compliance requirements, and perform security audits.
Key features
The logging framework provides these customization capabilities:
- Data retention configuration
- Configure how long logs are stored and understand the resource requirements when data retention periods are increased. This helps you balance storage costs with compliance and troubleshooting needs.
- External object storage
- Configure external object storage instead of using the default provided object store. This provides flexibility to use your existing storage infrastructure and manage costs effectively.
- Audit log routing
- Configure a pipeline to route audit logs to your Security Information and Event Management (SIEM) tools for security monitoring and compliance.
Note: A SIEM tool is not included. You can control what audit data is collected. For more information on configuring OpenTelemetry Collector, see Routing audit logs to SIEM.
- Log-to-metrics conversion
- Convert logs to metrics using recording rules. This enables you to create custom metrics from log data and monitor trends over time.
Log storage and archival
The logging system uses LokiStack for log storage and management. You can configure audit log archive storage to persist logs for long-term retention and compliance purposes. The system provides insights on resource requirements when you increase data retention periods, helping you plan storage capacity effectively.
Integration with external systems
You can integrate the logging system with external tools and platforms:
- External object storage systems (such as S3-compatible storage) for log archival
- SIEM tools for security monitoring and analysis
- Log analysis platforms for advanced querying and visualization
These integrations enable you to leverage existing enterprise tools and workflows while maintaining centralized log management.
Benefits
Implementing comprehensive logging provides several benefits:
- Faster troubleshooting through detailed event history
- Compliance with regulatory requirements for log retention
- Security monitoring and incident response capabilities
- Performance analysis and optimization insights
- Audit trails for accountability and forensics