March 17, 2020 By Charles Comiskey
Michael Shi
2 min read

Clients today are increasingly accessing archived data for ad hoc insights over extended periods of time, forensic investigation, and archive for regulatory and business compliance.

The challenge: Running cloud operations at volume may result in large archives of log and activity event data.

The solution: Archiving with hourly frequency.

We are excited to introduce an improved archiving feature for IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA that breaks up archives into an hourly cadence. For most customers, the benefits will be seamless. If you have a custom workflow based on your archived logs, it may have to be updated to accommodate the change.

What’s changing

Starting April 6, 2020, we will slowly transition our customers to a new archive file format that breaks up archives into an hourly cadence instead of a daily cadence.

This feature will allow you to receive archived logs faster, and you’ll be able to unzip just a portion of your logs instead of an entire day’s worth. Additionally, the new folder format better supports Hive partitions.

As we transition you to the new format, you might observe a small overlap between the older daily format archives and the new archives. Expect to see hourly archives in your provider within 24 hours of the logs being received by us.

The new archive directory format for all providers will look like this:

year=YYYY/month=MM/day=DD/<accountID>.<YYYY>-<MM>-<DD>.<HH>00.json.gz (where HH is hours in 24 format)

The current archive directory format looks like this:


The current archive directory format for S3 looks like this:


The content of the archive files will remain the same as before.

What you need to do

For most customers, no re-configuration is needed. We will gradually be rolling out these changes starting on April 6, 2020, to accounts with archiving enabled.

If you have systems that consume your archived files (like a data warehouse), you’ll have to update those systems to expect the new archiving directory structure. Nothing in the Export API or other LogDNA app experiences will change.

You can learn more about archiving in our archiving guide.

Learn more

IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA are two IBM Cloud services delivered in partnership with LogDNA to deliver full Log Management Service capabilities. Both services are hosted on IBM Cloud in many world-wide regions, allowing you to co-locate your logging and cloud activity events.

Setup is simple and allows you to quickly gain insights into your workloads. Click on the Observability link to gain visibility into the performance and health of your workloads today.

Realize additional value from your data

Archiving your data to IBM Cloud Object Storage opens your data in new exciting ways to realize additional value from your operational logs. Learn more about two of them here:

More from Cloud

IBM Cloud Reference Architectures unleashed

2 min read - The ability to onboard workloads to cloud quickly and seamlessly is paramount to accelerate enterprises digital transformation journey. At IBM Cloud, we're thrilled to introduce the IBM Cloud® Reference Architectures designed to empower clients, technical architects, strategists and partners to revolutionize the way businesses harness the power of the cloud. VPC resiliency: Strengthening your foundation Explore the resilience of IBM Cloud Virtual Private Cloud through our comprehensive resources. Dive into our VPC Resiliency white paper, a blueprint for building robust…

Enhance your data security posture with a no-code approach to application-level encryption

4 min read - Data is the lifeblood of every organization. As your organization’s data footprint expands across the clouds and between your own business lines to drive value, it is essential to secure data at all stages of the cloud adoption and throughout the data lifecycle. While there are different mechanisms available to encrypt data throughout its lifecycle (in transit, at rest and in use), application-level encryption (ALE) provides an additional layer of protection by encrypting data at its source. ALE can enhance…

Attention new clients: exciting financial incentives for VMware Cloud Foundation on IBM Cloud

4 min read - New client specials: Get up to 50% off when you commit to a 1- or 3-year term contract on new VCF-as-a-Service offerings, plus an additional value of up to USD 200K in credits through 30 June 2025 when you migrate your VMware workloads to IBM Cloud®.1 Low starting prices: On-demand VCF-as-a-Service deployments begin under USD 200 per month.2 The IBM Cloud benefit: See the potential for a 201%3 return on investment (ROI) over 3 years with reduced downtime, cost and…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters