January 19, 2021 By Charles Comiskey
Michael Shi
2 min read

The hourly archiving service is ready to proceed for all IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA accounts.

Earlier in 2020, we shared a preview of a new feature for our LogDNA based offerings — a switch from daily to hourly archiving. Since then, we have worked closely with several clients to improve the capability. The rollout to hourly archive has been long-awaited and the service is ready to proceed for all IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA accounts. The transition is starting now and is expected to complete over the remaining week.

What is changing?

This feature will allow you to receive archived logs faster, and you’ll be able to unzip just a portion of your logs instead of an entire day’s worth. Additionally, the new folder format better supports Hive partitions.

As we transition you to the new format, you might observe a small overlap between the older daily format archives and the new archives. Expect to see hourly archives in your provider within 24 hours of the logs being received by us. These details can also be found in our documentation for IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA.

The new archive directory format for all providers will look like this:

year=YYYY/month=MM/day=DD/<accountID>.<YYYY>-<MM>-<DD>.<HH>00.json.gz (where HH is hours in 24 format)

The current archive directory format looks like this:

/<accountID>.<YYYY>-<MM>-<DD>.<clusterId>.json.gz

The current archive directory format for S3 looks like this:

YYYY/MM/<accountID>.<YYYY>-<MM>-<DD>.<clusterId>.json.gz

The content of the archive files will remain the same as before.

What you need to do

For most customers, no reconfiguration is needed. We will be rolling out these changes to accounts with archiving enabled.

If you have systems that consume your archived files (like a data warehouse), you’ll have to update those systems to expect the new archiving directory structure. Nothing in the Export API or other LogDNA app experiences will change. You can learn more about archiving in our archiving guide.

Learn more

IBM Log Analysis with LogDNA and IBM Cloud Activity Tracker with LogDNA are two IBM Cloud services delivered in partnership with LogDNA to deliver full Log Management Service capabilities. Both services are hosted on IBM Cloud in many worldwide regions, allowing you to co-locate your logging and cloud activity events.

Setup is simple and allows you to quickly gain insights into your workloads. Click on the Observability link to gain visibility into the performance and health of your workloads today.

Realize additional value from your data

Archiving your data to IBM Cloud Object Storage opens your data in new exciting ways to realize additional value from your operational logs. Learn more about three of them here:

More from Announcements

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity 

2 min read - To meet customer demands, applications are expected to be performing at their best at all times. Simultaneously, applications need to be flexible and cost effective, and therefore supported by an underlying infrastructure that is equally reliant, performant and secure as the applications themselves.   Easier said than done. According to EMA's 2024 Network Management Megatrends report only 42% of responding IT professionals would rate their network operations as successful.   In this era of hyper-distributed infrastructure where our users, apps, and data…

IBM named a Leader in Gartner Magic Quadrant for SIEM, for the 14th consecutive time

3 min read - Security operations is getting more complex and inefficient with too many tools, too much data and simply too much to do. According to a study done by IBM, SOC team members are only able to handle half of the alerts that they should be reviewing in a typical workday. This potentially leads to missing the important alerts that are critical to an organization's security. Thus, choosing the right SIEM solution can be transformative for security teams, helping them manage alerts…

IBM and MuleSoft expand global relationship to accelerate modernization on IBM Power 

2 min read - As companies undergo digital transformation, they rely on APIs as the backbone for providing new services and customer experiences. While APIs can simplify application development and deliver integrated solutions, IT shops must have a robust solution to effectively manage and govern them to ensure that response times and costs are kept low for all applications. Many customers use Salesforce’s MuleSoft, named a leader by Gartner® in full lifecycle API management for seven consecutive times, to manage and secure APIs across…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters