August 3, 2020 By Simon Metson 3 min read

Introducing open source framework and tools built for compliance automation.

Over the past few years, IBM Cloudant has been engaging in more and more compliance-related work. This is a good thing; it gives customers confidence in the systems holding their data, and it gives us a way to critically evaluate and improve our product and processes.

A compliance accreditation is made up of controls. In some cases, lots of controls—all of which need to be consistently and continually executed, with evidence of successful (or otherwise) execution collected for audit. This is complicated by the size and dynamism of a cloud estate, especially for a service the size of IBM Cloudant.

Auditors need to see a lot of evidence data and can “sample” any device at any point within the audit period. As we got more involved in compliance activities, we quickly realised that we needed some automation to scale collection and verification of evidence if we were going to be successful in our compliance activities.

We have built an automation framework to address the problems that compliance at scale presents. It ensures a continuity of evidence, while also helping operationally manage compliance posture by verifying that evidence as it is collected.

The good news is that the framework and tools we’ve built for our compliance automation is now open source!

Introducing Auditree

From the Auditree homepage: Auditree is an opinionated set of tools to enable the digital transformation of compliance activities. It is designed to be familiar to DevOps teams and software engineers, using tools they are likely already interacting with daily.”

What does that mean? Well, we’ve built a framework and set of associated tools that helps us turn compliance problems into engineering problems. Instead of clipboards and spreadsheets, we have unit tests and data in Git. Cloudant has been using this since 2017, and other teams in IBM have been adopting it since 2018.

It’s no secret that Cloudant uses Chef and a range of continuous integration/continuous delivery tools to manage our estate. We also write a lot of things in Python. We’ve used ideas from CI/CD and configuration management in Auditree.

Evidence is collected from APIs by “fetchers” and verified by “checks.” These are just decorated Python unit tests, so they are generally easy for developers to build, while also being flexible and capable. Checks produce reports, and their status (pass/fail/warn) can be sent to notifiers—Slack, issue trackers etc.

Those decorators manage writing/reading evidence in a “locker.” The locker is a Git repository with some additional metadata tracking provided by the framework. We chose Git because it is tamper evident, gives us history of data in an efficient manner, and, most importantly, familiar to engineers. If data hasn’t changed since the last execution, Git won’t update the file, hence we track metadata such as last retrieval time alongside the evidence. Evidence is given a TTL (time to live) in the locker and only retrieved if that has expired to avoid unnecessary and possibly expensive calls to the providers of evidence.

The framework is designed to fail and recover cleanly, with each execution being a fresh install of the framework and fetchers/checks. We run the set of fetchers/checks every eight hours. This is key for two reasons: 1) It allows us to build up a consistent and comprehensive body of evidence, and 2) it means that our global team has the opportunity to identify and remediate issues or deviations before the next run.


While the execution of the framework is key—providing both operational oversight and a body of evidence—we’ve found there are other parts of the compliance puzzle that need specific tools to address.


Harvest collates evidence (well, any file in Git) over time, which allows you to build and run reports and analysis over the contents of the locker, beyond what checks are doing. This is useful when answering a data request over a period of time or if you want to have periodic roll ups of data for reporting to the organisation.


While automated data collection is key to scaling compliance activities, there are some pieces of evidence—for example, a penetration test report—that may not be retrievable in this manner. Plant provides a way to manually and correctly place data in the locker so that it can be verified by checks.


One of the first checks we built was test_abandoned_evidence. This is important because it will catch valid data that hasn’t been updated for a long time—a possible sign of a misconfiguration somewhere. However, while this is important for detecting problems, is also creates an issue—if I no longer use a service, I won’t collect evidence from it, and this check will start to fire. I also can’t just remove the evidence files; they may still be relevant for future audits. We need a way to correctly manage the retirement of evidence, and that is Prune.

Over to you

Auditree is a set of building blocks to help engineers address compliance issues at scale. We’re open sourcing it because we think it is useful, but also to see how others are addressing the same problems. We have a library of fetchers and checks (and a fairly long backlog of things to bring into the open source domain) in Arboretum, and we would welcome contributions from the community there.

Was this article helpful?

More from Cloud

Enhance your data security posture with a no-code approach to application-level encryption

4 min read - Data is the lifeblood of every organization. As your organization’s data footprint expands across the clouds and between your own business lines to drive value, it is essential to secure data at all stages of the cloud adoption and throughout the data lifecycle. While there are different mechanisms available to encrypt data throughout its lifecycle (in transit, at rest and in use), application-level encryption (ALE) provides an additional layer of protection by encrypting data at its source. ALE can enhance…

Attention new clients: exciting financial incentives for VMware Cloud Foundation on IBM Cloud

4 min read - New client specials: Get up to 50% off when you commit to a 1- or 3-year term contract on new VCF-as-a-Service offerings, plus an additional value of up to USD 200K in credits through 30 June 2025 when you migrate your VMware workloads to IBM Cloud®.1 Low starting prices: On-demand VCF-as-a-Service deployments begin under USD 200 per month.2 The IBM Cloud benefit: See the potential for a 201%3 return on investment (ROI) over 3 years with reduced downtime, cost and…

The history of the central processing unit (CPU)

10 min read - The central processing unit (CPU) is the computer’s brain. It handles the assignment and processing of tasks, in addition to functions that make a computer run. There’s no way to overstate the importance of the CPU to computing. Virtually all computer systems contain, at the least, some type of basic CPU. Regardless of whether they’re used in personal computers (PCs), laptops, tablets, smartphones or even in supercomputers whose output is so strong it must be measured in floating-point operations per…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters