August 3, 2020 By Simon Metson 3 min read

Introducing open source framework and tools built for compliance automation.

Over the past few years, IBM Cloudant has been engaging in more and more compliance-related work. This is a good thing; it gives customers confidence in the systems holding their data, and it gives us a way to critically evaluate and improve our product and processes.

A compliance accreditation is made up of controls. In some cases, lots of controls—all of which need to be consistently and continually executed, with evidence of successful (or otherwise) execution collected for audit. This is complicated by the size and dynamism of a cloud estate, especially for a service the size of IBM Cloudant.

Auditors need to see a lot of evidence data and can “sample” any device at any point within the audit period. As we got more involved in compliance activities, we quickly realised that we needed some automation to scale collection and verification of evidence if we were going to be successful in our compliance activities.

We have built an automation framework to address the problems that compliance at scale presents. It ensures a continuity of evidence, while also helping operationally manage compliance posture by verifying that evidence as it is collected.

The good news is that the framework and tools we’ve built for our compliance automation is now open source!

Introducing Auditree

From the Auditree homepage: Auditree is an opinionated set of tools to enable the digital transformation of compliance activities. It is designed to be familiar to DevOps teams and software engineers, using tools they are likely already interacting with daily.”

What does that mean? Well, we’ve built a framework and set of associated tools that helps us turn compliance problems into engineering problems. Instead of clipboards and spreadsheets, we have unit tests and data in Git. Cloudant has been using this since 2017, and other teams in IBM have been adopting it since 2018.

It’s no secret that Cloudant uses Chef and a range of continuous integration/continuous delivery tools to manage our estate. We also write a lot of things in Python. We’ve used ideas from CI/CD and configuration management in Auditree.

Evidence is collected from APIs by “fetchers” and verified by “checks.” These are just decorated Python unit tests, so they are generally easy for developers to build, while also being flexible and capable. Checks produce reports, and their status (pass/fail/warn) can be sent to notifiers—Slack, issue trackers etc.

Those decorators manage writing/reading evidence in a “locker.” The locker is a Git repository with some additional metadata tracking provided by the framework. We chose Git because it is tamper evident, gives us history of data in an efficient manner, and, most importantly, familiar to engineers. If data hasn’t changed since the last execution, Git won’t update the file, hence we track metadata such as last retrieval time alongside the evidence. Evidence is given a TTL (time to live) in the locker and only retrieved if that has expired to avoid unnecessary and possibly expensive calls to the providers of evidence.

The framework is designed to fail and recover cleanly, with each execution being a fresh install of the framework and fetchers/checks. We run the set of fetchers/checks every eight hours. This is key for two reasons: 1) It allows us to build up a consistent and comprehensive body of evidence, and 2) it means that our global team has the opportunity to identify and remediate issues or deviations before the next run.


While the execution of the framework is key—providing both operational oversight and a body of evidence—we’ve found there are other parts of the compliance puzzle that need specific tools to address.


Harvest collates evidence (well, any file in Git) over time, which allows you to build and run reports and analysis over the contents of the locker, beyond what checks are doing. This is useful when answering a data request over a period of time or if you want to have periodic roll ups of data for reporting to the organisation.


While automated data collection is key to scaling compliance activities, there are some pieces of evidence—for example, a penetration test report—that may not be retrievable in this manner. Plant provides a way to manually and correctly place data in the locker so that it can be verified by checks.


One of the first checks we built was test_abandoned_evidence. This is important because it will catch valid data that hasn’t been updated for a long time—a possible sign of a misconfiguration somewhere. However, while this is important for detecting problems, is also creates an issue—if I no longer use a service, I won’t collect evidence from it, and this check will start to fire. I also can’t just remove the evidence files; they may still be relevant for future audits. We need a way to correctly manage the retirement of evidence, and that is Prune.

Over to you

Auditree is a set of building blocks to help engineers address compliance issues at scale. We’re open sourcing it because we think it is useful, but also to see how others are addressing the same problems. We have a library of fetchers and checks (and a fairly long backlog of things to bring into the open source domain) in Arboretum, and we would welcome contributions from the community there.

More from Cloud

Get ready for change with IBM Cloud Training

2 min read - As generative AI creates new opportunities and transforms cloud operations, it is crucial to learn how to maximize the value of these tools. A recent report from the IBM Institute for Business Value found that 68% of hybrid cloud users already have a formal, organization-wide policy or approach for the use of generative AI. That same report also noted that 58% of global decision makers say that cloud skills remain a considerable challenge. Being proactive in your learning can significantly…

Data center consolidation: Strategy and best practices

7 min read - The modern pace of data creation is staggering. The average organization produces data constantly—perhaps even continuously—and soon it’s investing in servers to provide ample storage for that information. In time, and probably sooner than expected, the organization accrues more data and outgrows that server, so it invests in multiple servers. Or that company could tie into a data center, which is built to accommodate even larger warehouses of information. But the creation of new data never slows for long. And…

Hybrid cloud examples, applications and use cases

7 min read - To keep pace with the dynamic environment of digitally-driven business, organizations continue to embrace hybrid cloud, which combines and unifies public cloud, private cloud and on-premises infrastructure, while providing orchestration, management and application portability across all three. According to the IBM Transformation Index: State of Cloud, a 2022 survey commissioned by IBM and conducted by an independent research firm, more than 77% of business and IT professionals say they have adopted a hybrid cloud approach. By creating an agile, flexible and…

Tokens and login sessions in IBM Cloud

9 min read - IBM Cloud authentication and authorization relies on the industry-standard protocol OAuth 2.0. You can read more about OAuth 2.0 in RFC 6749—The OAuth 2.0 Authorization Framework. Like most adopters of OAuth 2.0, IBM has also extended some of OAuth 2.0 functionality to meet the requirements of IBM Cloud and its customers. Access and refresh tokens As specified in RFC 6749, applications are getting an access token to represent the identity that has been authenticated and its permissions. Additionally, in IBM…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters