You’re biased. I’m biased. It’s inevitable. And the first step to overcoming it is to acknowledge it. But don’t just take my word for it, take Gartner’s.
In a recent report, Gartner acknowledges that bias is an intrinsic part of analytics; it is “inherent in the development of analytic models, data selection and the associated algorithms.” This is not a call to neutrality; on the contrary, the report seems to argue that complete neutrality or objectivity is impossible.
Gartner is using provocative thinking to make a very important and rather subtle point: understanding bias is not the same as eliminating bias. A robust analytics methodology and sound decision-making procedures are good things in themselves: they lead to better decisions that are more firmly rooted in data. But they’re still biased. If you’re pursuing analytics, the essential choice is whether to identify and correct for these inherent biases, or persist in the delusion that you don’t have bias at all.
Four recommendations from Gartner for managing bias
The full Gartner report is available for purchase, but their summary page lists four key recommendations that can help you understand and manage the inherent bias in your analytic efforts:
- Use independent auditors to detect and analyze bias in your digital personality. This seems to be Gartner’s central point, and it’s one that I totally agree with: it’s both possible and very useful to determine the extent and nature of your bias.
- Compare the bias in data and analytic models with the business understanding of that bias. To me, this means that different kinds of models are fine (e.g. broad vs. narrow), but not every model is a good fit for every kind of question. Keep this in mind as you build models or adapt to new questions.
- Perform periodic reviews of analytics with an objective board of standards. This makes sense to me, because analytics should not be allowed to become an end in itself. It’s important to know when you’ve reached the limits of a particular model…and to have a board with the authority to convince any reluctant stakeholders.
- Review boards should challenge exclusive ownership of analytics to avoid a self-reinforcing cycle. I’ve seen this too often: when analytics is idolized, there can be a tendency to fit every new finding to an existing analytic model. Periodic reviews can introduce a ‘sanity check’ and make sure the analysis still fits the business need.
To see more, you’ll need to obtain and read the full Gartner report.
But wait…isn’t “unbiased” a good thing?
All this seems like very sound advice. But it might also seem to put analytics vendors in an awkward position. Analytics solutions are often described as a means to become unbiased: different authors have done so recently on this very blog. But I think there’s no real disagreement here. For instance, this blog says there’s “nothing wrong” with biased data; it can be “massively useful.” The key factor is in knowing what to ask.
I believe this is where the Gartner and IBM arguments start to resemble one another. You can think of “unbiased” as shorthand for “well-understood.” If you understand your biases—the capabilities and limits of your mindset and tools—you can get more useful analytics results. And isn’t that, ultimately, the point?
Tools like IBM Cognos Analytics and IBM Watson Analytics can help provide useful analytics. They aren’t magic cure-alls; they need to be part of a comprehensive and self-reflective analytic culture. But they do have one important strength that can help with your bias assessment and bias mitigation. Because the tools are built for both line-of-business and IT users, it’s much easier to equip multiple people with the relevant abilities: to infuse domain-specific expertise into their processes, perform sophisticated analysis, and cross-check and benefit from one another’s work.
One step closer to an analytics culture
I think that Gartner’s report is a good reminder of an often-overlooked reality: as long as data depends on humans, and humans are asking the analytic questions and designing the analytics systems, there will be some element of bias. Keeping that truth in mind is one step toward improving your analytics culture and getting better insights from your data.
Note: The full citation for the Gartner report is Beyer, Mark A.; Kuntnick, Dale; and Dayley, Alan. “Embrace Your Bias to Enable Analytic Clarity” Gartner, 23 June 2017. Doc ID G00319217. Available for purchase here.