Like painting the proverbial bridge, organizations are discovering that cleaning and maintaining enterprise data is a task that never ends. What's worse, it's possible to spend a sizable chunk of time—and cash—to clear out errors, omissions, and gaps in years of accumulated data, only to see the same problems reappear over time in enterprise data stores.
The challenge of maintaining data quality has become so large and intense that local fixes and one-time solutions are insufficient. Even information management projects and tools like master data management are not enough to govern data quality without a corporate commitment and culture-building effort. But how do you get that commitment? For that matter, where do you start when defining the problem?
Five years ago, IBM's Steven Adler convened a group of customers to tackle the question of governance, forming the IBM Data Governance Council to investigate and implement best practices—and to advise IBM on how to proceed. At a recent conference, I invited Adler to join Brett Gow from IBM Global Business Services and Lise Neely from the IBM InfoSphere team to discuss the state of the issue and the market.
How did we get to "data governance"?
ADRIAN: Steven, when you started the council several years ago, were a lot of people concerned about data governance?
ADLER: There were a lot of concerned people; they just didn't know what it was called. They were concerned about data quality and security. It had dawned on them that they might be spending too much money to protect worthless data and not enough on valuable data, and they couldn't tell the difference.
People were governing the use of information—they just weren't doing it with forethought, or consistency, or with a plan. Everybody makes governance decisions. Every decision is a policy—but how do you know whether it's the right policy, whether your outcomes are consistent with your goals? The challenge today isn't just to govern. It's to govern well.
GOW: Often, people don't understand the complexity [of governance]. Many think it's project-based or focused on technology. Others think it's an overlay to existing processes and procedures—activities, roles, and responsibilities different from what they're doing today. They are surprised when they understand the dynamics, the politics.
ADRIAN: For data managers, what are the symptoms that there's a data governance problem?
NEELY: If you are facing the same fire drill over and over, there is a process breakdown somewhere, and it's time to look at the root cause and address it there.
ADRIAN: Is it challenging to convert the abstract vision to something somebody in the organization might be willing to fund, and staff, and organize and drive?
GOW: Absolutely. There may be a visionary, but that's rare. More often it's regulatory mandates, audit issues that indicate the organization needs to take action. Sometimes governance is wrapped around specific initiatives: data warehouse development, master data management, or data quality issues.
ADLER: Following the credit crisis, regulators stepped in to do "stress tests." They discovered firms lacked systemic ways of reporting simple things; different business processes existed; people didn't want to share information. Regulation has had a catalytic impact.
How do I get started?
ADRIAN: So, where should we begin? Can I get started without the corporate vision statement? Start with business value, at the data manager's desk?
NEELY: Think about key initiatives driving the business. Even compliance is a business issue. If you begin by assessing your current state relative to your key business and IT initiatives, you'll see the biggest and quickest return.
ADLER: You can start anywhere. Executive sponsorship without a crisis is difficult to obtain. Often, programs just start. Nobody's going to build a council; nobody's sure if they're called a steward. They want to build systemic decision making, or control for glossary terms, or metadata, or master data management.
Define your sustainable goals for the program. Do you want to clean up metadata? Drive more revenue? Decide what to measure to demonstrate why you're not achieving your goals today.
When data problems pop up, people think of policy enforcement because policies have already been created. That's way too late; start with the premise that maybe your policies aren't effective. Collect evidence, real facts. Communicate, measure and audit results, and compare them to your goals.
ADRIAN: Lise, how do we collect this data and justify the ongoing effort?
NEELY: Say I need to understand the sources of customer information and how the organization uses them to get insight and create products and services that meet customers' needs. I have to understand the information supply chain, work with the business to figure out available metrics, and improve that information.
ADRIAN: So I don't need a definition for every data element in the organization. I need to know about the ones that relate to this problem and this goal.
NEELY: Right—something I can measure now, with a baseline. As data quality improves, measure it in terms of both data quality and impact on the business.
ADRIAN: Now I engage stakeholders with "I understand why we're having some of these problems that you've been calling me about. Let's talk about how we can fix them."
GOW: Absolutely. Identify root causes and potential solutions. Rally around the fastest or cheapest to fix, and determine the appropriate solution based on business need. Address the bulk of the problem or most of the core issues. Deliver that in a timely and cost-effective way.
Growing governance beyond the project
ADRIAN: Once we've identified a problem, articulated the value of its solution, and found some stakeholder support, how do we go to the more general principle of data governance?
ADLER: You've described a project-oriented approach. But people don't know what they don't know. They aren't aware that there are stewards or a chief privacy officer focusing on governance topics, and so they fix their own problem. There's no coordination. Governance isn't just something you do once in a while. It's a systemic approach to solving problems over time. You won't achieve 100 percent data quality overnight; you must learn from your successes and mistakes over time.
Teach people that even without a formal governance program, you're still governing: you're just not doing it with a systemic process. Monitoring and auditing are your most important tools because they tell you what you know and don't know.
NEELY: What bad thing would happen, and to whom? The policy decision becomes what we can put in place to prevent it, and how we can provide insight so we can take corrective action if something is breaking down. If you apply that approach to an initiative that people really care about—growing revenue, reducing cost, or controlling risk—you'll get a lot of attention.
ADRIAN: Sounds great when we're sitting around talking. Do people really get beyond the project, or are other steps needed to institutionalize this?
ADLER: There are six steps: set goals, define metrics, choose a decision-making model, communicate policy, measure outcomes, [and] audit. But even if it's one person solving one problem, it needs to be recorded. Every decision articulates policy.
Get experts in the room to make a decision you can't make on your own. Everybody gets that epiphany. You know the value of governance in your organization. Nobody has to tell you again.
ADRIAN: Data managers might say, "Sounds great, but I don't talk to those people." How do we shorten the cycle, get them into a room to go after this problem?
NEELY: Start small and build upon success. For example, understanding the banking customer is relevant to withholding requirements. As a result of data governance initiatives that one client took, they were better able to understand the risk profile and holdings across a particular group of customers, and to reduce reserve requirements because they understood their exposure. When word of this success got out, other groups raised their hands, so the project built upon its success and had a very positive effect.
ADRIAN: What kind of skills do data managers need for this?
NEELY: Communication. I've got to understand the purpose, what the business is doing, translate what I'm doing technically into what it does for the business. That's what gets traction. That's what gets attention. That's what gets budget.
ADRIAN: Is this the kind of thing that an outside party can help facilitate?
GOW: It's often critical. Many clients think they can initiate information governance as a self-service exercise. But because of politics and their involvement in the environment, they don't have the true perspective. Consultants can help people understand that they're not alone; competitors have the same problems, and tried-and-true approaches can yield some benefits. What freezes people is thinking they're not going to be able to deliver and they're going to be culpable.
ADLER: When we began the council, we didn't have enough experience, so we built a maturity model—a benchmarking tool. Many engagements and countless meetings around the world later, we know how to do it right. There is a science to doing governance well, and there are disciplines that need to be learned. People like Brett and Lise have the experience and know-how to help customers. Working with a partner will help you get it right. Without that experience and those disciplines, it's harder.
ADRIAN: Steven, let me close with you. What do we need for the next step?
ADLER: We must provide more automation because governance is difficult. You can set goals in the abstract, but today there's no system for keeping them somewhere to be compared to outcomes. "Define your metrics" is nice to say, but there's no automation for collecting this information and deciding which facts are relevant and which ones require policy changes. I can talk about decision-making models, but most firms don't have tools to decide which model to use and when: when to be transparent and open and involve lots of people, and when to use the autocratic model.
Those processes are done today without any knowledge. Until we can provide automation that helps firms in a much more systemic way, we're not going to be able to move this market forward.
Dig deeper into Information management on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Keep up with the best and latest technical info to help you tackle your development challenges.
Software development in the cloud. Register today to create a project.
Evaluate IBM software and solutions, and transform challenges into opportunities.