Most businesses collect data but are unable to use it to generate business value or deliver insights in a timely fashion. Data volume and data types continue to grow, as do the different types of data citizens—ranging from business users to data scientists. As a result, data management and delivery often become critical bottlenecks. Enter DataOps.
DataOps (data operations) refers to practices that bring speed and agility to end-to-end data pipelines process, from collection to delivery. The term DataOps and related concepts are at early stages of awareness and adoption, so many working definitions exist today. Research leaders, like Gartner and MIT, have focused their definitions around improving communication between data stakeholders and implementing automation within data flows and lifecycles to enhance delivery practices. Others are simply describing it as “DevOps for data.”
IBM defines DataOps as the orchestration of people, process, and technology to deliver trusted, high-quality data to data citizens fast. The practice is focused on enabling collaboration across an organization to drive agility, speed, and new data initiatives at scale. Using the power of automation, DataOps is designed to solve challenges associated with inefficiencies in accessing, preparing, integrating and making data available.
It is equally important to know what it is not. DataOps is not: a product; a single event or step; a specific team or person. As a rule of thumb, DataOps methodology or practices you implement should consider interaction between these aspects:
People and Process
DataOps supports highly productive teams with automation technology to deliver huge efficiency gains in project outputs and time. However, to experience the benefits, the internal culture needs to evolve to truly be data-driven. With more business segments requiring and wanting to manage data to drive contextual insights, the time is right to 1) increase the quality and speed of data flowing to the organization and 2) get commitment from leadership to support and sustain a data-driven vision across the business.
This type of transformational change begins by understanding the true goals of the business. How does data inform the decisions and services impacting customers? How can data help maintain a competitive advantage in the market? What are the revenue priorities that data can help us solve?
DataOps leaders will need to align business goals to any pilot project deliverables to demonstrate the linkage between executive stakeholders and the ability to exhibit quick, tangible results. They will also need to define the roles all data citizens play to drive the culture and DataOps practice forward. Each organization has unique needs where stakeholders in IT, data science, the lines of business, and everyone in between need to add value to drive success. What roles each play for your business requires deep collaboration across all functions and commitment to sustainability of a practice.
Technology
Tooling is necessary to support any practice that relies on automation. At the core of DataOps is your organization’s information architecture. Do you know your data? Do you trust your data? Are you able to quickly detect errors? Can you make changes incrementally without “breaking” your entire data pipeline? If you are unsure how to answer these questions, the first step is to take inventory of your data governance and data integrationtools and practices.
As you consider tooling to support a DataOps practice within your businesses, think about how automation in the five critical areas below can transform your data pipeline:
- Data curation services
- Metadata management
- Data governance
- Master data management
- Self-service interaction
Implementing tools can be a tangible way to show progress in adoption of DataOps, but doing so requires a holistic vision. Companies that focus on one element at the expense of others are unlikely to realize the benefits from implementing DataOps practices. The technology conversation and implementation should not live siloed from the ongoing planning regarding people and process. The tooling lives to support and sustain the culture.
Continue learning about the IBM DataOps Program
The shift to adopt DataOps is real. According to a recent survey, 73 percent of companies plan to invest in DataOps. IBM is here to help you on your path to a DataOps practice with a prescriptive methodology, leading technology, and the IBM DataOps Center of Excellence, where experts work with you to customize an approach based on your business goals and identify the right pilot projects to drive value for your executive team.
Accelerate your DataOps learning and dive into the methodology by reading the whitepaper Implementing DataOps to deliver a business-ready data pipeline.