Businesses of all sizes, in all industries are facing a data quality problem. 73% of business executives are unhappy with data quality and 61% of organizations are unable to harness data to create a sustained competitive advantage1. With the average cost of bad data reaching $15M, ignoring the problem is a significant pitfall. 

 To help companies avoid that pitfall, IBM has recently announced the acquisition of, a leading provider of data observability solutions. Data observability takes traditional data operations to the next level by using historical trends to compute statistics about data workloads and data pipelines directly at the source, determining if they are working, and pinpointing where any problems may exist. 

 The data observability difference 

With traditional approaches, data issues are reported by data users as they try to access and use the data and may take weeks to fix, if they’re found at all. Instead, starts at the data source, collecting data pipeline metadata across key solutions in the modern data stack like Airflow, dbt, Databricks and many more. It then builds historical baselines on data pipeline behavior so it can detect and alert on anomalies while the data pipelines run. Automated resolution of anomalies through workflows is then enacted to trigger changes without impacting delivery SLAs. 

 Catching data quality problems at the source helps enable the delivery of more reliable data. Mean time to discovery (MTTD) is improved as issues are detected in real time and pipelines execute instead of reacting afterward. Moreover, mean time to repair (MTTR) is also improved as contextual metadata helps data engineers focus on the source of the problem, rather than debugging where the problem stems from. In this way, monitoring both static and in motion pipelines while delivering high quality metadata enables a faster time to value than would otherwise be possible. 

 Data observability as part of a data fabric will be available to IBM clients through our data fabric architecture. While the data observability capability may be utilized independently, we recommend leveraging a more complete data fabric architecture in conjunction with it to help automate the data lifecycle. In addition to data observability, IBM clients can take advantage of use cases such as multicloud data integration, data governance and privacy, customer 360, and MLOps and trustworthy AI. Data observability will also integrate with these other use cases for improved results where both are applied. For example, multicloud data integration benefits from the ability to resolve data anomalies in real time so that the delivery of reliable data isn’t interrupted, no matter where it resides. Data governance & privacy is helped by having more reliable data with constant management instead of a snapshot approach. Customer 360 is improved when fewer data quality issues make their way to applications and skew customer views. And MLOps and trustworthy AI benefit from the more holistic look at the lifecycle from data to implementation of AI. The net result is a virtuous cycle where each data fabric use case strengthens the other. 

 The acquisition of expands IBM’s end-to-end enterprise observability capability. will be a core component of observability use case alongside IBM Observability by Instana APM and Watson® Studio on IBM Cloud Pak® for Data. Instana delivers end-to-end observability across applications, data and machine learning; provides data observability for dynamic and static pipelines; and Watson Studio provides model observability for reliable, trusted AI across its lifecycle. In this way, they collectively deliver an end-to-end enterprise observability and reliability solution. 

Looking toward the future 

We’re pleased to be bringing into our suite of data & AI solutions. Not only does it signify the continued evolution and improvement of our data fabric approach, but it also brings additional value to clients across the data lifecycle from end to end.  

You can learn more about the IBM’s data fabric solution by visiting our website, starting a trial, or kick-starting your project with the IBM data and AI elite team. 

 Check out the Data Differentiator to learn more about Data Fabric. 


Was this article helpful?

More from Cloud

IBM Tech Now: April 8, 2024

< 1 min read - ​Welcome IBM Tech Now, our video web series featuring the latest and greatest news and announcements in the world of technology. Make sure you subscribe to our YouTube channel to be notified every time a new IBM Tech Now video is published. IBM Tech Now: Episode 96 On this episode, we're covering the following topics: IBM Cloud Logs A collaboration with IBM and Anaconda IBM offerings in the G2 Spring Reports Stay plugged in You can check out the…

The advantages and disadvantages of private cloud 

6 min read - The popularity of private cloud is growing, primarily driven by the need for greater data security. Across industries like education, retail and government, organizations are choosing private cloud settings to conduct business use cases involving workloads with sensitive information and to comply with data privacy and compliance needs. In a report from Technavio (link resides outside, the private cloud services market size is estimated to grow at a CAGR of 26.71% between 2023 and 2028, and it is forecast to increase by…

Optimize observability with IBM Cloud Logs to help improve infrastructure and app performance

5 min read - There is a dilemma facing infrastructure and app performance—as workloads generate an expanding amount of observability data, it puts increased pressure on collection tool abilities to process it all. The resulting data stress becomes expensive to manage and makes it harder to obtain actionable insights from the data itself, making it harder to have fast, effective, and cost-efficient performance management. A recent IDC study found that 57% of large enterprises are either collecting too much or too little observability data.…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters