IBM Manta Data Lineage

Visualize the flow of data, from origin to consumption

Book a live demo Contact us

IBM® Manta Data Lineage is a data lineage platform that increases data pipeline transparency so businesses can determine data accuracy throughout their models and systems.

As businesses integrate AI into their workflows and data becomes more complex, data quality, provenance and lineage are increasingly important. In fact, IBM’s 2023 CEO study found the number one barrier to generative AI adoption is concerns about the lineage of data. 

IBM offers an automated data lineage platform that automatically scans your applications to build a powerful map of all data flows. The platform then delivers the info through a native user interface (UI) and other channels to both technical and nontechnical users.

With IBM Manta Data Lineage, data operations teams get comprehensive visibility and control of their data pipeline. By improving your understanding and use of dynamic metadata, you can ensure that data is managed efficiently and accurately across complex systems.

Watch demo video
Benefits Reduced time for impact analysis

By automating the process of collecting existing data elements and illustrating their interconnections across the data ecosystem, the IBM Manta Data Lineage platform can significantly decrease time spent on impact analysis.

Boost in Data Ops productivity

The platform enhances productivity and efficiency by leveraging metadata and the history of its changes. It uses a code-based approach that eliminates common lineage errors created by manual lineage processes.

Scalability

IBM Manta Data Lineage is adept at data lineage mapping even in complex and extensive tech environments. The platform can handle large volumes and the discovery and illustration of complex transformation expressions.

Granularity

The platform allows for a detailed view of data pipelines with a step-by-step flow analysis that incorporates color-coding, dynamic filtering and historical lineage at the column and attribute level.

Risk mitigation

The platform provides the ability to address compliance issues related to risk models, data privacy and regulations such as GDPR.

Vast technology coverage

The platform supports open standards, enhancing metadata management and future-proofing data processes, with support for over 50 technologies, programming languages, relational databases, ETL and various modeling tools.

Features Automated data lineage

Automates scanning and mapping of data flows. 

Active tags

Improves visibility and issue resolution in data pipelines with this industry-first feature.

Enhanced collaboration

Facilitates better collaboration between technical and business users. 

Efficient error tracing

Enables tracing of errors to their origin and quick resolution. 

Metadata management

Incorporates business, technical and operational metadata for comprehensive data understanding. 

Support for various environments

Supports over 50 technologies, including various operating systems, and is compatible with hybrid and multicloud environments.

 

Capabilities
Out of the box 3rd party integrations

Push harvested lineage into your data catalog by using any of the supported integrations.

Full automation

Achieve fully automated, code-level lineage that seamlessly integrates into your applications.

Extensible architecture

Build your own metadata ingestion process without a formal scanner by using the platform's ability to easily extend lineage to current and to-be-implemented applications and solutions.

Historical lineage

Discover and act on changes in your data pipelines. Use the data pipeline audit trail for troubleshooting and root cause investigation.

Robust, flexible visualizations

Explore, navigate, filter, aggregate and tune lineage visualization into different levels of detail and granularity.

Open standards

Receive support for evolving metadata standards such as OpenLineage.

How it works

There are 4 steps to implementing IBM Manta Data Lineage

This infographic shows you how the Data Lineage tool works to gather your data and contextualize it, and how you can adjust the filters and the level of detail you want to see in your workflow.

Step 1 Harvest lineage Deploy Data Lineage’s connectivity module to gather metadata from mission-critical and analytic systems in your hybrid, cloud or on-premises environment. Use the OpenManta framework with APIs to enhance your metadata even for custom-built systems not supported by existing scanners. 

Step 2 Contextualize lineage with semantics Data Lineage adds semantics to enrich the attribute-level lineage with indirect data dependencies, transformation logic, evolution over time or external metadata such as profiling information, quality scores, PII labels and more.

Step 3 Tailor lineage to your needs Adjust the lineage to the level of detail for specific use cases. Search through all the lineage and use intelligent filtering to hide details that are not currently relevant. Create domains and perspectives for specific projects, review historical versions, get notified of and act on changes.

Step 4 Activate lineage Explore lineage with the native interface and integrate it into your workflows by using open and robust APIs. Improve your data quality, data privacy and data governance processes. Integrate with CI/CD pipelines to boost the productivity of data engineers. 

Use cases 

See the results of IBM Manta Data Lineage in action in different contexts such as data pipeline analytics, data governance and compliance, cloud migrations, and mergers and acquisitions.

Data Ops

Amplify value 

Gain agility through automation by starting with a healthy functional data pipeline. IBM can streamline impact and root cause analyses, speed up incident resolution, reduce the cycle time of data analytics and increase the value of analytics.

Put Data Lineage to work for improved production quality

Data teams often dedicate a significant portion of their resources conducting manual impact and root cause analyses. These analyses, while critical to preventing broken releases and recurring incidents, are often time-consuming due to the growing complexity of data systems. You need the ability to fully understand your data pipeline and dependencies without hundreds of hours of manpower spent.

IBM can help you reduce that manual effort without sacrificing quality

You can improve your change management with fully automated impact and root cause analysis, incident resolution and debugging, freeing up your team for bigger projects. Use the visibility feature to see how a planned change will potentially influence other parts of the environment. Easily trace any data-related issue to its source and remove it right away—all while preventing it from happening again in the future.

Increase productivity

Save time, energy and money with full visibility of your data pipeline. Fully automate impact and root cause analyses so they require minimal manual labor, freeing up your team for bigger projects.

Get faster incident resolution time

Easily trace any data-related issue back to the source and remove it right away—all while preventing it from happening again in the future.

Reduce broken releases

Act proactively with accurate automated impact analysis that creates immediate visibility of how a planned change will potentially influence other parts of the environment.

Data governance and compliance

Enable consistency, accuracy and trust 

You already have the data that you need. Manta Data Lineage can help you tap into the power of that data. IBM’s automated lineage will enable you to build trust in data across your organization and maximize the value of your data governance framework.

Your data management and governance strategies rely on healthy data

Compliance can be a pain if your data is inaccurate, incomplete or missing. Data accuracy is critical in ensuring protection against non-compliance events, of which a single event can cost organizations an average of USD 4 million in revenue (a 45% global increase since 2011) (link resides outside of ibm.com). You need an immediate, complete and comprehensive overview of all your data flows, sources, transformations and dependencies to take control of your data assets.

IBM can help you increase trust in your data, ensure accurate reporting, see a clear and easily adjustable overview of how crucial calculations were derived, and improve your data governance framework to reinforce your overall data management strategy—all at an enterprise-wide level.  

Automated lineage discovery

Say goodbye to the costly, lengthy, manual processes of lineage collection and updates. Save costs in the initial phase of implementation with IBM’s automation. 

Improve accuracy with an added semantic layer

Put an end to second guessing. Make more informed decisions with accurate reporting and forecasting through detailed lineage with an added semantic layer. 

Create full transparency for improved trust

Access end-to-end lineage that can be adjusted to every user’s technical understanding to increase data transparency and trust, bringing greater empowerment to all. 

Enable full regulatory compliance 

Strict regulations? IBM also provides a complete overview of the regulated data that your organization is processing. This helps you meet your regulatory requirements and avoid hefty penalties for non-compliance. 

Cloud migrations

Expedite initiatives

Take advantage of seamless cloud or hybrid migrations with complete data lineage. Set the right expectations, stick to the plan, save resources, and escape the do-break-debug-redo cycle.

Keep your data migrations on track

Cloud and hybrid environments are critical for digital transformation efforts.  However, changing your data environment is easier said than done.  In fact, according to Gartner (link resides outside of ibm.com), 83% of data migrations exceed budget, fall behind schedule or fail. Moreover, 80% of organizations that conduct a lift-and-shift migration do not achieve meaningful cost savings.

Don’t let your project become a statistic.  With IBM’s enterprise-wide automated data lineage platform, you can avoid falling victim to the common and complex pitfalls during migrations and save time, money and resources.

Reduce project costs

Every migration starts with a plan, but with IBM’s enterprise-wide data lineage that runs on all platforms, it’s finally complete. No blind spots, no muddy waters.

Harness better predictability

Understand the impact of your data on each stage of the migration, including during the testing and debugging phase, which can cut the time spent on the analysis phase.

Leave nothing behind

With data lineage, each object in the migrated system is mapped and its dependencies are clearly documented for a seamless transition. 

Mergers and acquisitions

Generate greater value

McKinsey Global Institute estimates that companies could generate USD 9.5 trillion to USD 15.4 trillion in business value (link resides outside of ibm.com) by investing in artificial intelligence tools such as data lineage.

Streamline M&A processes

Accelerate integration and data consolidation while understanding the level of compliance in inherited systems.  Avoid surprises in your merger or acquisition. Mergers and acquisitions happen for many reasons, and the context is always changing.

IT teams may be going into their new data environment completely blind, unsure of what applications, processes and data they inherited. Even in cases of the friendliest agreements, the acquiring company might still be blindsided by the sheer quantity of scripts in the new environment can be overwhelming and expensive to handle. In most merger and acquisition situations, manual data lineage is overwhelmingly expensive.

Scan and process nearly any environment

IBM’s automated data lineage platform uses metadata to build a map of how data is transformed, used and processed in your environment. It can detect changes within data tables to show you how they affect downstream applications. You can highlight relevant information in the context of the data pipeline with color-coded active tags.  With over 50 scanners available, you can visualize the movement of data in almost any environment and save time and money by finding areas where revisions are needed. Moreover, lineage delivers visibility at multiple levels to provide understanding across all departments and job titles.

More use cases

Organizational challenges

Data lineage value

IBM Knowledge Catalog with Manta

Decreasing trust in reports and AI

Trust in data

Build trust in reports with a business-friendly lineage summary view with the ability to drill into technical details.

Slow delivery of new insights

Impact analysis

Connect to 50+ technologies for automated lineage capture to complete in-depth analysis and prevent data incidents.

Regulatory compliance audits

Regulatory compliance

Trace sensitive data by using business terms, classifications and other governance artifacts at the column level to ensure regulatory compliance.

Growing number of data incidents

Root cause analysis

Monitor changes in data flow with historical lineage to resolve data pipeline failures and remediate data quality issues.

Architecture migrations

Application modernization

Application modernization

Scan to-be-modernized systems to identify dependencies. 

Industries

See how the IBM Manta Data Lineage platform can improve the power of your current data by assisting your organization with compliance, data management, accuracy, risk and impact analyses, automated workflows and easy-to-view data maps across various industries.

Finance

If you’re operating in the financial services industry, you’re probably facing a growing list of data-related regulations, such as GDPR, CECL, IFRS 9, BASEL III and BSA/AML.

Additionally, you can be migrating away from a legacy system in favor of a hybrid or cloud environment. With IBM’s enterprise-wide data lineage platform, you can gain the trust that you need in your data for regulatory compliance, data governance, cloud migration efforts and more. You can also automate routine tasks and enable self-service wherever possible, reducing the burden of manual tasks on your data engineers and developers. Improve your data governance with a complete view of all your data flows, transformations and dependencies.

Read the blog
Healthcare

Healthcare data ecosystems are sensitive, diverse and interconnected, integrated with many applications, microservices and infrastructures. Those connections rely on countless dependencies, with the nature of those dependencies obscured in a “black box” for most users, challenging their data accuracy.

IBM’s data lineage tool illuminates data dependencies, tracking the journey of data as it moves through complex systems and undergoes various transformations along the way. This information can help you to make crucial decisions for your patients while keeping them safe from breaches and regulatory compliance violations. IBM’s impact analysis and revision comparison capabilities can help you prevent mismatched and other entity conflict problems during electronic health record (HER) system updates.

IBM Data Lineage can find missing or broken tables that are needed to properly bill insurance companies and locate missing or incomplete patient data that can impact clinical decisions. You can also create automated workflows for your team, improving productivity.

Read the blog
Insurance

If you’re operating in the insurance industry, you’re likely grappling with massive amounts of protected data. Or maybe you are facing a growing need to migrate away from traditional platforms that are no longer efficient at processing claims, but you can’t wait for months.

With IBM’s data lineage platform, you can gain the visibility into your data environment that is crucial for not only efficient migration efforts, but also regulatory compliance, data governance and more. Use data flow maps to see exactly where data from insurance claims originated from and how it changed. Integrate your current data and use it to maximize your data governance framework.

Pharmaceutical

You likely have millions or even trillions of pieces of data in your pharmaceutical company. This data may relate to clinical trials, pharmacy and distribution, formulations and patents. It’s likely that each type of data contains variations, including differences in currency, units of measurement, volume of data needed, patent requirements and stages, FDA approvals and related regulations.

This data is required to make critical, often life-saving decisions when getting medication formulated properly, manufactured and distributed to the right people quickly. Data lineage can help you untangle your complex data environment so you can trust your current data and make better, safer decisions. IBM Manta Data Lineage uses automated lineage to help you stay compliant and keep all patents up to date and filed with complete and accurate information for faster FDA approval.

You can view data samples based on specific characteristics that profiling discovered. For example, locate a missing percentage of values in a column to understand if the data set can be used. Design lineage can also help you map out changes to reference data that should be valid from a certain date.

Resources What is data lineage?

Read more about how data lineage helps you improve your organization’s data use, analysis and governance.

Your guide to data lineage

Get to know why IBM Data Lineage is a must for your data toolkit by understanding what data lineage is, why it matters, how to activate metadata and how to create data lineage.

Trust your data

See how data lineage helps you understand the power and potential of your data from end to end.

Manta Data Lineage end user orientation

Learn how to use the IBM Manta Data Lineage interface and features to visualize, interpret and compare revisions of your data in only 2 hours.

How to use IBM Manta Data Lineage to navigate industry compliance

See how IBM Manta Data Lineage helps you build a data governance framework for compliance.

The six benefits of data lineage for financial services

Discover how data lineage helps financial services with compliance, data accuracy and analyses, cloud migration efficiency and more.

Related products IBM® Knowledge Catalog

Activate data for AI and analytics with intelligent cataloging and policy management.

IBM Databand®

Deliver trustworthy and reliable data with continuous data observability.

IBM DataStage®

Build a trusted data pipeline with a modernized ETL tool on a cloud-native insight platform.

Master data management

Drive faster insights by delivering a comprehensive view of an entity’s data and relationships across the enterprise data fabric.

Take the next step

Explore how IBM enables the creation of a governed, compliance-ready data foundation. Implement data transparency with IBM Manta Data Lineage today so you can see your data history, flow and results to make it work for you from end to end. To learn more about IBM Manta Data Lineage, book a demo.

Book a live demo Contact us
More ways to explore Documentation Lineage (IBM Knowledge Catalog)