As a data leader, you know that deriving value from data comes down to providing the right data at the right time—regardless of where it resides. That ability hinges on having a modern data architecture in place as part of your data strategy.
A fit-for-purpose data architecture translates business needs into data and system requirements and manages the protection and flow of data through an organization. Keep in mind that it’s not a one-size-fits-all formula. The framework should be driven by the business requirements and support short-term and long-term objectives. “Gone are the days of a single, structured, data-at-rest architecture,” says Paul Christensen, data elite architect, IBM Expert Labs. “Today’s businesses are driven by data in motion and at rest, data in many forms, and data in varying degrees of quality and trust.”
With data distributed more than ever both on premises and in the cloud, data architecture solutions are essential for meeting the specialized needs of the business, applying data analytics and using data and AI at scale. For most organizations today, a modern data architecture isn’t just an option—it’s an urgent necessity.
How do you find and determine those specialized needs to select the right technology? A data topology helps you classify and manage real-world scenarios to build a modern data architecture that considers the users, use, constraints, and flow of data and is highly resilient to future needs.
Meet watsonx
Sign up for AI updates
The rise of cloud modernization will not necessarily reduce complexity or cost, remove data silos or manage governance and compliance. In fact, research shows that 68% of data is going unused.¹
Enter data fabric, an architectural approach to simplifying data access and facilitating self-service data consumption for better decision-making. A data fabric includes the appropriate controls to support the required data flows, processes and consumers of that data within an organization. This modern data architecture smooths the way for the end-to-end integration of various data pipelines and cloud environments through intelligent and automated capabilities.
The foundation of a data fabric is federated active metadata—often referred to as the data that describes data. Databases or data sources and targets are also key components. Those sources need to be selected based on their capabilities to support whatever workload is required, whether it’s transactional, operational or hybrid transactional and analytical processing, and involving AI, business intelligence, reporting or advanced analytics.
“Customers might have up to nine different database types, and many instances of each. A data fabric brings order to those data silos and data fragmentation that customers are trying to manage,” says Edward Calvesbert, a product leader for the database portfolio at IBM.
Through a virtualization layer, a data fabric pulls together real-time data from multiple sources, including existing systems, databases, data lakes, data warehouses, edge and in-memory repositories. These sources may run transactional, operational or analytic workloads and store structured and unstructured data types. This orchestration provides a centralized reach across all points of your data landscape.
With these end-to-end capabilities, a data fabric helps ensure your data from various sources can be successfully combined, accessed and governed so that business users, data scientists, data engineers and data analysts can put data to work. It also allows for innovation at scale in areas such as AI by providing governed data sets to fuel your AI applications.
We’ve talked about data fabric. But what about data mesh, another approach that streamlines enterprise-wide use of data in a data-driven architecture?
Data fabric and data mesh are both data architecture concepts. Each follows a use-case-driven design and seeks to solve the challenges of data sprawl, data governance and data availability. Data fabric and data mesh approaches also both rely on ongoing data discovery and self-service data knowledge catalogs. The good news is that these data architecture concepts are complementary.
The differences? Data mesh architectures are domain-specific and technology-agnostic, designed for analytic use cases. By comparison, data fabric architectures are designed for both operational and analytic use cases. While data fabric provides a unified view of all data assets, the actual data storage may be decentralized, centralized or a mixture of both. Likewise, data fabric architectures support multiple organizational structures, from federated to distributed. Finally, data fabric architectures use artificial intelligence and machine learning technologies to automate data discovery, data classification and policy enforcement.
Now that you’ve seen the potential of a data fabric architecture, explore these use cases to narrow down the best area of focus to meet your organization's objectives.
1 Rethink Data: Put More of Your Business Data to Work – From Edge to Cloud (PDF, The link resides outside ibm.com), Seagate Technology, July 2020.
2 “Data Integrity Trends: Chief Data Officer Perspectives in 2021 (The link resides outside ibm.com). Corinium, 18 June 2021.