As businesses settle into hybrid working, how can leaders better support employee wellbeing? Helena...

The need to integrate things, people, teams, resources, business units, offices and everything...

What advice would you give to aspiring security leaders? “Take your time.”

Peerspot users review the highest reviewed solutions in the cloud data warehouse market.

The metamorphosis of data in driving more intuitive solutions has made business leaders begin to understand the importance of continuously building or exploring data management strategies. Though Gartner listed data fabric in its top trends for data analytics in the 2021 report, data experts believe that even the data mesh architecture has tapped into the potential of leveraging data to fetch more valuable and actionable business insights. These architectures enable enterprises to handle and share different data from heterogeneous data sources.

In the data-connected world, the two are used interchangeably, thus, it is crucial to understand the fundamental difference between the two strategies to incorporate the right one.

Expersight defines a data fabric as “a design concept which serves as an integrated layer of data and connecting processes. A data fabric utilises continuous analytics over existing, discoverable and referenced metadata to support the design, deployment and utilization of integrated and reusable datasets across all environments, including hybrid and multi-cloud platforms.”

Technology centric: The data fabric architecture integrates data in the existing infrastructure and adds a layer of additional operating technology layer that integrates all the data together and prepares the data to get meaningful insights proactively.

Data is a by-product: The data fabric architecture identifies and integrates real-time data from different sources to a specific destination or location through data-integration technologies. Thus, this architecture treats data as a by-product of the process of moving and integrating data at a centralised location.

Centralised: Data fabric provides a single unified platform to coordinate data management across multiple sources and technologies. This architecture helps to enhance data governance, establish standardised security policies for all connected APIs and ensures homogenised protection across different data end-points. It offers centralised data security and governance policies which are implemented consistently across varied environments.

Undistributed ownership: Data fabric provides a single unified platform for handling data across multiple technologies. The centralised form of integrating different data and systems sometimes formats data differently and thus downstream data consumers such as data scientists, data engineers, and data analysts. However, what is to be noted is that data management is unified and not the actual storage. 

Data mesh architecture is a new strategy of decentralising the data and bringing ownership to each business domain such as sales or customer support. The objective of data mesh is to establish coherence between data coming from different domains across an enterprise.

Organisation Centric: Data mesh focuses on bringing organisational change in the data architecture and extrapolates the current infrastructure with new deployments in business domains. Thus, brings a smarter and more competent way of utilising human efforts in data management.  This organisational-centric approach helps in the production of analytical data for easy consumption by teams who are the most competent in performing it. 

Distributed Ownership: The data mesh architecture devolves the ownership of creating and handling the data pipelines to the ones who are closest to the data i.e., domain experts. Thus, domain experts ensure that data consumers, like analysts and engineers, across the business access quality data. This helps in standardising data format for exposing and integrating data from varied sources. This enables interoperability and harmonises data.

Decentralised: The decentralised approach enables data consumers to access and query data without depending on the data to be integrated to a source point first, thus, they can leverage data as and when they need it. This helps to avoid the bottlenecks of a centralised data management team of a single enterprise-wide data warehouse or data lake that stores all data coming from varied sources.

Data-as-a-product: Data mesh supports distributed data topology with each domain handling its own data pipelines. This change is significant as every business domain adapts software product thinking and views data-as-a-product. This enables domain experts to conform to standards while producing and sharing datasets for addressing a specific business.

Both concepts ultimately ensure that business data from disparate sources is readily available for downstream data consumers for fetching valuable and actionable insights, and thus can co-exist. The infrastructure analysis of an enterprise is helpful in determining the right approach. For instance, a data decentralised approach might work better for enterprises having operational maturity and mutable data flowing from multiple sources.

Platforms like the Data Product platform offer a unique approach to incorporating the concept of both architectures to proving the best results to the enterprises. This platform aggregates and manages data of all the business entities (such as a customer, product, location or order) and further prepares and delivers the data as an integrated data product to the data consumers. The integrated data provides a real-time view of all enterprise data and supports modern data architectures. The platform delivers business outcomes swiftly and in a cost-effective manner by driving varied use cases, including Customer 360, cloud migration, test data management, data privacy, legacy modernisation etc. 

Vendors like K2view, IBM, Stardust and Oracle offer enterprises modern data management solutions for fast access and querying of the many types of data.

Redesigning conventional data infrastructure requires enterprises to adapt to changes at both technological and organisational levels. With data management platforms in the market, enterprises can make more informed decisions on the data management strategy to be implemented. As discussed above, both Data Fabric and Data Mesh offer powerful solutions for collecting and integrating data. Businesses can choose data from disparate sources for enhanced decision-making. Thus, data fabric and data mesh, for best results, should be used as complementary technologies.

Copyright © 2022 IDG Communications, Inc.