InfoQ Live July 19: How to Successfully Build and Deploy Applications on the Cloud? Register Now

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Avdi Grimm describes the future of development, which is already here. Get a tour of a devcontainer, and contrast it with a deployment container.

Fran Mendez discusses event-driven or asynchronous APIs, comparing AsyncAPI with OpenAPI/Swagger, AMQP/MQTT/Kafka with HTTP, and publish/subscribe with request/response.

Lola Priego, Jose del Pozo discuss how they improved the user input accuracy, normalized lab data using a scoring algorithm, and how this work finishes with an AI to diagnose health.

In this podcast Shane Hastie spoke to Rachel Obstler of Heap Analytics about product-led growth, designing for customer self-service and how high growth products are constantly experimenting.

This article shares how Apache DolphinScheduler was updated to use a more modern, cloud-native architecture. This includes moving to Kubernetes and integrating with Argo CD and Prometheus. This improves substantially the user experience of deploying, operating, and monitoring DolphinScheduler.

Learn how to migrate an application to serverless and what are the common mistakes to avoid. Register Now.

Learn how cloud architectures help organizations take care of application and cloud security, observability, availability and elasticity. Register Now.

Understand the emerging software trends you should pay attention to. Attend in-person on Oct 24-28, 2022.

InfoQ Homepage News Six Governance Topologies for Data Mesh

Piethein Strengholt, author of Data Management at Scale, recently published an article presenting six data-mesh governance topologies and domain granularity. Each topology adapts the data mesh strategy to balance requirements like data ownership, organization structure, pace of change, technology, and others.

Data mesh is an enterprise data architecture that adapts and applies the learnings in building distributed architectures to the domain of data. Data mesh recommends creating self-serve data infrastructure, treating data as a product, and organizing teams and architecture based on business domains.

According to Strengholt, organizations transitioning to data mesh must make trade-offs. Depending on their context, they could value quality over control, simple over complex structures, etc. Organizations must adapt the data mesh concept to implement what works best for them today and use one of the following topologies as a starting point:

Source: https://towardsdatascience.com/data-mesh-topologies-and-domain-granularity-65290a4ebb90

Applying the purest theoretical form of data mesh looks ideal, but it is hard to achieve because it requires standardization and managing concerns about capability duplication, network access, etc. Strengholt named this implementation a fine-grained, fully federated mesh topology. It is often used in organizations born on the cloud, that are relatively young, and have highly skilled software engineers.

Organizations that value quality and compliance over agility can implement the fine-grained and fully governed mesh topology because it adds a central distribution layer to address distribution concerns. In Strengholt's experience, many organizations, particularly financial institutions and governments, tend to use it. However, organizations with hard-to-maintain legacy systems or lacking highly skilled software engineers can implement the "hybrid federated mesh" topology where federation happens on the consuming side domains and centralization in the others.

Organizations in supply-chain management, product development, or transportation industries require a high level of specialization. The value-chain-aligned mesh topology addresses this scenario by grouping domains that work together. Each value chain will behave like a larger domain, and crossing its limits requires data products to adhere to centralized standards.

Large-scale organizations with complex architectures and many applications require multiple governance levels, alignments, and decompositions, Strengholt wrote. Implementing some data mesh will require more trade-offs. In the coarse-grained aligned mesh topology, domains hold many applications, and boundaries are set by mirroring business units or regional viewpoints. This topology contradicts a pure data-mesh implementation and introduces new risks like creating silos.

Finally, some large-scale organizations can implement the coarse-grained governed mesh to overcome complexity, peer-to-peer distribution, and interoperability deviation. This topology adds a central distribution layer and relaxes controls within larger boundaries.

Becoming an editor for InfoQ was one of the best decisions of my career. It has challenged me and helped me grow in so many ways. We'd love to have more people join our team.

The Open Stack for Modern Data Apps. Kubernetes Based. Cloud Delivered. Developer Ready. Sign up to try Astra for free.

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

You need to Register an InfoQ account or Login or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

Real-world technical talks. No product pitches. Practical ideas to inspire you and your team. QCon San Francisco - Oct 24-28, In-person. QCon San Francisco brings together the world's most innovative senior software engineers across multiple domains to share their real-world implementation of emerging trends and practices. Uncover emerging software trends and practices to solve your complex engineering challenges, without the product pitches.Save your spot now

InfoQ.com and all content copyright © 2006-2022 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with. Privacy Notice, Terms And Conditions, Cookie Policy