Composability Is Not a Trend - It Is the New Default Architecture
5 mins
read
l
Saransh Khasria
l
August 7, 2026
Let’s craft intelligent solutions together—turning your data into unstoppable growth.
For years, the standard answer to “how should we build our data platform” was to purchase one comprehensive system and place ingestion, transformation, orchestration, and governance inside it, all owned by a single vendor. That default is breaking down. In its place, a new architectural philosophy has taken hold across modern data teams: composability. Rather than one platform that performs every function adequately, organizations are assembling a stack of specialized tools, each excelling at one function, that can be swapped, upgraded, or replaced independently as requirements evolve.
Enterprise cloud infrastructure spend reached $129 billion in Q1 2026 alone, an increase of 35% YoY and the ninth consecutive quarter of accelerating growth (Synergy Research Group, 2026). That growth is not, by itself, evidence that composability is winning. It does, however, confirm that the underlying infrastructure decisions organizations are making right now carry more weight than ever, which is precisely why the question of which layers should be locked to a single vendor and which should not deserves a deliberate answer rather than a default one.
Three tools have become the backbone of this shift: dbt for transformation, Apache Airflow for orchestration, and a cloud-native governance layer — AWS, Azure through Microsoft Fabric and Purview, or Google Cloud through Dataplex — for metadata, cataloging, and lineage. This article examines why the shift is happening, how the three layers fit together, and where a composable architecture still requires a genuine trade-off rather than a clean win.
Monolithic data platforms made a promise: simplicity through consolidation. In practice, organizations encountered three recurring problems.
Vendor lock-in on the layers that change most often. When ingestion, transformation, and orchestration all reside inside one platform, changing any single component requires re-architecting the whole system. Market share alone does not settle this question — AWS holds roughly 30% of global cloud infrastructure spend, Azure sits around 24-25%, and Google Cloud is close to 13% — but growth rates tell a more dynamic story. Google Cloud grew 63% YoY in Q1 2026, Azure grew 40%, and AWS grew 19% (Synergy Research Group, 2026). Organizations that standardize their highest-churn logic on one hyperscaler’s proprietary tooling are binding their most frequently modified workflows to a market position that is still actively shifting.
Slow iteration. Monolithic systems tend to couple business logic to infrastructure. A schema change or a new transformation rule can require touching multiple layers of a tightly bound system, slowing down the one thing data teams are under the most pressure to deliver — insight, faster.
Opaque lineage. As pipelines grow, tracing where a number originated — which source, which transformation, which job — becomes harder inside a black-box system. When something breaks, teams lose hours tracing the failure instead of resolving it.
Composability addresses the first two of these directly by decoupling transformation and orchestration from any single vendor. The third — lineage and governance — turns out to have a more complicated answer, which is worth stating plainly rather than glossing over.
dbt’s core contribution was not a new way to write SQL — it was bringing software engineering discipline to a part of the data workflow that had historically lacked it. Transformation logic becomes version-controlled code. Every model can be tested, documented, and reviewed like any other piece of software, with changes moving through CI/CD instead of being run manually.
This matters for composability because it decouples transformation logic from the infrastructure that executes it. A dbt project does not depend on whether it runs on Snowflake, BigQuery, Databricks, Redshift, or Fabric’s Warehouse — the logic layer is portable, meaning an organization can change its underlying warehouse without rewriting its transformation rules from scratch.
Adoption reflects how far this pattern has moved into enterprise defaults. dbt Labs surpassed $100 million in annual recurring revenue in early 2025, crossed the 5,000-customer mark, and reported 85% YoY growth in adoption among Fortune 500 companies specifically (dbt Labs, February 2025). Independent estimates put combined dbt Core and dbt Cloud usage at roughly 50,000 teams globally (Forbes, February 2025). In October 2025, dbt Labs merged with Fivetran to form a combined data infrastructure company with close to $600 million in annual revenue (Sacra, 2026). Worth noting directly: a transformation vendor merging with an ingestion vendor is itself a form of consolidation, and it does not fully square with a decoupling argument — the honest read is that individual categories within the composable stack can still consolidate commercially even as the broader architecture stays cloud-agnostic. That is a real tension, not a footnote to wave away.
If dbt owns the “what” of transformation, Airflow owns the “when” and “how” of execution. As pipelines grow — more sources, more dependencies, more downstream consumers — orchestration complexity grows with them. Airflow’s pipelines-as-code model, with DAGs defined in Python, keeps orchestration logic versioned, testable, and extensible in the same way transformation logic is.
Airflow’s adoption scale is substantial: over 30 million downloads per month and more than 80,000 organizations using it in some capacity, up from roughly 25,000 organizations in 2020 (Apache Airflow project blog, April 2025). The Astronomer-sponsored State of Airflow 2026 report, based on responses from more than 5,800 data practitioners across 122 countries, found that 32% of Airflow users already have generative AI or MLOps use cases in production, rising to 62% among Astronomer’s managed Astro customers (Astronomer, January 2026). Among enterprises with 50,000 or more employees, more than half report running Airflow for mission-critical workloads (Astronomer, 2025).
Because Airflow’s DAGs are cloud-agnostic Python, the same orchestration logic can trigger an AWS Glue job, an Azure Synapse pipeline, a GCP Dataflow job, or a Fabric notebook from a single control plane, without a rewrite when a workload moves between clouds. This is the layer that prevents orchestration itself from becoming another form of lock-in.
This is where lineage and cataloging happen, and where the tooling looks meaningfully different depending on which cloud holds the data estate.
On AWS, the Glue Data Catalog provides the technical metadata store, Lake Formation layers fine-grained, tag-based access control on top of it, and Amazon DataZone adds a business-facing catalog with governed data sharing across accounts. On Azure, Microsoft Purview is the unified governance plane, now natively embedded in Microsoft Fabric rather than bolted on — because Fabric centers on OneLake, a single logical data lake underlying every Fabric workload, governance and lineage do not require moving data between services to be tracked. Fabric’s adoption has moved quickly: Microsoft reported roughly 21,000 paying Fabric organizations in mid-2025 (VentureBeat, 2025), a figure that had grown to more than 31,000 organizations by a Microsoft partner briefing in April 2026 (Adastra, April 2026). On Google Cloud, Dataplex unifies data quality, lineage, and cataloging across BigQuery, Cloud Storage, and other engines, with automatic lineage capture for BigQuery-native transformations.
The trade-off worth naming directly: hyperscaler-native governance tools capture lineage with less operational overhead than a separately deployed open-source alternative, because they read directly from the platform’s own execution logs. The cost is portability. Purview’s lineage graph, Dataplex’s catalog, and DataZone’s metadata model are not interchangeable, and a genuinely multi-cloud pipeline will produce lineage fragmented across two governance planes rather than one unified graph. This is the layer where composability, as an architectural principle, is deliberately not applied in full — and that is a design choice, not an oversight.
A representative pipeline looks like this:
Airflow triggers the pipeline on a schedule or event, coordinating extraction, dbt runs, and downstream jobs from one control plane, regardless of which cloud each step executes in. dbt executes transformation logic against the warehouse or lakehouse in use, producing tested, documented, modeled datasets. The cloud governance layer captures technical and business metadata as both Airflow and dbt execute, producing a lineage graph and access-control layer scoped to that cloud’s data estate.
dbt models are portable across warehouse adapters. Airflow DAGs are portable across execution targets. The governance layer can be swapped if an organization migrates its primary cloud, but that swap is the least trivial of the three, since lineage history does not migrate cleanly between Purview, Dataplex, and DataZone. Treating all three layers as equally portable would be a mistake; the value of composability here comes from being precise about which layers earn that flexibility and which do not.
Lower lock-in on the layers that change fastest. Transformation logic and orchestration logic are modified most frequently as business logic evolves. Keeping them cloud-agnostic protects the highest-churn part of the stack.
Room to adapt as hyperscaler growth shifts. With Azure and Google Cloud both growing revenue significantly faster than AWS as of Q1 2026 (Synergy Research Group, 2026), organizations with transformation and orchestration logic decoupled from a specific cloud are better positioned to shift workloads as pricing, region availability, or AI infrastructure access changes.
A known, bounded cost for the governance layer. Because the governance layer is the least portable of the three, cloud selection for that layer deserves treatment as a more durable, harder-to-reverse decision than the choice of transformation or orchestration tooling — with migration effort budgeted accordingly if a multi-cloud or cloud-exit strategy is ever on the table.
Organizations do not need to rebuild their entire stack overnight. The most common path is incremental: move transformation logic into dbt against the current warehouse first, even while orchestration and governance remain on legacy tooling. Introduce Airflow once transformation logic is decoupled — this is the point where pipelines-as-code pays off across ingestion, transformation, and downstream jobs. Adopt the primary cloud’s native governance layer — Purview for Fabric and Azure-centric estates, Glue Data Catalog or DataZone for AWS-centric estates, Dataplex for GCP-centric estates — once multiple tools are in play. Waiting until lineage is already fragmented across tools makes retrofitting considerably more expensive.
The composable data stack is not a rejection of the hyperscalers. It is a specific, deliberate claim about which layers should be cloud-agnostic and which can reasonably be cloud-native, made at the outset rather than discovered under pressure later. dbt and Airflow decouple the highest-churn logic — transformation and orchestration — from any single vendor. The governance layer trades some of that portability for deeper, lower-overhead integration with the cloud where the data actually lives. Getting that trade-off right, layer by layer, is what separates a genuinely composable architecture from a monolith with more vendors in it.
This is also where the right implementation partner matters. DecisionTree Analytics & Services works with enterprise teams across data engineering, data warehousing, business intelligence, and AI/ML to design and implement exactly this kind of layered architecture — choosing where to standardize, where to stay portable, and how to sequence the migration so that lineage and governance are never an afterthought.
