The Builder: Outliving the Modern Data Stack

The software engineering that never was

The data field is cursed by management decision making. From the start of enterprise computers, databases fell under management purchasing decisions while programming runtimes fell under devs. This incentivized open, best standards for software, and vendor locks for data. ELT pipelines, where software developers would write pipelines (in C, Java) to load data to databases like Oracle were for a long time the norm in how data made it into these systems - but this was the “enterprise data era”.

Come 2010s, suddenly analytical databases were making their way into SMEs through the cloud, opening up a new market that no longer could rely on the same Java developers to build ELT connectors.

This renaissance brought about the ELT revolution - suddenly those analytical environments made their way into business units where there were no Java devs. A true grassroots movement emerged, where data-curious people from various backgrounds started becoming “end to end Business intelligence managers”, “data engineers”, “data scientists” - a new breed of business programmer. These guys are often SQL first, scripting second and so transforming data in SQL, which became a managed parallel cloud runtime became the path of least resistance.

I entered the field around this time. Having finished business studies and with a brief stint in tech support, in 2012 I was a “data analyst”, later a “Business intelligence manager”, and by 2018 a “data engineer”.

From engineering to sexy data science

Around 2018 in Berlin, the data space was becoming popular and I was seeing perhaps 10x more companies hiring data people than back in 2012, with ever increasing scopes like streaming, big data, data science.

I was now hiring for a data team, and I was getting a ratio of 30:1 Analyst to engineer applications (around that time it dawned on me that we need dlt, for those python first non-engineers).

The new world now looked very different from the java ETL and oracle databases of the past. We now had 2 branches in data, each with their own specialists - “business intelligence” and “machine learning”.

The modern data stack

Rent a few connectors, slap some dbt on top, add tableau or something that doesn’t make you scream when working, and what do you have? A stack of blocks we give to analysts so they can feel like engineers without touching infra or code. A “safety over freedom” playground.

But Builders need access, control and customizability, while these building blocks are restrictive, unmanageable, and manual to set up.

Data platform engineers had the right idea

It usually takes some time to understand the shortcomings of a new approach. In the case of data teams, we learned that democracy doesn’t mean freedom to make chaos. You see, democracy is a form of governance, not of freedom - freedom without governance is anarchy.

In platform engineering, we tried to solve this by building “golden paths”. We centralized the complexity and observability and enforced the rules through infrastructure. The logic was sound: centralized governance to allow decentralized speed.

Unfortunately, a platform team requires a number of skills that are not easily found or developed, so while this approach works, it’s too expensive for the mainstream.

Building becomes magic (the collapse of friction)

Finally, LLMs arrived and changed the physics of building in ways that majorly impact the data field.

• Acceleration: The "imagination-implementation gap" for working prototypes collapsed to a fraction of the time it used to take. In data, most products are “R&D vehicles” or working prototypes that only get upgraded if the core value of that is higher than the cost.
• Consolidation: If being "cross-functional" took years of career work, in 2026, an engineer can write SQL, build a React frontend, and deploy a Python model in one afternoon because the LLM handles the syntax.
• Abstraction: Writing code is now democratized. This forces us "up" the driver or specialist chain. You can't make a career just by knowing where the bricks go anymore, you have to know how to design the whole building so the LLMs can converge it into reality.

The diagram above shows the evolution of data roles from a "Generalist" past to a "Specialized" middle era, and finally to a fractured future.

• 2010 (the democratization): The job was simple. If you could script (python/sql/shell/batch/ruby/r), you did everything: Ingestion, Analysis, and Reporting. Scope was small, teams were small and the “team of one” were common.
• 2018 (the great specialization): The "Modern Data Stack" era. We fractured into distinct roles: Data Engineers (python, infra), Data Scientists(python, stats), and Analysts (SQL/reporting). This was the era of the "Golden Path" and comfortable specialization.
• 2026 (the great Schism): The market splits again, but this time based on belief rather than tooling.
  • Consumer: those who lack a "Builder Mindset" and rely on pre-built black boxes are hitting a dead end. They are being squeezed out by automation used by people who can deliver more with less.
  • Builder: the return of the generalist, powered by AI. This "team of One" handles data engineering, AI, and DevOps with a pinch of frontend end-to-end.

If we are to look at the originators of the modern data stack, dbt's Tristan himself says indeed MDS term has outlived its usefulness because you only really need analytics. "Let’s move on. Analytics is how I plan on speaking about and thinking about our industry moving forwards" he says. And how far away is a proprietary stack from adding a semantic layer with chat-bi and commoditizing the analysts too?

Meanwhile, here I sit at the builder's side, combining AI with the classic data stack to replace analytics. What we do see going away is tools that add impedance - because now, my peers build end to end pipelines in Cursor. The fracture feels tangible. I see it more like Benn, analytics is a function.

While I collected my information from our users, trends and job posts, it’s certainly biased, so I asked the hivemind where we stand to get a differently biased sliced. The diagram struck a nerve, with over 300 upvotes overnight. Of course, as we say some companies live in the past, and according to the comments it looks like the split is about 2/3 in the 2026 bucket and 1/3 in the 2018 bucket with nobody from dinosaur legacy times finding their way to reddit.

The sentiment is summarized as follows:

• Acceptance: Despite the complaints, many agree the shift is real. As one commenter put it, "Infrastructure knowledge is a must for any developer... just because you don't do DevOps doesn't mean you don't have to know it".
• "It's a Mirror": Many engineers feel this diagram perfectly captures their current reality. One user noted, "It's like I'm looking in a mirror", while another described their last sprint as doing everything from ECS deployments to dbt work, admitting, "I have no clue how to define myself".
• Fierce competition and unreasonable requirements: There is a strong sentiment that this "Full Stack" trend is just a way for companies to get a "10x engineer for 1/10 budget". Users joked that their title should be "Full-Stack Data Engineer Solution Architect Enabler". The thread highlights the absurdity of job descriptions, with one user mocking listings that expect "20 years of hands-on experience in... 25 big data ML AI GenAI techs".

The Builder’s Dilemma: 10x Expectations, 1x Tooling

The Reddit crowd isn't wrong. If you look at 2026 through the lens of 2018 capabilities, it looks like a nightmare. Asking one person to manage ingestion, transformation, DevOps, and frontend using the heavy, disjointed tools of the "Specialization Era" is a recipe for burnout.

But there is no going back. The shift to full stack is an inevitable consequence of the friction collapsing.

To collapse this friction and survive as a full stack builder, you have to stop building houses of cards. You need tools that treat pipelines as AI-enabled software, not as tax-o-meters.

The "Modern Data Stack" prison fortress

The modern data stack nowadays has come to mean many things, some good and some bad, but at its origins, the “modern data stack” story is a multi million dollar marketing machine written by the vendors of the old age, commonly attributed to Fivetran and sometimes to dbt Labs.

Democratisation of data access sounds nice on paper, but the dark side effect of this is how the story served to usurp builder’s abilities in teams. By promising democratization, they turned non-technical stakeholders into internal mouthpieces. These consumers became the loudest advocates for the stack because it granted them a title without requiring the ability to program or the rigor of engineering. To the vendor, a dependent consumer is the perfect customer: they don't ask for the source code, they don't know how to optimize the "usage-based" bill, and they are entirely dependent on the vendor to keep their jobs. And in my experience, for many of these professionals being SQL-only has become a box that professionals are forced into, often because the tools they use don't provide a path to grow into deeper engineering practices.

By making it easy for everyone to consume, vendors earned more. Spaghetti or thousands of SQL models? All good as long as it’s making bank. Too complex or other problems? Just consume more! Hire more! Buy more tools! Run more tests!

And the real problem isn’t that you’d be paying a vendor per row to load data - but that you’d be rejecting the culture of capable builders that can operate outside vendor sandcastles. After all - ingestion pipelines are among the simplest forms of automation - if that’s not doable in your team, then you stick to reporting like it’s 2010?

We spent the last few years using a "Modern Data Stack" (MDS) designed for a consumer. Low-code UI, drag-n-drop connectors, and safety-first SQL orchestrators. We optimized for people who didn't want to touch infrastructure or Python and built cages that look like fortresses.

• Rigid: You can’t version control a drag-and-drop setting easily.
• Black-boxed: When it breaks, you can’t fix it; you have to open a support ticket.
• Anti-automation: You can’t ask Copilot to script through the UI.

But code is the only interface that scales with AI. If you want to leverage the acceleration of LLMs, your stack must be defined in code.

.

AI Barbarians at the gates

But AI brought the walls down. Barbarian builders are at the gates and they’re strong, because they practiced. They don’t wait for permission from finance and they don’t need to be “enabled”. They build faster than that. The prison fortress doesn’t stand a chance. The vendors scream for 'Governance' to protect their walls, but what they really fear is the transparency of code, because you can't charge a gatekeeper's toll on a bridge you don’t own.

Reclaiming the "Builder" Identity

We are seeing the emergence of a class of data professionals who are tired of being treated like consumers.

• We are Builders. We don't want training wheels and child locks.
• We prefer text over GUIs. Because text is searchable, diff-able, and generatable, versionable, debuggable, ownable, portable.
• We value autonomy and ownership. We want to know how the sausage is made, and we want the power to change the recipe.

dlt is for builders

As a builder in the space, the degradation of team capabilities driven by vendor marketing always rubbed me the wrong way. We built dlt because we were tired of dependency. The original "Modern Data Stack" story was a cage built for people who are afraid of learning. dlt is for those who can manage those feelings and push through (growth mindset), to build the new wave of data products. It is a foundational brick in the Builder’s Data Stack:

• It’s code-native: Because text is the only interface that scales with AI.
• It’s low-impedance: It runs where you run, from a local notebook to a massive Airflow cluster, without artificial vendor walls.
• It’s autonomous: It gives the power back to the engineer to define the recipe, not just buy the pre-packaged meal.

dltHub, the builder's stack, is coming soon.

dltHub is extending dlt to the rest of the stack, to cater to full-stack builders end to end without impedance. Sign up to our waiting list to be the first to know when dltHub for small teams launches this spring.