Airbyte to dlt in five days: how Yespark's CTO migrated ingestion without writing Python

Presented by Guillaume Wrobel at the dlthub community meetup Paris 2026-02-05

Yespark's data engineer left in November 2025. The co-founder and CTO, inherited the stack: Airbyte for ingestion, dbt for transforms, Snowflake for storage, Metabase for BI. He describes himself as "an engineer, but not a real developer." By his own account he did not write a single line of Python during the migration that followed.

Five working days later, Airbyte was gone. Every source — Postgres, Zendesk, HubSpot, Google Ads, Google Sheets, GA4, Google Search Console, App Store and Play reviews — was running on dlt, orchestrated by Prefect on the same Scaleway worker they already owned. He presented the whole thing at a dltHub community meetup.

Four phases of paying for ingestion

Yespark is a European parking platform: 110k+ spaces, 100+ employees, roughly €50M revenue, which means real ingestion volume across a dozen sources — Postgres, Zendesk, HubSpot, Google Ads, Google Sheets — feeding a Snowflake warehouse.

The interesting part of Yespark's story isn't the last move. It's the three moves before it, because each one was a rational decision that solved the previous tool's problem and introduced a new one. Read together they're a tour of how ingestion actually gets paid for.

Phase 1 — Fivetran (2021–2022). They started where a lot of teams start: a fully managed connector service that just works. And it did work. The problem was the meter. Fivetran priced on Monthly Active Rows, and as Yespark's volume grew the bill grew with it — into the tens of thousands per year for what is, mechanically, moving rows from A to B. Fivetran's pitch is that you pay so you never think about ingestion. That holds right up until the amount you're paying is itself the thing you think about.

Phase 2 — Airbyte Standard (2023). The obvious response to an expensive managed tool is a cheaper managed tool. Airbyte's hosted plan was materially cheaper than Fivetran, so they switched. But the pricing was still volume-based, which means the same structural problem in a smaller font: the cost scales with the data, not with the value, and Yespark's data was only going up. Cheaper is not the same as solved. It just moves the day you have the conversation again.

Phase 3 — Airbyte OSS (2024–2026). So they did what the volume math pushes every growing team toward: self-host the open-source version and stop paying per row. On paper this is the win — the connectors are free, you run them on your own Scaleway instance, the marginal cost of another million rows is zero. In practice the bill didn't disappear. It changed currency. You stop paying Airbyte in dollars and start paying it in operations: a dedicated instance to keep alive, Docker to manage, memory to budget, disks that fill with oversized logs, and connector updates that land on the maintainer's schedule and break on yours.

Yespark's own job history is the receipt. Syncs partially succeeding after ten attempts. "Something went wrong in the connector," with the actual cause buried in logs. Wrobel's one-line summary of the whole ingestion layer: it's usually where the trouble happens.

None of this is an Airbyte indictment — the OSS project is doing exactly what self-hosting means. It's the standard cost of owning a separate ingestion platform: you don't just run your pipelines, you operate the thing that runs them. That's the bill that doesn't show up on any invoice, and it's the one that gets more expensive as the team gets smaller.

Which is the state Yespark was in when the data engineer left. One CTO, four tools he hadn't set up, and an ingestion platform that was the most operationally demanding piece of all of them.

Here's the pattern across the three phases, because it's the thing that predicts the fourth:

Every move traded one kind of cost for another. Managed tools charge money to remove operational load; self-hosting removes the money and hands the operational load back. The question was never "managed or self-hosted." It was "which cost do you want to carry, and does the tool let you carry as little of it as possible."

Phase 4 — dlt: stop buying ingestion, absorb it

The three phases all treated ingestion as a thing you acquire — a product you pay for in money or in ops. dlt is a different move: it isn't a platform you run alongside your stack, it's a Python library your existing runtime imports. There's nothing extra to operate because there's nothing extra. The pipelines run on the Prefect worker Yespark already had.

Wrobel had first heard about dlt on the Modern Data Network in May 2025, off the back of an Adrian post on turning Airbyte YAML into Python pipelines with an LLM. The reasons he moved were operational, not ideological:

• Regain control over connections. No mysterious connector between the source and the warehouse. The code is the connector.
• One less thing to operate. dlt runs on the Prefect worker that already existed. No dedicated Airbyte instance, no Docker layer to babysit, no separate memory budget.
• Python-native. Which matters more than it used to, because Python is the language an LLM writes fluently.
• Customization without waiting. No waiting on an official connector to land or a broken one to get fixed upstream.
• Same tool for ingestion and orchestration. dlt inside Prefect, so notifications, retries, and scheduling all live in one place.

Read those together and it's one point: dlt let Yespark delete a layer instead of adding one. Claude Code put it well in a slide Wrobel showed — the main gain is eliminating a separate piece of infrastructure to maintain, not leaving Airbyte itself.

The row that earns the whole migration is "Orchestration": dual becomes single. Two systems to reason about becomes one.

The part that makes this a dlt story and not a Claude story

Here is the honest center of it. A CTO who doesn't write Python migrated a production ingestion stack in five days by driving Claude Code with --dangerously-skip-permissions and copy-pasting his way through the console. He called himself a copy-paste monkey, and meant it as a compliment to the tooling.

That only holds up because of what sat underneath the agent.

An LLM will happily generate an ingestion script that runs. "It runs" was always the junior bar. The senior bar is the stuff that doesn't show up in a first successful run: are credentials read through a config system or off disk? Does incremental loading actually track state, or does it silently full-refresh a large table every night? Does the sync sample before it loads, or discover the schema is wrong at row five million? Wrobel's Tuesday was "incremental mode for large tables" — that's not a detail the model volunteers. It's a detail the pipeline library has to make the default path.

dlt is what turns generated code into an engineered pipeline. Schema inference, incremental state, load IDs, the secrets.toml boundary for credentials, native REST support for the sources Airbyte never had a connector for — those are the guardrails that let a non-Python CTO ship something that survives contact with production. The agent wrote the code. The library made the code correct by default.

Both a bare LLM and dlt-plus-LLM produce running code. Only one produces a pipeline you'd leave unattended at 4am.

Five days, in order

The timeline is worth reading as-is, because it shows the migration was incremental — never a big-bang cutover.

Postgres first because it's the highest-volume, highest-risk source — validate the hard case before touching anything else. New sources next, including ones Airbyte handled awkwardly. Only on Thursday, once the replacement was proven source-by-source, did Airbyte get deleted. Friday was spent on the operational layer that OSS Airbyte kept costing them: automatic recovery and alerting into Slack.

The result Wrobel showed was a Prefect dashboard: eleven runs, 0% failure rate, sub-two-minute durations, every source landing in Snowflake on schedule. A Postgres sync of five million rows completing in about an hour, with a tailored Slack notification instead of a log dive. His caption, in French, translates to "without having written a single line of Python in my life."

After migration: 11 runs, 0% failure rate

What this costs, honestly

Moving to dlt does not make ingestion free of engineering. It relocates it. You no longer operate an Airbyte instance, but you now own Python pipeline code — which means someone has to read it, review it when the agent edits it, and catch it when a source's API changes under you. The failure mode moves from "the connector broke and I'm reading Docker logs" to "the pipeline code needs a change and I need to understand what it does."

For Yespark that's a good trade, because the code is inspectable and lives next to everything else in one repo, and because the agent handles most of the editing. But it is a trade, not a free lunch. The open items on Wrobel's own TODO make the point: a Google Ads developer token still to claim, a Zendesk ticket-comments source copied from GitHub that still throws errors, efficiency-price elasticity of the Scaleway and Snowflake compute still to measure. Real migrations have a TODO list. This one does too.

The takeaway for anyone still self-hosting Airbyte OSS

If you're running Airbyte OSS to save money, price the operational cost, not just the license. The dedicated instance, the Docker maintenance, the disk that fills with logs, the connector updates that break on someone else's schedule, the second orchestration surface layered on top of your real one — that's the bill. Yespark deleted all of it in a week by collapsing ingestion into the orchestration layer they already ran.

The agent made the migration fast. dlt made it survive Friday.

Want to see what makes agent-generated pipelines hold up in production? The dlt AI Workbench packages the skills and rules that teach coding agents to use dlt correctly: github.com/dlt-hub/dlthub-ai-workbench. Start with the docs at dlthub.com/docs.