Celebrating 1,000 dlt OSS customers in production

Milestone Announcement

Earlier today, Marcin announced the release of dlt version 1.0.0, marking a significant milestone in its evolution into a stable, production-ready library for data movement.

Alongside this release, we're excited to share another major achievement: dlt has experienced 9x growth this year, recently surpassing 1,000 open-source customers in production. This milestone highlights the growing trust and adoption of dlt across the data engineering community.

Companies that use OSS dlt include Posthog, Dollar Shave Club, Flatiron Health, Harness, Taktile, brands such as Erewhon, financial institutions such as Sparebank 1 SR-Bank and multiple Fortune 1000 companies.

This achievement is a testament to the incredible support from our community and the trust you’ve placed in our product. We’re truly grateful for every one of you who has helped us reach this exciting point.

Reflection on the Journey and Growth

When we first launched the dlt Slack community in May '23, our goal was simple: to empower a new generation of Python users to create and manage data independently within their organizations. Initially, dlt was a tool designed to handle JSON documents.

Since then, we've grown significantly thanks to continuous feedback and contributions from our incredible community. We now have over 2,200 developers in our Slack, making it one of the most active spaces for Data Engineering. Our AI assistant fields, on average, more than 400 questions daily, and we’re about to hit 10,000 custom pipelines created by the community. With this ongoing input, dlt has evolved into a comprehensive library tailored for users who value simplicity, transparency, and flexibility, avoiding unnecessary complexity and external dependencies. dlt has become the go-to Python library for data movement, with over 500k monthly downloads.

When speaking with companies about why they adopt dlt, a clear pattern emerges. Over the past decade, many organizations have heavily invested in cloud infrastructure (e.g., BigQuery, Snowflake, Databricks) and the Modern Data Stack (e.g., Airflow, dbt). With dlt, they can easily extend their data infrastructure. We are closely integrated both with the Modern Data Stack and with the emerging Pythonic ecosystem of high-performance data libraries (i.e. yarrow, pola-rs, ibis, duckdb, delta-rs ). dlt efficiently processes industrial-size amounts of data in Python, including in constrained environments like AWS Lambda.

The most popular data sources for dlt users are Python objects, Arrow tables, as well as Pandas data frames, databases, and filesystems.

Adjusting our Slack support to “normal”

While we will continue to invest in dlt, we think of the 1.0.0 as a good opportunity to scale down our “great” level of free individual support in our Slack community to “normal”.

With the incredible growth we’ve seen in recent months leading up to the 1.0.0 release, our small team has recently dedicated up to 20% of our time to answering questions from individual developers in the technical help Slack channel and an increasing number of private company support channels.

So far, your support questions have enabled us to improve our products and docs. To maintain efficiency, we will change the way we answer these questions.

• About 60% or more of questions can be sufficiently answered by our AI docs assistant. OpenAI o1-preview knows about dlt, and we have built dlt for humans and LLMs. We will encourage you to use LLMs more.
• About 15% are general short questions such as about best practices. We will continue to do our best to reply to these.
• About 25% of questions are deeper coding + infra questions, which we will no longer answer in Slack unless you can reduce them to simple questions.

As our community continues to expand and we begin focusing more on our commercial offerings, we will be evolving our support approach. While we can no longer maintain the same level of direct assistance we’ve provided over the past year, this shift allows us to serve our growing community better and prioritize developing new features and offerings.

We believe that the library is now robust enough, the community large and active enough, and our documentation and tutorials extensive enough so that the community can answer many of the Slack questions. We will continue to invest in AI assistants and have also started rolling out a dlt certification program and a dlt consulting program to increase the knowledge around dlt. Our working students will continue to stay active, answering simple questions about dlt functionality, and we will continue adding the answers to frequently asked questions into our docs.

We will continue to listen as we adopt these changes. If you have any feedback let us know by contacting Akela on Slack.

If your company is interested in purchasing dedicated support, please reach out to us via our contact page.