Skip to main content
Version: devel

dlt Fundamentals Course

In this course you will learn the fundamentals of dlt alongside some of the most important topics in the world of Pythonic data engineering.

Lessons​

Lesson 1: Quick Start Open In Colab GitHub badge​

Discover what dlt is, run your first pipeline with toy data, and explore it like a pro using DuckDB, sql_client, and dlt datasets!

Lesson 2: dlt Resources and Sources Open In Colab GitHub badge​

Learn to run pipelines with diverse data sources (dataframes, databases, and REST APIs), master dlt.resource, dlt.source, and dlt.transformer, and create your first REST API pipeline!

Lesson 3: Pagination & Authentication & dlt Configuration Open In Colab GitHub badge​

Since it is never a good idea to publicly put your API keys into your code, different environments have different methods to set and access these secret keys. dlt is no different. Master pagination and authentication for REST APIs, explore dlt's RESTClient and manage secrets and configs.

Lesson 4: Using dlt's pre-built Sources and Destinations Open In Colab GitHub badge​

Now that you took a data source and loaded it into a duckdb destination, it is time to look into what other possibilities dlt offers. In this notebook we will take a look at pre-built verified sources and destinations and how to use them.

Lesson 5: Write disposition and incremental loading Open In Colab GitHub badge​

Learn to control data behavior with dlt write dispositions (Append, Replace, Merge), master incremental loading, and efficiently update and deduplicate your datasets.

Lesson 6: How dlt works Open In Colab GitHub badge​

Discover the magic behind dlt! Learn its three main steps — Extract, Normalize, Load — along with default behaviors and supported file formats.

Lesson 7: Inspecting & Adjusting Schema Open In Colab GitHub badge​

dlt creates and manages the schema automatically, but what if you want to control it yourself? Explore the schema and customize it to your needs easily with dlt!

Lesson 8: Understanding Pipeline State & Metadata Open In Colab GitHub badge​

After having learnt about pipelines and how to move data from one place to another. We now learn about information about the pipeline itself. Or, metadata of a pipeline that can be accessed and edited through dlt. This notebook explores dlt states, what it collected and where this extra information is stored. It also expands a bit more on what the load info and trace in dlt is capable of.

Homework & Certification​

As a final step, you can complete the homework quiz. Successful completion will earn you a course certification.

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.