Skip to main content

__init__

data load tool (dlt) — the open-source Python library for data loading

How to create a data loading pipeline with dlt in 3 seconds:

  1. Write a pipeline script
import dlt
from dlt.sources.helpers import requests
dlt.run(requests.get("https://pokeapi.co/api/v2/pokemon/").json()["results"], destination="duckdb", table_name="pokemon")

  1. Run your pipeline script

    $ python pokemon.py

  2. See and query your data with autogenerated Streamlit app

    $ dlt pipeline dlt_pokemon show

Or start with our pipeline template with sample PokeAPI (pokeapi.co) data loaded to bigquery

$ dlt init pokemon bigquery

For more detailed info, see https://dlthub.com/docs/getting-started

TSecretValue​

When typing source/resource function arguments it indicates that a given argument is a secret and should be taken from dlt.secrets.

TCredentials​

When typing source/resource function arguments it indicates that a given argument represents credentials and should be taken from dlt.secrets. Credentials may be a string, dictionary or any other type.

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.