Skip to main content
Version: devel View Markdown

Deploy a pipeline with dltHub

dltHub ships a managed orchestrator built around the @dlt.hub.run decorators. The schedule and the data dependencies live in your Python code, not in a separate DAG (directed acyclic graph) file or YAML.

This page walks through four stages: scaffolding a workspace, running a pipeline ad-hoc, deploying it via __deployment__.py, and scheduling it with a trigger. For the full feature surface (follow-up chains, freshness gates, refresh cascade, tags, timezones) see Triggers and scheduling.

1. Install and scaffold a workspace

If you don't have uv yet, follow the uv installation guide. Then scaffold a workspace:

uvx dlthub-init@latest

This creates the .dlt/.workspace marker that activates workspace mode, along with pyproject.toml and .dlt/ config and secrets files. For other install options and setup paths, see Installation.

2. Run a pipeline ad-hoc

Any regular dlt pipeline can be run on the platform without a deployment file. Given a pipeline.py with a top-level pipeline.run(...) call:

uv run dlthub run pipeline.py

This deploys the file ad-hoc, executes it on the platform, and streams logs back to your terminal. Use this for one-off runs and smoke tests. See Quick deploy.

3. Add __deployment__.py

To run the same pipeline as a managed job, decorate the entrypoint and declare it in a workspace __deployment__.py:

import dlt
from dlt.hub import run


@run.pipeline("ingest_breweries")
def ingest_breweries():
pipeline = dlt.pipeline(
pipeline_name="ingest_breweries",
destination="warehouse",
dataset_name="brewery_data",
)
pipeline.run(brewery_source())
# __deployment__.py
from ingest_breweries import ingest_breweries

__all__ = ["ingest_breweries"]

Deploy with:

uv run dlthub deploy

To trigger a run, invoke the job by name:

uv run dlthub run ingest_breweries

See Deployments for the manifest layout, reconciliation rules, and the full lifecycle.

4. Schedule it

Pass a trigger= to the decorator to make the job cron-driven:

from dlt.hub.run import trigger


@run.pipeline(
"ingest_breweries",
trigger=trigger.schedule("0 * * * *"), # every hour, on the hour
)
def ingest_breweries():
...

Run dlthub deploy again to push the trigger to the platform. The scheduler then runs the job on its cron. Trigger factories also include trigger.every("5m") (fixed interval) and trigger.once(...) (single run). See Basic triggers.

Adjust the schedule from the platform

You can also change the cron from the dltHub platform's Manage Schedule dialog, useful for ad-hoc pauses or tweaks without redeploying. To make a change permanent, update the decorator and run dlthub deploy.

Advanced features

See also

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.