dltHub AI Harness
40 skills across 10 toolkits. ai-harness.mddlthub.com/cheatsheet
Getting started
Prerequisite: uv (fast Python package manager).
curl -LsSf https://astral.sh/uv/install.sh | sh# Install dlt with workspace + hub support
uv pip install --upgrade "dlt[workspace]" && uv pip install "dlt[hub]"
# Set up your workspace (auto-detects your coding assistant)
uv run dlthub ai init
# Or specify your agent explicitly:
# uv run dlthub ai init --agent claude
# uv run dlthub ai init --agent cursor
# uv run dlthub ai init --agent codexThe guided entry point. Names a use case, checks the workspace, and hands off to the right toolkit in a few prompts.
The entry point for building anything with dlthub. Routes to the right toolkit and installs it on demand.
Improve existing skills based on the current session. Capture debugging patterns, doc references, workflow improvements.
Safely manage dlthub secrets in *.secrets.toml. Uses MCP tools to never expose raw values.
Find a dlt source for a given API or data provider. Use when the user asks about a source, wants to find a connector, or asks to implement a pipeline for a specific data source.
Create a dlt REST API pipeline. Use for the rest_api core source, or any generic REST or HTTP API source.
Add a new REST API endpoint/resource to an existing dlt pipeline.
Adjust a working dlt pipeline for production: remove dev limits, verify pagination, configure incremental loading, handle rate limits and 429s.
Speed up a slow REST API pipeline. Parallelize resources, switch to async, tune page size and concurrency.
Validate schema and data after a successful dlt pipeline load. Check row counts, schema, data types.
Query, explore, or view data loaded by a dlt pipeline. Covers dlt dataset API, ibis, and ReadableRelation.
Debug and inspect a dlt pipeline after running it. Inspect traces, load packages, schema, and diagnose errors.
Connect to a relational database and discover what is in it. Postgres, MySQL, MSSQL, Oracle, SQLite, or anything SQLAlchemy speaks.
Create a dlt pipeline from a SQL database source. Use for the sql_database core source, not for REST APIs or files.
Add another table or view from the same database to a pipeline that already works.
Take a table to production: remove dev limits, add incremental loading on a cursor column, configure merge keys.
Speed up a slow database extract. Pick a faster backend, tune chunk size, parallelize tables, cut reflection overhead.
Check the loaded data after a run. Inspect table schemas, fix data types, verify column mappings from the source database.
Query the loaded tables with the dlt dataset API, ibis expressions, or plain SQL.
Diagnose a failed run: connection failures, driver issues, missing credentials, failed jobs.
Create a pipeline that reads files from local disk, S3, GCS, Azure, or SFTP. Use for the filesystem core source.
Read only new or modified files on each run. Filter by modification date, then merge on a primary key to deduplicate.
Speed up file reads. Choose a faster reader, read files in parallel, narrow the glob, stream in chunks.
Start a data quality workflow on a pipeline. Discover tables, read the schema, and propose check candidates.
Turn schema hints and business rules into column level checks and load metrics.
Execute defined checks against loaded data. Reads the destination only, no re-extraction.
Read the results: what failed, how badly, and what to do about it.
Make a pipeline faster or lighter on memory. Diagnose the bottleneck stage, then apply parallelism, workers, buffers, or batching.
Annotate dlt pipeline sources for transformation. Map data sources to canonical business concepts.
Build a business entity graph from annotated sources and taxonomy.
Generate a Canonical Data Model using Kimball dimensional modeling. Star schema from your ontology.
Write dlthub transformation functions that map source tables to CDM entities using ibis.
Switch a transformation from full replace to incremental so each run only processes new or changed rows.
Debug a failing transformation. Fix SQL dialect errors after deployment, recover a failed run, find silently dropped columns.
Explore dlt pipeline data locally. Connect, profile tables, plan charts with ibis + altair, and write an analysis plan.
Assemble a marimo notebook from analysis_plan.md. Reads chart specs, generates Python file, validates, and launches.
Verify the dlthub workspace is ready for dltHub Platform. Use when deploying for the first time, or when another skill reports missing prerequisites.
Prepare production credentials and destinations. Set up profile-scoped secrets and production destinations.
Deploy dlt pipelines to dltHub Platform. Assumes workspace is verified and credentials are set.
Debug a failed or misbehaving dltHub Platform deployment. Check job status and logs.
The guided entry point. Names a use case, checks the workspace, and hands off to the right toolkit in a few prompts.
The entry point for building anything with dlthub. Routes to the right toolkit and installs it on demand.
Improve existing skills based on the current session. Capture debugging patterns, doc references, workflow improvements.
Safely manage dlthub secrets in *.secrets.toml. Uses MCP tools to never expose raw values.
Find a dlt source for a given API or data provider. Use when the user asks about a source, wants to find a connector, or asks to implement a pipeline for a s...
Create a dlt REST API pipeline. Use for the rest_api core source, or any generic REST or HTTP API source.
Add a new REST API endpoint/resource to an existing dlt pipeline.
Adjust a working dlt pipeline for production: remove dev limits, verify pagination, configure incremental loading, handle rate limits and 429s.
Speed up a slow REST API pipeline. Parallelize resources, switch to async, tune page size and concurrency.
Validate schema and data after a successful dlt pipeline load. Check row counts, schema, data types.
Query, explore, or view data loaded by a dlt pipeline. Covers dlt dataset API, ibis, and ReadableRelation.
Debug and inspect a dlt pipeline after running it. Inspect traces, load packages, schema, and diagnose errors.
Connect to a relational database and discover what is in it. Postgres, MySQL, MSSQL, Oracle, SQLite, or anything SQLAlchemy speaks.
Create a dlt pipeline from a SQL database source. Use for the sql_database core source, not for REST APIs or files.
Add another table or view from the same database to a pipeline that already works.
Take a table to production: remove dev limits, add incremental loading on a cursor column, configure merge keys.
Speed up a slow database extract. Pick a faster backend, tune chunk size, parallelize tables, cut reflection overhead.
Check the loaded data after a run. Inspect table schemas, fix data types, verify column mappings from the source database.
Query the loaded tables with the dlt dataset API, ibis expressions, or plain SQL.
Diagnose a failed run: connection failures, driver issues, missing credentials, failed jobs.
Create a pipeline that reads files from local disk, S3, GCS, Azure, or SFTP. Use for the filesystem core source.
Read only new or modified files on each run. Filter by modification date, then merge on a primary key to deduplicate.
Speed up file reads. Choose a faster reader, read files in parallel, narrow the glob, stream in chunks.
Start a data quality workflow on a pipeline. Discover tables, read the schema, and propose check candidates.
Turn schema hints and business rules into column level checks and load metrics.
Execute defined checks against loaded data. Reads the destination only, no re-extraction.
Read the results: what failed, how badly, and what to do about it.
Make a pipeline faster or lighter on memory. Diagnose the bottleneck stage, then apply parallelism, workers, buffers, or batching.
Annotate dlt pipeline sources for transformation. Map data sources to canonical business concepts.
Build a business entity graph from annotated sources and taxonomy.
Generate a Canonical Data Model using Kimball dimensional modeling. Star schema from your ontology.
Write dlthub transformation functions that map source tables to CDM entities using ibis.
Switch a transformation from full replace to incremental so each run only processes new or changed rows.
Debug a failing transformation. Fix SQL dialect errors after deployment, recover a failed run, find silently dropped columns.
Explore dlt pipeline data locally. Connect, profile tables, plan charts with ibis + altair, and write an analysis plan.
Assemble a marimo notebook from analysis_plan.md. Reads chart specs, generates Python file, validates, and launches.
Verify the dlthub workspace is ready for dltHub Platform. Use when deploying for the first time, or when another skill reports missing prerequisites.
Prepare production credentials and destinations. Set up profile-scoped secrets and production destinations.
Deploy dlt pipelines to dltHub Platform. Assumes workspace is verified and credentials are set.
Debug a failed or misbehaving dltHub Platform deployment. Check job status and logs.