# dltHub AI Harness Cheatsheet

Synced from [dlt-hub/dlthub-ai-harness](https://github.com/dlt-hub/dlthub-ai-harness) on branch `master`.

Source tree: [workbench/](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench)

## rest-api-pipeline

Build dlt pipelines for REST APIs: find a source, scaffold the pipeline, add endpoints and incremental loading, debug it, validate the data and view it in a report. Requires dlt[hub].

Source: [`rest-api-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/rest-api-pipeline)
Version: `0.1.2`
Category: `data`
Listed: yes
Dependencies: `init`
Workflow entry skill: `/find-source`
Tags: `dlthub`, `etl`, `data-pipeline`, `python`

### Skills

- [`/adjust-endpoint`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/adjust-endpoint/SKILL.md) — Adjust a working dlt pipeline for production — remove dev limits, verify pagination (including stuck or looping paginators), configure incremental loading, expand date ranges, and handle rate-limit/429 errors with retries, backoff, and request timeouts. Use when the user wants to remove .add_limit(), load more data, fix stuck or looping pagination, set up incremental loading, or make the pipeline retry/back off on 429s. For throughput/concurrency tuning (parallel resources, page size) when a working pipeline is slow, use optimize-rest-api-performance instead. For inspecting loaded data, fixing column types, or flattening nested structures after a load, use validate-data instead.
- [`/create-rest-api-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/create-rest-api-pipeline/SKILL.md) — Create a dlt REST API pipeline. Use for the rest_api core source, or any generic REST/HTTP API source. Not for sql_database or filesystem sources.
- [`/debug-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/debug-pipeline/SKILL.md) — Debug and inspect a dlt pipeline after running it. Use after a pipeline run (success or failure) to inspect traces, load packages, schema, data, and diagnose errors like missing credentials or failed load jobs. Also covers pipelines built on a verified source.
- [`/find-source`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/find-source/SKILL.md) — Find a dlt source for a given API or data provider. Use when the user asks about a source, wants to find a connector, or asks to implement a pipeline for a specific data source.
- [`/new-endpoint`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/new-endpoint/SKILL.md) — Add a new REST API endpoint/resource to an existing dlt pipeline. Use when the user wants to pull additional data from an API that already has a working pipeline.
- [`/optimize-rest-api-performance`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/optimize-rest-api-performance/SKILL.md) — Speed up a dlt REST API pipeline. Use when a REST/HTTP API pipeline is slow because of many sequential requests, nested child resources, or large responses, and the user wants higher throughput — parallelize resources, use async, tune page size and concurrency. For removing .add_limit(), fixing pagination, first-time incremental/merge setup, or retry/backoff on 429 / rate-limit errors use adjust-endpoint instead.
- [`/validate-data`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/validate-data/SKILL.md) — Validate schema and data after a successful dlt pipeline load. Use when the user wants to check if loaded data looks correct, inspect table schemas, fix data types, flatten nested structures, or refine the data shape.
- [`/view-data`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/skills/view-data/SKILL.md) — Query, explore, or view data loaded by a dlt pipeline. Use when the user asks to query data, explore loaded tables, check row counts, write Python that reads pipeline data, or asks questions like "show me the data", "what users are there", "how much did we spend". Covers dlt dataset API, ibis expressions, and ReadableRelation.

### Rules

- [`New ingestion pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/rest-api-pipeline/rules/workflow.md)

### MCP Servers

- dlt-workspace-mcp — `uv run dlthub ai mcp --stdio`

## bootstrap

Set up the Python environment for a dlthub workspace. Requires dlt[hub].

Source: [`bootstrap`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/bootstrap)
Version: `0.1.1`
Category: `data`
Listed: no
Dependencies: `init`
Tags: `dlthub`, `workspace`, `setup`

### Commands

- [`init-workspace`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/bootstrap/commands/init-workspace.md) — Sets up dlthub workspace. Ensures `uv`, Python env and dlthub are present. Installs LLM toolkit to kickstart future work.

## data-exploration

Get quick insights from a dlt pipeline: profile its tables, plan charts and assemble a marimo dashboard. Requires dlt[hub].

Source: [`data-exploration`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/data-exploration)
Version: `0.1.2`
Category: `data`
Listed: yes
Dependencies: `dlthub-platform`
Workflow entry skill: `/explore-data`
Tags: `dlthub`, `marimo`, `ibis`, `data-exploration`, `dashboards`, `altair`

### Skills

- [`/build-notebook`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-exploration/skills/build-notebook/SKILL.md) — This skill should be used when the user asks to "build the notebook", "launch the dashboard", "generate the marimo notebook", or when an analysis_plan.md artifact exists and the user wants to assemble or regenerate the dashboard. Reads chart specs with ibis queries and altair code from analysis_plan.md, assembles a marimo Python file, validates, and launches. Do NOT use for exploring data or planning charts (use explore-data), building pipelines (use rest-api-pipeline toolkit), or deploying (use dlthub-platform toolkit).
- [`/explore-data`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-exploration/skills/explore-data/SKILL.md) — This skill should be used when the user asks to "explore my data", "what can I learn from this pipeline", "what's the revenue trend", "show me charts", "visualize my pipeline", "analyze my data", "profile data quality", "what questions can I ask about my data", "map my data to business concepts", or wants to explore, profile, analyze, or chart data from a dlt pipeline. Connects to a pipeline, profiles tables or scans schema, plans charts with ibis + altair code, and writes an analysis_plan.md artifact. Do NOT use for building or fixing pipelines (use rest-api-pipeline toolkit), deploying pipelines (use dlthub-platform toolkit), or assembling the marimo notebook from an analysis plan (use build-notebook).

### Rules

- [`Data exploration workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-exploration/rules/workflow.md)

## dlthub-platform

Prepare and deploy your dlthub workspace and pipelines to the dltHub platform, and debug failed deployments. Requires dlt[hub].

Source: [`dlthub-platform`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/dlthub-platform)
Version: `0.3.3`
Category: `data`
Listed: yes
Dependencies: `init`
Workflow entry skill: `/setup-runtime`
Tags: `dlthub`, `deploy`, `platform`

### Skills

- [`/debug-deployment`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-platform/skills/debug-deployment/SKILL.md) — Debug a failed or misbehaving dltHub Platform deployment. Use when a runtime job fails, produces unexpected results, or the user wants to check job status and logs.
- [`/deploy-workspace`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-platform/skills/deploy-workspace/SKILL.md) — Deploy dlt pipelines to dltHub Platform. Use when the user says "deploy to dltHub", "launch on dltHub", "run on dltHub", "schedule pipeline", or wants to deploy a pipeline or notebook to dltHub.
- [`/prepare-deployment`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-platform/skills/prepare-deployment/SKILL.md) — Prepare production credentials and destinations for dltHub Platform. Use when setting up prod profile secrets, splitting dev/prod credentials, or configuring a production destination like Motherduck.
- [`/setup-runtime`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-platform/skills/setup-runtime/SKILL.md) — Verify dlthub workspace is ready for dltHub Platform. Use when user wants to deploy for the first time, or when another skill reports missing prerequisites like .workspace file or dlt[hub] dependency.

### Rules

- [`NEVER change job resources without explicit human permission`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-platform/rules/job-resources.md)
- [`Profiles`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-platform/rules/profiles.md)
- [`Deploy to dltHub Platform`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-platform/rules/workflow.md)

## init

Shared rules, secrets handling, background agent authoring and the workspace MCP server used by the other dlthub toolkits. Requires dlt[hub].

Source: [`init`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/init)
Version: `0.1.4`
Category: `data`
Listed: yes
Tags: `dlthub`, `init`, `rules`, `secrets`, `mcp`

### Skills

- [`/create-background-agent`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/init/skills/create-background-agent/SKILL.md) — Write a dltHub background agent: an AGENT.md that runs unattended on the platform, on a schedule, after a job fails, or when someone starts it from the web UI. Use when the user wants work to happen on its own after an event ('whenever a job fails, diagnose it for me', 'every Monday summarise what broke'), when they ask how to write, declare or deploy an AGENT.md, how to pick its access, tools, inputs or output, how to set run.agent, or how to ship an agent in a toolkit. Do NOT use for a host subagent in .claude/agents/, for scheduling a pipeline or a plain job (deploy-workspace), for diagnosing one failure right now (debug-deployment), or for setting AGENT__MODEL and the other AGENT__ variables, which belong to any agent job including the shipped job-inspector.
- [`/dlthub-router`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/init/skills/dlthub-router/SKILL.md) — The entry point for building anything with dlthub: route the user to the right workflow toolkit and install it on demand. MUST use when the user asks 'what can you do', 'what can I build', 'what are toolkits', 'how do I build a pipeline', 'I want to pull data from a REST API', 'ingest from a SQL database', 'ingest CSVs from S3 into a warehouse', 'make reports / dashboards', 'transform / model my data', 'add data quality checks', 'how do I deploy / schedule a pipeline', 'I'm new to dlthub', 'where do I start', 'my deployed job failed', or seems unsure what to do next after setup. Also use when the user names a data-engineering goal and no matching toolkit is installed yet. Do NOT use when the matching toolkit is already installed: go straight to its entry skill. Do NOT use when a specific task is in progress (debugging a pipeline, validating data, adding endpoints) and its toolkit is installed. Do NOT use when the user wants a guided end-to-end demo: use **quick-start**.
- [`/improve-skills`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/init/skills/improve-skills/SKILL.md) — Improve existing skills based on the current session. Use at the end of a session (or when the user asks) to capture new debugging patterns, data issues, data validation tracks, querying techniques, doc references, or workflow improvements learned during the session. Keeps skills lean and personalized.
- [`/setup-secrets`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/init/skills/setup-secrets/SKILL.md) — Safely manage dlthub secrets in *.secrets.toml. Use when the user directly asks to set up, configure, or inspect credentials (API keys, database passwords, tokens). Also use when writing Python code that needs to read secrets via dlt.secrets without exposing values. Do NOT use for pipeline creation, source discovery, or debugging pipeline execution — those skills call setup-secrets when they need credentials configured.
- [`/troubleshoot-pipeline-run`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/init/skills/troubleshoot-pipeline-run/SKILL.md) — General pipeline troubleshooting, when no troubleshooting guide specific to the source or runtime context is present.

### Rules

- [`setup`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/init/rules/dlthub-workspace.md)

### MCP Servers

- dlt-workspace-mcp — `uv run dlthub ai mcp --stdio`

## transformations

Turn raw dlt pipeline data into a Canonical Data Model: annotate sources, build an ontology, design a Kimball-style model and write @dlt.hub.transformation functions. Requires dlt[hub].

Source: [`transformations`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/transformations)
Version: `0.1.3`
Category: `data`
Listed: yes
Dependencies: `dlthub-platform`
Workflow entry skill: `/annotate-sources`
Tags: `dlthub`, `transformation`, `cdm`, `kimball`, `ibis`, `dimensional-modeling`

### Skills

- [`/annotate-sources`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/transformations/skills/annotate-sources/SKILL.md) — Annotate dlt pipeline sources for transformation. Use when the user wants to transform data, do data modelling, design a data model, describes their data sources and use cases, or wants to build a CDM from existing pipelines.
- [`/create-ontology`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/transformations/skills/create-ontology/SKILL.md) — Build a business entity graph (ontology) from annotated sources and taxonomy. Use after annotate-sources to design the entity model before CDM generation.
- [`/create-transformation`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/transformations/skills/create-transformation/SKILL.md) — Write dlthub transformation functions that map source tables to CDM entities. Use after generate-cdm to produce the transformation Python script.
- [`/debug-transformation`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/transformations/skills/debug-transformation/SKILL.md) — Debug dlthub transformation failures. Use when a transformation fails on a different destination than it was developed on, SQL dialect errors occur after deployment, pipeline recovery is needed after a failed run, or columns are silently dropped from output.
- [`/generate-cdm`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/transformations/skills/generate-cdm/SKILL.md) — Generate a Canonical Data Model (CDM) in DBML using Kimball dimensional modeling. Use after create-ontology to produce the implementation-ready CDM schema.
- [`/incremental-transformation`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/transformations/skills/incremental-transformation/SKILL.md) — Switch a dlthub transformation from full-replace to incremental loading. Use when the user wants to process only new or changed rows, reduce transformation run time, or schedule frequent transformation runs without reprocessing all data.

### Rules

- [`Transformations workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/transformations/rules/workflow.md)

### MCP Servers

- dlt-workspace-mcp — `uv run dlthub ai mcp --stdio`

## sql-database-pipeline

Load tables from any SQLAlchemy-supported database into a destination with dlt, and tune performance with the available backends. Requires dlt[hub].

Source: [`sql-database-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/sql-database-pipeline)
Version: `0.2.1`
Category: `data`
Listed: yes
Dependencies: `init`
Workflow entry skill: `/find-sql-database-source`
Tags: `dlthub`, `etl`, `data-pipeline`, `python`, `sql`, `database`

### Skills

- [`/add-table`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/add-table/SKILL.md) — Add a new table or view to an existing dlt SQL database pipeline. Use when the user wants to load additional tables from the same database that already has a working pipeline.
- [`/adjust-table`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/adjust-table/SKILL.md) — Adjust a working dlt SQL database pipeline for production — remove dev limits, add incremental loading, configure merge keys. Use when the user wants to remove .add_limit(), load the full table, or set up incremental loading on a cursor column. For speed/memory tuning (backend, chunk size, parallelism) use optimize-sql-performance instead. For inspecting loaded data or fixing column types/schema after a load, use validate-sql-database-data instead.
- [`/create-sql-database-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/create-sql-database-pipeline/SKILL.md) — Create a dlt pipeline from a SQL database source (postgres, mysql, mssql, oracle, sqlite, or any SQLAlchemy-supported database). Use when the user wants to load tables from a relational database to a destination like DuckDB, BigQuery, or Snowflake. Not for REST APIs or file sources.
- [`/debug-sql-database-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/debug-sql-database-pipeline/SKILL.md) — Debug and inspect a dlt SQL database pipeline after running it. Use after a pipeline run (success or failure) to inspect traces, load packages, schema, and diagnose errors like connection failures, missing credentials, driver issues, or failed load jobs.
- [`/find-sql-database-source`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/find-sql-database-source/SKILL.md) — Find and explore a SQL database source for a dlt pipeline. Use when the user wants to load data from a relational database (postgres, mysql, mssql, oracle, sqlite, or any SQLAlchemy-supported database), mentions a database connection, wants to discover available tables, or asks to build a pipeline from a SQL source.
- [`/optimize-sql-performance`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/optimize-sql-performance/SKILL.md) — Speed up a dlt SQL database pipeline. Use when extraction from a relational database (postgres, mysql, mssql, oracle, snowflake, etc.) is slow or memory-heavy and the user wants to optimize it — pick a faster backend, tune chunk size, parallelize tables, or reduce reflection overhead. For first-time incremental/merge setup or removing .add_limit() use adjust-table instead.
- [`/validate-sql-database-data`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/validate-sql-database-data/SKILL.md) — Validate schema and data after a successful dlt SQL database pipeline load. Use when the user wants to check if loaded data looks correct, inspect table schemas, fix data types, or verify column mappings from the source database.
- [`/view-sql-database-data`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/skills/view-sql-database-data/SKILL.md) — Query, explore, or view data loaded by a dlt SQL database pipeline. Use when the user asks to query data, explore loaded tables, check row counts, write Python that reads pipeline data, or asks questions like "show me the data", "what users are there", "how much did we spend". Covers dlt dataset API, ibis expressions, and ReadableRelation. NOT for querying the source database — use the pipeline's destination.

### Rules

- [`SQL database pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/sql-database-pipeline/rules/workflow.md)

### MCP Servers

- dlt-workspace-mcp — `uv run dlthub ai mcp --stdio`

## data-quality

Add column-level checks and load metrics to dlt pipelines, run them on every load and review failures with diagnosis. Requires dlt[hub].

Source: [`data-quality`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/data-quality)
Version: `0.1.3`
Category: `data`
Listed: yes
Dependencies: `dlthub-platform`
Workflow entry skill: `/setup-data-quality`
Tags: `dlthub`, `data-quality`, `validation`, `checks`, `metrics`

### Skills

- [`/define-data-quality-checks`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-quality/skills/define-data-quality-checks/SKILL.md) — Use when the user asks to "define checks", "add validation rules", "what checks should I add", "translate requirements into checks", or wants to map schema hints or business rules to dlthub data quality check and metric calls for a specific pipeline or table. Do NOT use to run checks (use run-data-quality) or to set up the pipeline environment (use setup-data-quality).
- [`/review-data-quality`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-quality/skills/review-data-quality/SKILL.md) — Use when the user asks to "review data quality results", "what failed", "show me data quality results", "analyze check results", "investigate data quality failures", or wants to understand check and metric outcomes from a pipeline run. Do NOT use to run new checks (use run-data-quality).
- [`/run-data-quality`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-quality/skills/run-data-quality/SKILL.md) — Use when the user asks to "run data quality checks", "execute checks", "run my data quality checks", "check the data now", "run validations", or wants to execute already-defined checks against a loaded pipeline. Do NOT use to define new checks (use define-data-quality-checks) or to review existing results (use review-data-quality).
- [`/setup-data-quality`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-quality/skills/setup-data-quality/SKILL.md) — Use when the user asks to "set up data quality", "enable data quality checks", "add data quality to my pipeline", "validate my pipeline data", "I want to check data quality", "check my tables for issues", or wants to start any data quality workflow on a dlt pipeline. Do NOT use for exploring or charting data (use data-exploration toolkit), running existing checks (use run-data-quality), or reviewing results (use review-data-quality).

### Rules

- [`Data quality conventions`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-quality/rules/dq-rules.md)
- [`Data quality workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/data-quality/rules/workflow.md)

## filesystem-pipeline

Load CSV, Parquet, JSONL or custom files from local disk, S3, GCS, Azure or SFTP into a destination with dlt. Requires dlt[hub].

Source: [`filesystem-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/filesystem-pipeline)
Version: `0.1.2`
Category: `data`
Listed: yes
Dependencies: `init`
Workflow entry skill: `/create-filesystem-pipeline`
Tags: `dlthub`, `filesystem`, `s3`, `gcs`, `azure`, `sftp`, `csv`, `parquet`, `jsonl`

### Skills

- [`/add-incremental-loading`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/filesystem-pipeline/skills/add-incremental-loading/SKILL.md) — Add incremental loading to a dlt filesystem pipeline — filter files by modification date, optionally filter records by a timestamp column, and switch to merge with a primary key to deduplicate updated records, so each run only reads new or modified files. Use after create-filesystem-pipeline produces a working replace-mode pipeline, or when the user wants to set up merge/dedup or move a pipeline from replace to incremental. For speed/memory tuning (faster reader, chunked streaming, parallel reads, glob narrowing) use optimize-filesystem-performance instead.
- [`/create-filesystem-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/filesystem-pipeline/skills/create-filesystem-pipeline/SKILL.md) — Create a dlt filesystem pipeline that reads files (CSV, Parquet, JSONL, or custom) from local disk, S3, GCS, Azure, or SFTP into a destination. Use for the filesystem core source. Not for REST APIs (rest_api) or databases (sql_database).
- [`/optimize-filesystem-performance`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/filesystem-pipeline/skills/optimize-filesystem-performance/SKILL.md) — Speed up a dlt filesystem pipeline. Use when reading files (CSV, Parquet, JSONL) from local disk, S3, GCS, Azure, or SFTP is slow or memory-heavy and the user wants to optimize it — choose a faster reader, read files in parallel, narrow the glob, or stream in chunks. For first-time incremental/merge setup, or to skip files already loaded in previous runs, use add-incremental-loading instead.

### Rules

- [`Filesystem pipeline workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/filesystem-pipeline/rules/workflow.md)

## quick-start

A guided path from ingestion to visualization, transformation and deployment in 3-5 prompts. Requires dlt[hub].

Source: [`quick-start`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/quick-start)
Version: `0.1.2`
Category: `data`
Listed: yes
Dependencies: `dlthub-platform`, `rest-api-pipeline`, `data-exploration`, `transformations`
Workflow entry skill: `/quick-start`
Tags: `dlthub`, `quick-start`, `onboarding`, `demo`, `router`

### Skills

- [`/quick-start`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/quick-start/skills/quick-start/SKILL.md) — The guided entry point for dltHub AI Harness use cases — ingestion from APIs, data exploration, transformations, deployment, data quality. Use when the user names a use case or wants to be oriented before starting: 'I want to ingest from Stripe', 'show me how to go from data to dashboard', 'take me through the full workflow', 'explore the harness', 'what can I do with dlthub', 'give me a quick start', 'show me a demo', 'walk me through ingestion to visualization', 'I want to try everything end-to-end', 'teach me dltHub'.

### Commands

- [`start`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/quick-start/commands/start.md) — Run the quick-start guided workflow

### Rules

- [`Quick Start workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/quick-start/rules/workflow.md)

## one-shot

Build a minimal GitHub issues pipeline and deploy it to the dltHub platform in one go. Requires dlt[hub].

Source: [`one-shot`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/one-shot)
Version: `0.1.0`
Category: `data`
Listed: yes
Dependencies: `dlthub-platform`
Workflow entry skill: `/deploy-run-sample-pipeline`
Tags: `dlthub`, `pipeline`, `one-shot`, `data-pipeline`, `python`

### Skills

- [`/deploy-run-sample-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/one-shot/skills/deploy-run-sample-pipeline/SKILL.md) — Deploy and run the pre-shipped Jaffle Shop sample pipeline on dltHub Platform — the final onboarding step after uvx dlthub-start. Use when the user says 'deploy the pipeline', 'deploy the pre-built pipeline', 'deploy the onboarding pipeline', 'do the onboarding task', 'finish the onboarding task', or is ready to complete onboarding. Assumes scaffolding, login, and playground workspace connection are already done.

### Rules

- [`One-shot workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/one-shot/rules/workflow.md)

## dlthub-init-skills

Build and deploy a minimal custom REST API pipeline to the dltHub platform after running uvx dlthub-init. Requires dlt[hub].

Source: [`dlthub-init-skills`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/dlthub-init-skills)
Version: `0.1.0`
Category: `data`
Listed: yes
Dependencies: `dlthub-platform`
Workflow entry skill: `/deploy-minimal-ingestion-pipeline`
Tags: `dlthub`, `init`, `pipeline`, `deploy`, `rest-api`

### Skills

- [`/deploy-minimal-ingestion-pipeline`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-init-skills/skills/deploy-minimal-ingestion-pipeline/SKILL.md) — Build and deploy a minimal custom REST API pipeline to dltHub Platform. Use when the user says "Help me build and deploy a minimal pipeline", "I've finished onboarding, now what?", or "I just ran dlthub-start and want to build my own pipeline".

### Rules

- [`dlthub-init-skills — workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/dlthub-init-skills/rules/workflow.md)

### MCP Servers

- dlt-workspace-mcp — `uv run dlthub ai mcp --stdio`

## performance

Find the slow stage of a dlt pipeline (extract, normalize or load) and tune parallelism, workers, buffers, file rotation and batching. Requires dlt[hub].

Source: [`performance`](https://github.com/dlt-hub/dlthub-ai-harness/tree/master/workbench/performance)
Version: `0.2.3`
Category: `data`
Listed: yes
Dependencies: `dlthub-platform`
Workflow entry skill: `/optimize-performance`
Tags: `dlthub`, `performance`, `optimization`, `parallelism`, `throughput`, `memory`

### Skills

- [`/optimize-performance`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/performance/skills/optimize-performance/SKILL.md) — Make a dlt pipeline faster or lighter on memory. Use when the user says a pipeline is slow, takes too long, runs out of memory, uses too much RAM, or wants to optimize, speed up, parallelize, or increase throughput. Covers source-agnostic levers (parallelism, workers, buffers, file rotation); for source-specific tuning use the pipeline toolkit's own optimize skill. Also decides whether a dltHub platform job genuinely needs a larger instance (more memory/CPU) — the last resort after tuning.

### Rules

- [`Performance tuning workflow`](https://github.com/dlt-hub/dlthub-ai-harness/blob/master/workbench/performance/rules/workflow.md)
