Find your data, without asking who, where, why

dltHub’s new datasets UI and lineage graphs are now available in public preview. We are launching Workspace Explorer today which brings two new views - Data Lineage and Job Graph tabs - into the dltHub UI.

  • Matthaus Krzykowski

    Co-Founder & CEO

3 min read

Summarize withClaudeChatGPT

dltHub’s new datasets UI and lineage graphs are now available in private preview. We are launching Workspace Explorer today which brings two new views - Data Lineage and Job Graph tabs - into the dltHub UI.

When thinking about data quality, good teams and good operators ask for sources, provenance, and how data was managed. Questions like “Where did this figure come from?” or “What depends on this pipeline?” shouldn’t require an answer from whoever built the pipeline.

Whether you’re a CFO explaining figures to an auditor or a COO planning a change to an operational system, the connections need to be visible so data flows become company knowledge, not personal knowledge.

That’s a challenge as teams and agents are adding pipelines faster than anyone can document them. When dependencies aren’t clear, teams hesitate to change or remove anything. Old jobs stay in place, workarounds accumulate, and each change takes longer to check. A total rebuild is daunting.

Today, we’re introducing Datasets & Lineage on dltHub, with maps built from the information your pipelines record as they run, so you can see how your data flows. Together, three graphs are now in one place, and they make how your data flows company knowledge, accessible beyond the people who built it:

  • The lineage graph shows the sources and pipelines where data comes from;
  • The job dependency graph shows what triggers what;
  • The schema graph shows the tables and relationships inside a dataset.

See how your data flows

The lineage graph traces data from SQL databases, filesystems and REST APIs through pipelines to datasets. Clicking a node shows what dlt recorded about it.

For reporting and audit requests, this provides a source trail the team can use as supporting evidence. When someone asks which systems feed a dataset, the connections are there to inspect without reconstructing them by hand.

See what depends on what

The Job Graph tab in Workspace Explorer shows which jobs trigger others on success, failure or completion. Engineers can check downstream dependencies before changing a job, and see what still relies on older jobs before removing them.

When a job or pipeline’s last run has failed, its node turns red, along with everything downstream. The team can see the failed job and the chain that may be affected, helping them focus their response when a delivery deadline or SLA is at risk.

See what’s inside every dataset

Open a dataset in the new datasets UI to see its schema graph. This reveals the tables and the child tables dlt creates during loading, and how they relate.

Engineers can inspect the relationships of tables from a pipeline that an agent built and check whether they match what they expected. New hires can learn the tables and relationships from the same diagram, without needing the original author to walk them through the flow or the reasoning.

Start with a dataset your business relies on

The datasets UI and lineage graphs are available now in public preview. Open a dataset in dltHub to explore its tables and relationships.

Workspace Explorer, with its Data Lineage and Job Graph tabs, is currently in private preview. Reach out to us if you would like to get early access: support@dlthub.com.

Without needing any other tools, your pipelines already record these connections as they run, and dltHub turns them into a map your team can use.

Keep reading

All posts