Skip to main content
Version: devel View Markdown

Telemetry API

Telemetry of a workspace's pipelines: pipeline runs and their traces, the results job runs declare, and per-pipeline and per-dataset statistics.

Base URL and authentication​

Each workspace keeps its data on the data plane of its region. Its services are not on api.dlthub.com and do not accept API keys. To call them:

  1. Read dataplane_url from GET https://api.dlthub.com/api/v1/workspaces/{workspace_id}. This is the base URL of every data plane service: Dataplane, Logs and Telemetry API.
  2. Exchange your API key for a data plane token with POST https://api.dlthub.com/api/v1/workspaces/{workspace_id}/-issue-dataplane-token. The token is valid for one workspace until expires_at, currently 10 minutes after it was issued; request a new one before then.
  3. Call the service at dataplane_url with Authorization: Bearer <token>.

Every workspace member can read logs, telemetry and file lists. Only workspace owners can read or change variables, so workspace API keys, which are DEVELOPER or VIEWER, cannot. Uploads use a separate token: POST .../deployments/-issue-upload (or /configurations/-issue-upload) returns an upload_token and the upload_url to send the files to, both valid for one upload and 10 minutes.

The data planes today:

Regiondataplane_url
Europe (eu1, europe-west4)https://eu1.dlthub.com
US East (us1, us-east4)https://us1.dlthub.com

Versioning​

The API version is part of the path. Endpoints under /v1 are stable: breaking changes ship only under a new version prefix. New endpoints, optional parameters and response fields can be added to /v1 at any time, so ignore fields and enum values you do not know.

Endpoints​

List dataset overview​

GET /telemetry/v1/workspaces/{workspace_id}/datasets/-/stats

One row per dataset, aggregated over the pipeline runs whose started_at falls in the time range [start, end). Most recently run first by default.

Datasets are grouped by name across destinations. Runs without a dataset, for example ones that failed before loading, are not counted.

Search with q over the dataset name, filter on what a row displays, and sort with paired sort and order lists.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace

Query parameters

NameTypeDescription
startstring (date-time) or nullLower bound (inclusive) on started_at, as a naive local time read in timezone, e.g. 2026-03-01T00:00:00. Defaults to 7 days before end.
endstring (date-time) or nullUpper bound (exclusive) on started_at, as a naive local time read in timezone, e.g. 2026-03-11T00:00:00. Defaults to now.
timezonestringIANA timezone in which start and end are read, e.g. Europe/Berlin. Default UTC.
qstring or nullCase-insensitive substring match on dataset_name.
dataset_namearray of string or nullExact match on dataset_name. Repeat the parameter to match any of several values.
latest_statusarray of string or nullExact match on latest_status. Repeat the parameter to match any of several values. One of: succeeded, failed.
latest_destination_namearray of string or nullExact match on latest_destination_name. Repeat the parameter to match any of several values.
sortarray of string or nullKeys to sort by, applied in the order given. Pairs positionally with order, which must have the same number of entries. One of: latest_pipeline_run_at, total_pipeline_run_count, success_ratio, average_duration_ms, total_rows_loaded, dataset_name.

Pagination: limit, offset, order, cursor, see Lists and pagination

Response (200, 4xx, 5xx)
{
"items": [
{
"average_duration_ms": 0,
"dataset_name": "<dataset_name>",
"distinct_pipeline_count": 0,
"latest_destination_name": "<latest_destination_name>",
"latest_pipeline_name": "<latest_pipeline_name>",
"latest_pipeline_run_at": "2026-01-01T00:00:00Z",
"latest_pipeline_run_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"latest_status": "succeeded",
"schema_migration_count": 0,
"succeeded_pipeline_run_count": 0,
"success_ratio": 0,
"table_count": 0,
"total_bytes_loaded": 0,
"total_pipeline_run_count": 0,
"total_rows_loaded": 0
}
],
"limit": 0,
"next_cursor": "<next_cursor>",
"offset": 0,
"total": 0
}

Get job result​

GET /telemetry/v1/workspaces/{workspace_id}/job-runs/{job_run_id}/result

The result a job run declared: for an agent, its status, summary, token usage and cost; for other jobs, the declared result. A run has at most one.

job_run_id is the job run's id from the runtime API, not a pipeline_run_id. Answers 404 when the run declared no result. For the agent's per-turn tool calls and everything else the job delivered, read /result-trace.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace
job_run_idstring (uuid)ID of the job run
Response (200, 4xx, 5xx)
{
"agent_status": "<agent_status>",
"cost_usd": "<cost_usd>",
"created_at": "2026-01-01T00:00:00Z",
"engine_version": 0,
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"input_tokens": 0,
"job_ref": "<job_ref>",
"job_run_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"loop_type": "<loop_type>",
"model": "<model>",
"output_tokens": 0,
"result": "<result>",
"result_type": "<result_type>",
"stop_reason": "<stop_reason>",
"summary": "<summary>",
"total_tokens": 0,
"turn_count": 0,
"updated_at": "2026-01-01T00:00:00Z"
}

Get job result trace​

GET /telemetry/v1/workspaces/{workspace_id}/job-runs/{job_run_id}/result-trace

The full result exactly as the job delivered it, including the agent's per-turn tool calls. Fields beyond type and job_ref are passed through unchanged. job_run_id is the job run's id from the runtime API, not a pipeline_run_id. Answers 404 when the run declared no result.

Results over 8 MiB answer 413 job_result_trace_too_large; the summary fields stay available at /result.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace
job_run_idstring (uuid)ID of the job run
Response (200, 4xx, 5xx)
{
"job_ref": "<job_ref>",
"type": "<type>"
}

List pipeline runs​

GET /telemetry/v1/workspaces/{workspace_id}/pipeline-runs

Lists the pipeline runs of a workspace whose started_at falls in the time range [start, end), newest first by default.

Search with q over the pipeline name; filter by status, pipeline, dataset, destination, job run, or empty run; sort with paired sort and order lists.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace

Query parameters

NameTypeDescription
startstring (date-time) or nullLower bound (inclusive) on started_at, as a naive local time read in timezone, e.g. 2026-03-01T00:00:00. Defaults to 7 days before end.
endstring (date-time) or nullUpper bound (exclusive) on started_at, as a naive local time read in timezone, e.g. 2026-03-11T00:00:00. Defaults to now.
timezonestringIANA timezone in which start and end are read, e.g. Europe/Berlin. Default UTC.
qstring or nullCase-insensitive substring match on pipeline_name.
statusarray of string or nullExact match on status. Repeat the parameter to match any of several values. One of: succeeded, failed.
pipeline_namearray of string or nullExact match on pipeline_name. Repeat the parameter to match any of several values.
dataset_namearray of string or nullExact match on dataset_name. Repeat the parameter to match any of several values.
destination_namearray of string or nullExact match on destination_name. Repeat the parameter to match any of several values.
job_run_idarray of string (uuid) or nullExact match on job_run_id. Repeat the parameter to match any of several values.
is_empty_runboolean or nullMatch is_empty_run. Omit to list both.
sortarray of string or nullKeys to sort by, applied in the order given. Pairs positionally with order, which must have the same number of entries. One of: started_at, duration_ms, total_rows_loaded, total_bytes_loaded, finished_at.

Pagination: limit, offset, order, cursor, see Lists and pagination

Response (200, 4xx, 5xx)
{
"items": [
{
"created_at": "2026-01-01T00:00:00Z",
"dataset_name": "<dataset_name>",
"destination_name": "<destination_name>",
"dlt_version": "<dlt_version>",
"duration_ms": 0,
"error_message": "<error_message>",
"error_step": "<error_step>",
"finished_at": "2026-01-01T00:00:00Z",
"has_schema_migration": false,
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"pipeline_name": "<pipeline_name>",
"started_at": "2026-01-01T00:00:00Z",
"status": "succeeded",
"total_bytes_loaded": 0,
"total_rows_extracted": 0,
"total_rows_loaded": 0,
"updated_at": "2026-01-01T00:00:00Z"
}
],
"limit": 0,
"next_cursor": "<next_cursor>",
"offset": 0,
"total": 0
}

Get pipeline run​

GET /telemetry/v1/workspaces/{workspace_id}/pipeline-runs/{pipeline_run_id}

One pipeline run (one dlt pipeline.run()), looked up by its id, with its per-table and per-resource metrics and its load packages. Answers 404 when it is not in the workspace.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace
pipeline_run_idstring (uuid)ID of the pipeline run
Response (200, 4xx, 5xx)
{
"created_at": "2026-01-01T00:00:00Z",
"dataset_name": "<dataset_name>",
"destination_name": "<destination_name>",
"dlt_plus_version": "<dlt_plus_version>",
"dlt_version": "<dlt_version>",
"duration_ms": 0,
"error_message": "<error_message>",
"error_step": "<error_step>",
"error_traceback": "<error_traceback>",
"extract_duration_ms": 0,
"finished_at": "2026-01-01T00:00:00Z",
"has_failed_load_jobs": false,
"has_pending_packages": false,
"has_schema_migration": false,
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"is_empty_run": false,
"job_run_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
"load_duration_ms": 0,
"load_package_count": 0,
"load_packages": [],
"normalize_duration_ms": 0,
"os_name": "<os_name>",
"os_version": "<os_version>",
"pipeline_name": "<pipeline_name>",
"python_version": "<python_version>",
"resources": [],
"started_at": "2026-01-01T00:00:00Z",
"status": "succeeded",
"tables": [],
"total_bytes_extracted": 0,
"total_bytes_loaded": 0,
"total_rows_extracted": 0,
"total_rows_loaded": 0,
"transaction_id": "<transaction_id>",
"updated_at": "2026-01-01T00:00:00Z"
}

Get pipeline run trace​

GET /telemetry/v1/workspaces/{workspace_id}/pipeline-runs/{pipeline_run_id}/trace

Returns the dlt trace of this pipeline run as dlt produced it: its extract, normalize and load steps and the execution context. Answers 404 when the run or its trace is not found.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace
pipeline_run_idstring (uuid)ID of the pipeline run
Response (200, 4xx, 5xx)
{
"execution_context": {},
"pipeline_name": "<pipeline_name>",
"steps": [],
"transaction_id": "<transaction_id>"
}

List pipeline overview​

GET /telemetry/v1/workspaces/{workspace_id}/pipelines/-/stats

One row per pipeline, aggregated over the pipeline runs whose started_at falls in the time range [start, end). Most recently run first by default.

Search with q over the pipeline name, filter on what a row displays, and sort with paired sort and order lists.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace

Query parameters

NameTypeDescription
startstring (date-time) or nullLower bound (inclusive) on started_at, as a naive local time read in timezone, e.g. 2026-03-01T00:00:00. Defaults to 7 days before end.
endstring (date-time) or nullUpper bound (exclusive) on started_at, as a naive local time read in timezone, e.g. 2026-03-11T00:00:00. Defaults to now.
timezonestringIANA timezone in which start and end are read, e.g. Europe/Berlin. Default UTC.
qstring or nullCase-insensitive substring match on pipeline_name.
pipeline_namearray of string or nullExact match on pipeline_name. Repeat the parameter to match any of several values.
latest_statusarray of string or nullExact match on latest_status. Repeat the parameter to match any of several values. One of: succeeded, failed.
latest_destination_namearray of string or nullExact match on latest_destination_name. Repeat the parameter to match any of several values.
sortarray of string or nullKeys to sort by, applied in the order given. Pairs positionally with order, which must have the same number of entries. One of: latest_pipeline_run_at, total_pipeline_run_count, success_ratio, average_duration_ms, total_rows_loaded, pipeline_name.

Pagination: limit, offset, order, cursor, see Lists and pagination

Response (200, 4xx, 5xx)
{
"items": [
{
"average_duration_ms": 0,
"latest_dataset_name": "<latest_dataset_name>",
"latest_destination_name": "<latest_destination_name>",
"latest_pipeline_run_at": "2026-01-01T00:00:00Z",
"latest_source_names": ["<latest_source_names>"],
"latest_status": "succeeded",
"pipeline_name": "<pipeline_name>",
"succeeded_pipeline_run_count": 0,
"success_ratio": 0,
"total_bytes_loaded": 0,
"total_duration_ms": 0,
"total_pipeline_run_count": 0,
"total_rows_extracted": 0,
"total_rows_loaded": 0
}
],
"limit": 0,
"next_cursor": "<next_cursor>",
"offset": 0,
"total": 0
}

Get telemetry watermark​

GET /telemetry/v1/workspaces/{workspace_id}/watermark

The most recent updated_at of any pipeline run in the workspace, or null when there are none. Poll this and refetch lists only when it changes.

Request

Path parameters

NameTypeDescription
workspace_idstring (uuid)ID of the workspace
Response (200, 4xx, 5xx)
{
"updated_at": "2026-01-01T00:00:00Z"
}

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.