Telemetry

New to the SDK? Getting started shows how to connect and use it.

`Telemetry`

One workspace's telemetry, read from the data plane.

Name	Type	Example	Description
`telemetry.pipeline_runs`	`PipelineRuns`	`telemetry.pipeline_runs.list()`	The dlt pipeline runs telemetry recorded here.

`Telemetry.status()`

status() -> TelemetryStatus

Return when a pipeline run in this workspace was last recorded or updated.

If that is older than the window you read, empty results may mean telemetry is behind rather than that nothing ran.

Returns: TelemetryStatus — The status; awaitable in async mode.

Raises

Exception	When
`DataplaneTokenRejected`	The data plane rejected the token the platform issued for it.
`NotAuthorized`	The caller lacks the access this call needs; `message` says why.
`ScopeMissing`	Reached without a workspace in scope.

`Telemetry.pipelines()`

pipelines(
    *,
    pipeline_name: str | None = None,
    latest_status: PipelineRunStatus | None = None,
    destination: str | None = None,
    since: datetime | None = None,
    until: datetime | None = None,
    limit: int | None = None,
    offset: int = 0,
) -> Iterable[PipelineActivity] | AsyncIterable[PipelineActivity]

Yield one row per pipeline that ran in the window.

The window holds the pipeline runs whose start falls in [since, until).

Arguments

Name	Type	Description
`pipeline_name`	`str \| None`	Narrow to one pipeline.
`latest_status`	`PipelineRunStatus \| None`	Narrow to pipelines whose most recent run ended this way. Only `SUCCEEDED` and `FAILED` are filterable.
`destination`	`str \| None`	Narrow to pipelines most recently writing here.
`since`	`datetime \| None`	Start of the window, zone-aware. Defaults to seven days before `until`.
`until`	`datetime \| None`	End of the window, zone-aware. Defaults to now.
`limit`	`int \| None`	Return at most this many. `None` walks to the end, or to row 10,100, past which the platform does not page.
`offset`	`int`	Skip this many, server-side.

Returns: Iterable[PipelineActivity] | AsyncIterable[PipelineActivity] — An iterable of per-pipeline activity; async-iterable in async mode.

Raises

Exception	When
`BadRequest`	The request was rejected as invalid; `message`, `code` and `fields` say why.
`DataplaneTokenRejected`	The data plane rejected the token the platform issued for it.
`NotAuthorized`	The caller lacks the access this call needs; `message` says why.
`ScopeMissing`	Reached without a workspace in scope.
`ValueError`	A status the platform cannot filter on, a naive `since` or `until`, or a negative `limit` or `offset`.

`Telemetry.datasets()`

datasets(
    *,
    dataset_name: str | None = None,
    latest_status: PipelineRunStatus | None = None,
    destination: str | None = None,
    since: datetime | None = None,
    until: datetime | None = None,
    limit: int | None = None,
    offset: int = 0,
) -> Iterable[DatasetActivity] | AsyncIterable[DatasetActivity]

Yield one row per dataset written to in the window.

The window holds the pipeline runs whose start falls in [since, until).

Arguments

Name	Type	Description
`dataset_name`	`str \| None`	Narrow to one dataset.
`latest_status`	`PipelineRunStatus \| None`	Narrow to datasets whose most recent run ended this way. Only `SUCCEEDED` and `FAILED` are filterable.
`destination`	`str \| None`	Narrow to datasets most recently written here.
`since`	`datetime \| None`	Start of the window, zone-aware. Defaults to seven days before `until`.
`until`	`datetime \| None`	End of the window, zone-aware. Defaults to now.
`limit`	`int \| None`	Return at most this many. `None` walks to the end, or to row 10,100, past which the platform does not page.
`offset`	`int`	Skip this many, server-side.

Returns: Iterable[DatasetActivity] | AsyncIterable[DatasetActivity] — An iterable of per-dataset activity; async-iterable in async mode.

Raises

Exception	When
`BadRequest`	The request was rejected as invalid; `message`, `code` and `fields` say why.
`DataplaneTokenRejected`	The data plane rejected the token the platform issued for it.
`NotAuthorized`	The caller lacks the access this call needs; `message` says why.
`ScopeMissing`	Reached without a workspace in scope.
`ValueError`	A status the platform cannot filter on, a naive `since` or `until`, or a negative `limit` or `offset`.

`PipelineRuns`

The dlt pipeline runs telemetry recorded for one workspace.

Every read covers a time window, which defaults to the last seven days. There is no count(); walk list().

`PipelineRuns.get()`

get(*, id: str) -> PipelineRun

Return one pipeline run by id, in full.

Arguments

Name	Type	Description
`id`	`str`	The pipeline run's uuid, from `PipelineRuns.list()` or `PipelineRunSummary.pipeline_run_id`. A job run id will not do — they are different things.

Returns: PipelineRun — The pipeline run; awaitable in async mode.

Raises

Exception	When
`BadRequest`	The request was rejected as invalid; `message`, `code` and `fields` say why.
`DataplaneTokenRejected`	The data plane rejected the token the platform issued for it.
`NotAuthorized`	The caller lacks the access this call needs; `message` says why.
`NotFound`	No such pipeline run in this workspace.
`ScopeMissing`	Reached without a workspace in scope.

`PipelineRuns.trace()`

trace(*, id: str) -> PipelineRunTrace

Read one pipeline run's dlt trace, without reading the run itself.

Arguments

Name	Type	Description
`id`	`str`	The pipeline run's uuid.

Returns: PipelineRunTrace — The trace; awaitable in async mode.

Raises

Exception	When
`BadRequest`	The request was rejected as invalid; `message`, `code` and `fields` say why.
`NotFound`	No such pipeline run, or it has no stored trace. One the platform never linked to a job run has none.
`ScopeMissing`	Reached without a workspace in scope.

`PipelineRuns.list()`

list(
    *,
    job_run_id: str | None = None,
    pipeline_name: str | None = None,
    status: PipelineRunStatus | None = None,
    dataset: str | None = None,
    destination: str | None = None,
    empty_only: bool | None = None,
    since: datetime | None = None,
    until: datetime | None = None,
    limit: int | None = None,
    offset: int = 0,
) -> Iterable[PipelineRun] | AsyncIterable[PipelineRun]

Yield pipeline runs in the window, newest first.

The window holds the pipeline runs whose start falls in [since, until).

Arguments

Name	Type	Description
`job_run_id`	`str \| None`	Narrow to the pipelines one job run executed.
`pipeline_name`	`str \| None`	Narrow to one pipeline.
`status`	`PipelineRunStatus \| None`	Narrow to one outcome. Only `SUCCEEDED` and `FAILED` are filterable.
`dataset`	`str \| None`	Narrow to runs that wrote to one dataset.
`destination`	`str \| None`	Narrow to runs that wrote to one destination.
`empty_only`	`bool \| None`	True for succeeded runs that extracted nothing to load. False for every other run, failed ones included. `None` for both.
`since`	`datetime \| None`	Start of the window, zone-aware. Defaults to seven days before `until`.
`until`	`datetime \| None`	End of the window, zone-aware. Defaults to now.
`limit`	`int \| None`	Return at most this many. `None` walks to the end, or to row 10,100, past which the platform does not page.
`offset`	`int`	Skip this many, server-side.

Returns: Iterable[PipelineRun] | AsyncIterable[PipelineRun] — An iterable of pipeline runs; async-iterable in async mode.

Raises

Exception	When
`BadRequest`	The request was rejected as invalid; `message`, `code` and `fields` say why.
`DataplaneTokenRejected`	The data plane rejected the token the platform issued for it.
`NotAuthorized`	The caller lacks the access this call needs; `message` says why.
`ScopeMissing`	Reached without a workspace in scope.
`ValueError`	A status the platform cannot filter on, a naive `since` or `until`, or a negative `limit` or `offset`.

`PipelineRun`

One dlt pipeline run, as telemetry recorded it.

A job run executes whatever pipelines its job calls, so one run can produce several of these. Addressed by its own uuid, which is not a job run id. Get one from PipelineRuns.list() or PipelineRunSummary.pipeline_run_id.

Attributes

Name	Type	Description
`id`	`str`	ID of the pipeline run; not a job run ID.
`pipeline_name`	`str`	Name of the dlt pipeline.
`status`	`PipelineRunStatus`	`failed` when any step raised, otherwise `succeeded`.
`dataset_name`	`str \| None`	Dataset the run loaded into, from its load step; `None` when the run did not reach load.
`destination_name`	`str \| None`	Destination the run loaded to, from its load step; `None` when the run did not reach load.
`started_at`	`datetime \| None`	Start of the run from the dlt trace, UTC.
`finished_at`	`datetime \| None`	End of the run from the dlt trace, UTC.
`duration_ms`	`int \| None`	`finished_at` − `started_at` in milliseconds.
`total_rows_extracted`	`int`	Rows written by the extract step.
`total_rows_loaded`	`int`	Rows that dlt prepared for loading in this run (normalized rows). dlt does not report per-row load counts, so this is set even when loading failed.
`total_bytes_loaded`	`int`	Bytes that dlt prepared for loading in this run (size of the normalized files). dlt does not report per-row load counts, so this is set even when loading failed.
`error_step`	`str \| None`	First step that raised, for example `extract`, `normalize` or `load`.
`error_message`	`str \| None`	Exception message of that step.
`has_schema_migration`	`bool \| None`	True when the load step created or changed user tables (`_dlt_*` tables are ignored).
`dlt_version`	`str \| None`	dlt version that ran the pipeline, from the trace's execution context.
`tables`	`tuple[PipelineTable, ...]`	The tables it wrote, with rows per table. Empty on a listed row — only `PipelineRuns.get()` reads them.
`load_packages`	`tuple[LoadPackage, ...]`	The packages it produced. Empty on a listed row, for the same reason.
`failed`	`bool`	Whether this pipeline run ended on an error.

`PipelineRun.trace()`

trace() -> PipelineRunTrace

Read the dlt trace this pipeline run produced.

Returns: PipelineRunTrace — The trace; awaitable in async mode.

Raises

Exception	When
`NotFound`	No trace was stored. A pipeline run the platform never linked to a job run has none.
`ScopeMissing`	Reached without a workspace in scope.

`DatasetActivity`

What landed in one dataset over a window.

Attributes

Name	Type	Description
`dataset_name`	`str`	Name of the dataset, across destinations.
`distinct_pipeline_count`	`int`	Number of distinct pipelines that ran into this dataset in the range.
`total_pipeline_run_count`	`int`	Pipeline runs in the range.
`succeeded_pipeline_run_count`	`int`	Pipeline runs in the range with status `succeeded`.
`failed_pipeline_run_count`	`int`	How many did not. Derived, as for a pipeline.
`success_ratio`	`float`	Succeeded ÷ total, from 0 to 1.
`total_rows_loaded`	`int`	Sum of normalized rows over succeeded runs in the time range.
`total_bytes_loaded`	`int`	Sum of normalized bytes (size of the normalized files) over succeeded runs in the time range.
`schema_migration_count`	`int`	Runs in the range with `has_schema_migration`.
`average_duration_ms`	`int \| None`	Mean `duration_ms` in the range, rounded; `None` when no run has a duration.
`latest_status`	`str \| None`	Status of the most recent pipeline run in the time range.
`latest_pipeline_run_at`	`datetime \| None`	When the most recent pipeline run in the time range started.
`latest_pipeline_name`	`str \| None`	Pipeline of the most recent pipeline run in the time range.
`latest_destination_name`	`str \| None`	Destination of the most recent pipeline run in the time range.

`LoadPackage`

One load package a pipeline run produced.

A run loads in packages, so a partial failure shows up here as some packages complete and others not.

Attributes

Name	Type	Description
`load_id`	`str`	dlt load id of the package.
`schema_name`	`str \| None`	dlt schema the package was created with.
`load_status`	`str \| None`	State dlt reported for the package; `loaded` means fully loaded.
`job_count`	`int`	Number of jobs in the package.
`failed_job_count`	`int`	Jobs that failed.
`total_rows_loaded`	`int`	Rows that dlt prepared for loading in this package (normalized rows, or extracted rows when normalize reported none). dlt does not report per-row load counts, so this is set even when loading failed.
`total_bytes_loaded`	`int`	Bytes that dlt prepared for loading in this package (size of the normalized files, or of the extracted files when normalize reported none). dlt does not report per-row load counts, so this is set even when loading failed.
`is_pending`	`bool`	Whether it is still waiting to be loaded.
`has_schema_migration`	`bool`	Whether loading it changed the destination's schema.
`completed_at`	`datetime \| None`	When dlt completed the package, UTC; `None` while not completed.

`PipelineActivity`

How one pipeline has been doing over a window.

Attributes

Name	Type	Description
`pipeline_name`	`str`	Name of the dlt pipeline.
`total_pipeline_run_count`	`int`	Pipeline runs in the range.
`succeeded_pipeline_run_count`	`int`	Pipeline runs in the range with status `succeeded`.
`failed_pipeline_run_count`	`int`	How many did not. Derived, since the platform reports the successes and the total.
`success_ratio`	`float`	Succeeded ÷ total, from 0 to 1.
`total_rows_extracted`	`int`	Sum of extracted rows over all runs in the range.
`total_rows_loaded`	`int`	Sum of normalized rows over succeeded runs in the time range.
`total_bytes_loaded`	`int`	Sum of normalized bytes (size of the normalized files) over succeeded runs in the time range.
`average_duration_ms`	`int \| None`	Mean `duration_ms` in the range, rounded; `None` when no run has a duration.
`total_duration_ms`	`int`	Sum of `duration_ms` over the runs in the range.
`latest_status`	`str \| None`	Status of the most recent pipeline run in the time range.
`latest_pipeline_run_at`	`datetime \| None`	When the most recent pipeline run in the time range started.
`latest_dataset_name`	`str \| None`	Dataset of the most recent pipeline run in the time range.
`latest_destination_name`	`str \| None`	Destination of the most recent pipeline run in the time range.
`latest_source_names`	`tuple[str, ...]`	Sources read by the most recent pipeline run in the time range.

`PipelineRunTrace`

The dlt trace one pipeline run produced.

Attributes

Name	Type	Description
`pipeline_run_id`	`str`	The pipeline run it belongs to.
`trace`	`Mapping[str, Any]`	The trace exactly as dlt wrote it — per-step timings, the resolved config, and the exception when a step raised. Left opaque: the platform stores and returns it verbatim and declares no shape for it, as with the definitions `Jobs.deploy_manifest` accepts.

`PipelineTable`

One table a pipeline run wrote, and what reached it.

Attributes

Name	Type	Description
`table_name`	`str`	Name of the table.
`schema_name`	`str`	The schema it sits in.
`rows_extracted`	`int \| None`	Rows the extract step wrote for this table; `None` when not extracted.
`rows_loaded`	`int \| None`	Rows that dlt prepared for loading for this table (normalized rows). dlt does not report per-row load counts, so this is set even when loading failed.
`bytes_loaded`	`int \| None`	Bytes that dlt prepared for loading for this table (size of the normalized files). dlt does not report per-row load counts, so this is set even when loading failed.
`load_job_state`	`str \| None`	`completed` or `failed`; `None` when the table was not loaded.
`write_disposition`	`str \| None`	From the resource hints; `None` when the extract step reported none.

`TelemetryStatus`

How current the platform's telemetry is for one workspace.

Attributes

Name	Type	Description
`updated_at`	`datetime \| None`	When a pipeline run in the workspace was last recorded or updated; `None` when there are none. If it is older than the time range you read, empty results may mean telemetry is behind rather than that nothing ran.

`PipelineRunStatus`

What became of one dlt pipeline run.

The platform records a run once dlt reports it finished, so only SUCCEEDED and FAILED occur; UNKNOWN covers values added later.

Members

Member	Value	Description
`SUCCEEDED`	`'succeeded'`	No dlt step raised an exception; some load jobs may still have failed.
`FAILED`	`'failed'`	Stopped on an error. `error_step` says where.
`UNKNOWN`	`'unknown'`	A status this SDK version does not know.

Telemetry

`Telemetry`

`Telemetry.status()`

`Telemetry.pipelines()`

`Telemetry.datasets()`

`PipelineRuns`

`PipelineRuns.get()`

`PipelineRuns.trace()`

`PipelineRuns.list()`

`PipelineRun`

`PipelineRun.trace()`

`DatasetActivity`

`LoadPackage`

`PipelineActivity`

`PipelineRunTrace`

`PipelineTable`

`TelemetryStatus`

`PipelineRunStatus`

DHelp

Ask a question

Telemetry​

Telemetry.status()​

Telemetry.pipelines()​

Telemetry.datasets()​

PipelineRuns​

PipelineRuns.get()​

PipelineRuns.trace()​

PipelineRuns.list()​

PipelineRun​

PipelineRun.trace()​

DatasetActivity​

LoadPackage​

PipelineActivity​

PipelineRunTrace​

PipelineTable​

TelemetryStatus​

PipelineRunStatus​

DHelp

Ask a question

`Telemetry`

`Telemetry.status()`

`Telemetry.pipelines()`

`Telemetry.datasets()`

`PipelineRuns`

`PipelineRuns.get()`

`PipelineRuns.trace()`

`PipelineRuns.list()`

`PipelineRun`

`PipelineRun.trace()`

`DatasetActivity`

`LoadPackage`

`PipelineActivity`

`PipelineRunTrace`

`PipelineTable`

`TelemetryStatus`

`PipelineRunStatus`