Logs API
Logs of job runs, read once a run has written them or followed live while it runs.
Base URL and authentication
Each workspace keeps its data on the data plane of its region. Its services are not on
api.dlthub.com and do not accept API keys. To call them:
- Read
dataplane_urlfromGET https://api.dlthub.com/api/v1/workspaces/{workspace_id}. This is the base URL of every data plane service: Dataplane, Logs and Telemetry API. - Exchange your API key for a data plane token with
POST https://api.dlthub.com/api/v1/workspaces/{workspace_id}/-issue-dataplane-token. The token is valid for one workspace untilexpires_at, currently 10 minutes after it was issued; request a new one before then. - Call the service at
dataplane_urlwithAuthorization: Bearer <token>.
Every workspace member can read logs, telemetry and file lists. Only workspace owners can read
or change variables, so workspace API keys, which are DEVELOPER or VIEWER, cannot. Uploads use
a separate token: POST .../deployments/-issue-upload (or /configurations/-issue-upload)
returns an upload_token and the upload_url to send the files to, both valid for one upload
and 10 minutes.
The data planes today:
| Region | dataplane_url |
|---|---|
Europe (eu1, europe-west4) | https://eu1.dlthub.com |
US East (us1, us-east4) | https://us1.dlthub.com |
Versioning
The API version is part of the path. Endpoints under /v1 are stable: breaking changes ship
only under a new version prefix. New endpoints, optional parameters and response fields can
be added to /v1 at any time, so ignore fields and enum values you do not know.
Endpoints
Stream workspace run logs
GET /logs/v1/workspaces/{workspace_id}/job-runs/{job_run_id}/log-stream
Follows a job run's log as server-sent events (text/event-stream), starting from the
beginning of the run. Each log event's data is one log line object as JSON. The stream ends
when the run finishes. An error event with {"error": "..."} also ends it, for example
Run unavailable when no run is found within about 60 seconds. Unknown run ids therefore
return 200 followed by an error event, not 404.
Request
- Parameters
- curl
- Python
Path parameters
| Name | Type | Description |
|---|---|---|
workspace_id | string (uuid) | ID of the workspace |
job_run_id | string (uuid) | ID of the job run |
curl "$DATAPLANE_URL/logs/v1/workspaces/$WORKSPACE_ID/job-runs/$JOB_RUN_ID/log-stream" \
-H "Authorization: Bearer $DATAPLANE_TOKEN"
import os
import requests
response = requests.get(
"<dataplane_url>/logs/v1/workspaces/<workspace_id>/job-runs/<job_run_id>/log-stream",
headers={"Authorization": f"Bearer {os.environ['DATAPLANE_TOKEN']}"},
stream=True,
)
response.raise_for_status()
for line in response.iter_lines():
print(line.decode())
Response (200, 4xx, 5xx)
- Body
- Errors
200 returns text/event-stream.
400Bad Request401Unauthorized403Forbidden500Internal Server Error
Each error returns the error body.
Get workspace run logs
GET /logs/v1/workspaces/{workspace_id}/job-runs/{job_run_id}/logs
Returns a finished job run's stored log as newline-delimited JSON (application/x-ndjson),
one log line object per line. Answers 404 while the run is still going, for a run that produced
no output, and for an unknown run. To follow a running job, use log-stream.
Request
- Parameters
- curl
- Python
Path parameters
| Name | Type | Description |
|---|---|---|
workspace_id | string (uuid) | ID of the workspace |
job_run_id | string (uuid) | ID of the job run |
curl "$DATAPLANE_URL/logs/v1/workspaces/$WORKSPACE_ID/job-runs/$JOB_RUN_ID/logs" \
-H "Authorization: Bearer $DATAPLANE_TOKEN"
import os
import requests
response = requests.get(
"<dataplane_url>/logs/v1/workspaces/<workspace_id>/job-runs/<job_run_id>/logs",
headers={"Authorization": f"Bearer {os.environ['DATAPLANE_TOKEN']}"},
stream=True,
)
response.raise_for_status()
for line in response.iter_lines():
print(line.decode())
Response (200, 4xx, 5xx)
- Body
- Errors
200 returns application/x-ndjson.
400Bad Request401Unauthorized403Forbidden404Not Found500Internal Server Error
Each error returns the error body.