Release highlights: 1.26

Breaking changes

• External schedulers now raise instead of warning. Resources with allow_external_schedulers=True no longer fall back to dlt state when a scheduler interval is missing. dlt raises ExternalSchedulerNotAvailable when no interval resolves, and JoinSchedulerError when the cursor type cannot coerce to a timestamp. Provide an interval, or drop the flag (#3877).

Join related tables with Relation.join()

Relation.join() composes a SQL join from a dataset's schema references, so you navigate parent/child tables (or annotated relationships) without writing the ON clause. Call it on a base-table relation and name the table to join. Columns from the joined table are prefixed with its name, or with the alias you pass (#3590).

import dlt

pipeline = dlt.pipeline(
    pipeline_name="shop", destination="duckdb", dataset_name="shop_data"
)
dataset = pipeline.dataset()

# join uses the schema's parent/child references, no ON clause needed
users_with_orders = dataset["users"].join("users__orders", alias="orders")
df = users_with_orders.select("name", "orders__order_id", "orders__total").df()

Read the scheduler interval with dlt.current.interval()

dlt.current.interval() returns the active (start, end) window from an external scheduler, or None when none is set. Any resource can read it, even ones not using Incremental, to scope requests, validation, or logging. Drive it from Airflow, the DLT_INTERVAL_START and DLT_INTERVAL_END env vars, or by injecting a TimeIntervalContext yourself (#3877).

import dlt

@dlt.resource
def my_resource():
    interval = dlt.current.interval()
    if interval is not None:
        start, end = interval
        # scope your requests to the [start, end) window
    yield {}

Snowflake query tags cover more operations

Snowflake query tagging now runs beyond load jobs. dlt tags sessions for storage setup, schema and state reads, schema updates, load completion, and table drops, each carrying a new operation field. Add {operation} to your query_tag template to see which dlt step a session is running (#3759).

[destination.snowflake]
query_tag='{{"operation":"{operation}", "source":"{source}", "resource":"{resource}", "table": "{table}", "load_id":"{load_id}", "pipeline_name":"{pipeline_name}"}}'

Shout-out to new contributors

Big thanks to our newest contributors:

• @bjoaquinc

Full release notes

View the 1.26.0 release notes