---
title: "Release highlights: 1.31"
description: Release highlights provide a concise overview of the most important new features, improvements, and fixes in a software update.
keywords: [dlt, data-pipelines, etl, release-notes, data-engineering]
---
# Release highlights: 1.31

## Breaking changes

- **pendulum 3 required.** dlt now requires `pendulum>=3.0.0` on every Python version ([#4033](https://github.com/dlt-hub/dlt/pull/4033)).
- **Datetimes in json columns serialize with `+00:00`.** Datetimes stored inside json columns now end with `+00:00` instead of `Z` ([#4033](https://github.com/dlt-hub/dlt/pull/4033)).
- **`Incremental.last_value` is the current cursor position.** `last_value` is the actual cursor value at call time ([#4033](https://github.com/dlt-hub/dlt/pull/4033)). Before the first batch it equals `start_value`, then the running value. `lag` no longer applies to it, and it keeps advancing when `end_value` is set. This affects rest_api `{incremental.last_value}` placeholders on dependent resources.
- **Datetime cursor values follow the declared cursor type.** `start_value`, `last_value` and `get_current_range()` return `pendulum.DateTime` only for pendulum cursors ([#4406](https://github.com/dlt-hub/dlt/pull/4406)). `datetime.datetime` cursors now return stdlib datetimes. The `lag` unit comes from the declared cursor type or the type of `initial_value`, not from the string format of the values, which used to read `lag=3` as 3 seconds for ISO strings.
- **`Relation.incremental()` advances state by default.** With a cursor bound to a resource, `Relation.incremental()` now advances the stored cursor on each run ([#4033](https://github.com/dlt-hub/dlt/pull/4033)). Pass `advance=False` for the old stateless filter. Calling `.incremental()` twice on a relation combines both filters with AND.
- **Filesystem timestamps are stdlib `datetime`.** `modification_date` and other timestamps from the fsspec layer and the `filesystem` source are now stdlib `datetime` objects, tz-aware in UTC ([#4388](https://github.com/dlt-hub/dlt/pull/4388)). Code that calls pendulum-only methods on them breaks.
- **Snowflake tz-aware timestamps default to `TIMESTAMP_LTZ`.** New Snowflake tables store tz-aware timestamps as `TIMESTAMP_LTZ` instead of `TIMESTAMP_TZ` ([#4397](https://github.com/dlt-hub/dlt/pull/4397)). This keeps values consistent across loader file formats, since parquet could not carry offsets. Existing columns keep their type. Set `use_timestamp_tz=True` on the destination to keep `TIMESTAMP_TZ` when a consumer depends on the stored offset.
- **Delta upsert now honors `hard_delete`.** On the Delta table format, `upsert` now deletes records that the `hard_delete` column marks as deleted and never inserts them ([#4305](https://github.com/dlt-hub/dlt/pull/4305)). With nested tables, this combination raises `SchemaCorruptedException` before the load starts.

## New `cdc` merge strategy

The new `cdc` merge strategy treats each load as a complete snapshot of the source system ([#4305](https://github.com/dlt-hub/dlt/pull/4305)). It inserts new keys, updates changed keys, and deletes destination records that the snapshot no longer contains. It needs a unique `primary_key`. Supported on duckdb (1.4+), motherduck, ducklake, Snowflake, Postgres, BigQuery, Databricks, MSSQL, Fabric, Athena with Iceberg, and filesystem with Delta.

```py
import dlt

@dlt.resource(
    write_disposition={"disposition": "merge", "strategy": "cdc"},
    primary_key="id",
)
def customers():
    # each run yields a complete snapshot of the source table
    yield [{"id": 1, "name": "ada"}, {"id": 2, "name": "alan"}]
```

## Skip unchanged records when merging

Set `skip_unchanged_rows` on an `upsert` or `cdc` resource so dlt updates only the records that actually changed ([#4305](https://github.com/dlt-hub/dlt/pull/4305)). An unchanged record keeps its `_dlt_load_id`, so a Snowflake stream or similar consumer sees only real changes. Set `row_version_column_name` to compare one version or hash column instead of every column.

```py
import dlt

@dlt.resource(
    write_disposition={
        "disposition": "merge",
        "strategy": "upsert",
        "skip_unchanged_rows": True,
    },
    primary_key="id",
)
def customers():
    yield [{"id": 1, "name": "ada"}]
```

## Limit a merge with SQL conditions

Two new write-disposition options limit a merge with a SQL condition ([#4305](https://github.com/dlt-hub/dlt/pull/4305)). `source_filter` chooses which loaded records to merge. `destination_scope` chooses which destination records a merge may delete or retire. Together they enable partition replace, where BigQuery and Snowflake prune partitions, and keyless `delete-insert`. They work with `delete-insert`, `scd2` and `cdc`. `upsert` takes `source_filter` only.

```py
import dlt

@dlt.resource(
    write_disposition={
        "disposition": "merge",
        "strategy": "delete-insert",
        "destination_scope": "day = '2026-08-01'",
    },
)
def events():
    # yield only the records for 2026-08-01
    yield [{"id": 1, "day": "2026-08-01"}]
```

## Stateful incremental reads from a dataset

`Relation.incremental()` now pushes the cursor range down to SQL and keeps state between runs ([#4033](https://github.com/dlt-hub/dlt/pull/4033)). When the cursor is bound to a resource, each run locks its range end to `MAX(cursor)` and advances the stored cursor, so repeated runs cover the full range without gaps or duplicates. It supports date and datetime cursors, `range_start` / `range_end` overrides, and dotted cursor paths that auto-join another table. Pass `advance=False` for the old stateless filter.

```py
from datetime import datetime, timezone

import dlt

pipeline = dlt.pipeline("events_pipeline", destination="duckdb", dataset_name="events")


@dlt.resource(write_disposition="append")
def recent_events(
    created_at=dlt.sources.incremental(
        "created_at",
        initial_value=datetime(2026, 1, 1, tzinfo=timezone.utc),
        # the cursor is unique, so rows at the range end are not deferred
        primary_key="created_at",
    ),
):
    dataset = pipeline.dataset()
    events = dataset.table("events").incremental(created_at)
    yield dlt.mark.with_hints(
        events, dlt.mark.make_hints(columns=dataset.schema.tables["events"]["columns"])
    )


# each run reads only rows past the stored cursor, then advances it
pipeline.run(recent_events())
```

## Context timezone for stored timestamps

dlt can now store timestamps in a single process-wide timezone instead of always UTC ([#4033](https://github.com/dlt-hub/dlt/pull/4033)). The context timezone decides how naive and tz-aware timestamps are normalized for the `timezone` column hint, on both the object and arrow paths. Set it for a block of code with `TimezoneContext`, and read the active zone with `dlt.current.timezone()`. The default is still UTC.

```py
import dlt
from dlt.common.configuration.container import Container
from dlt.common.configuration.specs import TimezoneContext

@dlt.resource
def events():
    yield {"id": 1, "created_at": "2026-01-01T12:00:00"}

pipeline = dlt.pipeline("my_pipeline", destination="duckdb")

with Container().injectable_context(TimezoneContext("Europe/Berlin")):
    # timestamps in this run are normalized to Europe/Berlin, not UTC
    pipeline.run(events())
```

## Shout-out to new contributors

Big thanks to our newest contributors:

* [@anxkhn](https://github.com/anxkhn)
* [@sdebruyn](https://github.com/sdebruyn)
* [@Sanjays2402](https://github.com/Sanjays2402)
* [@adrienModeo](https://github.com/adrienModeo)
* [@bunnysayzz](https://github.com/bunnysayzz)
* [@eeshsaxena](https://github.com/eeshsaxena)
* [@chjnett](https://github.com/chjnett)
* [@arose26](https://github.com/arose26)
* [@tushardev-365](https://github.com/tushardev-365)
* [@eddietejeda](https://github.com/eddietejeda)
* [@jtcurlin](https://github.com/jtcurlin)
* [@simpleqt](https://github.com/simpleqt)
* [@rooperuu](https://github.com/rooperuu)
* [@schrodervictor](https://github.com/schrodervictor)

**Full release notes**

[View the 1.31.0 release notes](https://github.com/dlt-hub/dlt/releases/tag/1.31.0)