Skip to main content
Version: devel

dlt.extract.incremental.lag

DATE_STR_FORMATS​

Detected formats of string cursors that are dates and not datetimes

apply_lag​

def apply_lag(lag: float,
              initial_value: TCursorValue,
              last_value: TCursorValue,
              last_value_func: LastValueFunc[TCursorValue],
              cursor_type: Optional[Type[Any]] = None) -> TCursorValue

View source on GitHub

Applies lag to last_value but prevents it to cross initial_value: observing order of last_value_func

Lag is in days for a date cursor and in seconds for a datetime one, as declared by cursor_type.

apply_lag_with_suppression​

def apply_lag_with_suppression(
        lag: Optional[float],
        last_value_func: LastValueFunc[TCursorValue],
        initial_value: Optional[TCursorValue],
        end_value: Optional[TCursorValue],
        last_value: Optional[TCursorValue],
        resource_name: Optional[str] = None,
        cursor_type: Optional[Type[Any]] = None) -> Optional[TCursorValue]

View source on GitHub

Conditionally apply lag to last_value, mirroring Incremental.last_value rules.

Returns last_value unchanged when:

  • lag is falsy or last_value is None
  • last_value_func is not max or min (logs warning)
  • end_value is set (lag auto-deactivated; logs info)

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.