dlt.pipeline.abort
TAbortPackageInfo Objects
class TAbortPackageInfo(TypedDict)
The single package to be aborted with a record and how its jobs will be processed.
terminal_jobs
retried or failed-in-flight jobs with terminal errors, will be moved to failed_jobs
transient_jobs
retried jobs with transient errors, will be moved to failed_jobs
interrupted_jobs
started jobs without a recorded outcome, will be moved to failed_jobs
committed_jobs
started jobs already committed to the destination, will be moved to completed_jobs
TAbortPlan Objects
class TAbortPlan(TypedDict)
Plan of what a pipeline abort will remove and where the local state rewinds to.
package_to_abort
the oldest normalized package (the one being loaded), aborted with a record; None otherwise
restore_from_load_id
oldest package being removed; local state and schemas rewind to its start snapshot
prepare_abort_packages
def prepare_abort_packages(
load_storage: LoadStorage,
normalize_storage: NormalizeStorage,
load_id: Optional[str] = None) -> Optional[TAbortPlan]
Plans an abort of packages starting at load_id (the oldest pending when None), or None
when nothing is pending. Packages older than load_id are left intact; load_id and
everything newer, extracted packages included, are removed and the local state rewinds to
load_id's start. The oldest pending normalized package (the one being loaded) is the only
one aborted with a record; its jobs are classified the way the loader will process them.
load_id may already be aborted (moved to loaded storage) so this planner can drive the
post-abort cleanup too.
execute_abort_plan
def execute_abort_plan(
load_storage: LoadStorage, normalize_storage: NormalizeStorage,
plan: TAbortPlan) -> Optional[Tuple[Optional[str], Dict[str, str]]]
Deletes the packages the plan marked for deletion and returns the restore snapshot for the caller to install (or None). The snapshot is read before any deletion because the package that carries it may itself be deleted.