Skip to main content
Version: devel

common.storages.load_storage

LoadItemStorage Objects

class LoadItemStorage(DataItemStorage)

[view_source]

__init__

def __init__(package_storage: PackageStorage,
writer_spec: FileWriterSpec) -> None

[view_source]

Data item storage using storage to manage load packages

LoadStorage Objects

class LoadStorage(VersionedStorage)

[view_source]

NORMALIZED_FOLDER

folder within the volume where load packages are stored

LOADED_FOLDER

folder to keep the loads that were completely processed

NEW_PACKAGES_FOLDER

folder where new packages are created

list_new_jobs

def list_new_jobs(load_id: str) -> Sequence[str]

[view_source]

Lists all jobs in new jobs folder of normalized package storage and checks if file formats are supported

list_normalized_packages

def list_normalized_packages() -> Sequence[str]

[view_source]

Lists all packages that are normalized and will be loaded or are currently loaded

list_loaded_packages

def list_loaded_packages() -> Sequence[str]

[view_source]

List packages that are completely loaded

list_failed_jobs_in_loaded_package

def list_failed_jobs_in_loaded_package(load_id: str) -> Sequence[LoadJobInfo]

[view_source]

List all failed jobs and associated error messages for a completed load package with load_id

commit_schema_update

def commit_schema_update(load_id: str, applied_update: TSchemaTables) -> None

[view_source]

Marks schema update as processed and stores the update that was applied at the destination

import_new_job

def import_new_job(load_id: str,
job_file_path: str,
job_state: TPackageJobState = "new_jobs") -> None

[view_source]

Adds new job by moving the job_file_path into new_jobs of package load_id

maybe_remove_completed_jobs

def maybe_remove_completed_jobs(load_id: str) -> None

[view_source]

Deletes completed jobs if delete_completed_jobs config flag is set. If package has failed jobs, nothing gets deleted.

get_load_package_info

def get_load_package_info(load_id: str) -> LoadPackageInfo

[view_source]

Gets information on normalized OR loaded package with given load_id, all jobs and their statuses.

get_load_package_state

def get_load_package_state(load_id: str) -> TLoadPackageState

[view_source]

Gets state of normlized or loaded package with given load_id, all jobs and their statuses.

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.