common.destination.capabilities
LoaderFileFormatAdapter Objects
class LoaderFileFormatAdapter(Protocol)
Callback protocol for loader_file_format_adapter
capability.
DestinationCapabilitiesContext Objects
@configspec
class DestinationCapabilitiesContext(ContainerInjectableContext)
Injectable destination capabilities required for many Pipeline stages ie. normalize
loader_file_format_adapter
Callable that adapts preferred_loader_file_format
and supported_loader_file_formats
at runtime.
supported_table_formats
type: ignore[name-defined] # noqa: F821
recommended_file_size
Recommended file size in bytes when writing extract/load files
escape_identifier
Escapes table name, column name and other identifiers
escape_literal
Escapes string literal
casefold_identifier
Casing function applied by destination to represent case insensitive identifiers.
has_case_sensitive_identifiers
Tells if destination supports case sensitive identifiers
supports_clone_table
Destination supports CREATE TABLE ... CLONE ... statements
max_table_nesting
Allows a destination to overwrite max_table_nesting from source
supported_merge_strategies
type: ignore[name-defined] # noqa: F821
max_parallel_load_jobs
The destination can set the maxium amount of parallel load jobs being executed
loader_parallelism_strategy
The destination can override the parallelism strategy
generates_case_sensitive_identifiers
def generates_case_sensitive_identifiers() -> bool
Tells if capabilities as currently adjusted, will generate case sensitive identifiers
merge_caps_file_formats
def merge_caps_file_formats(
destination: str, staging: str, dest_caps: DestinationCapabilitiesContext,
stage_caps: DestinationCapabilitiesContext
) -> Tuple[TLoaderFileFormat, Sequence[TLoaderFileFormat]]
Merges preferred and supported file formats from destination and staging. Returns new preferred file format and all possible formats.