common.destination.capabilities
LoaderFileFormatSelector Objects
class LoaderFileFormatSelector(Protocol)
Selects preferred and supported file formats for a given table schema
MergeStrategySelector Objects
class MergeStrategySelector(Protocol)
Selects right set of merge strategies for a given table schema
DataTypeMapper Objects
class DataTypeMapper(ABC)
__init__
def __init__(capabilities: "DestinationCapabilitiesContext") -> None
Maps dlt data types into destination data types
to_destination_type
@abstractmethod
def to_destination_type(column: TColumnSchema,
table: PreparedTableSchema) -> str
Gets destination data type for a particular column
in prepared table
from_destination_type
@abstractmethod
def from_destination_type(db_type: str, precision: Optional[int],
scale: Optional[int]) -> TColumnType
Gets column type from db type
ensure_supported_type
@abstractmethod
def ensure_supported_type(column: TColumnSchema, table: PreparedTableSchema,
loader_file_format: TLoaderFileFormat) -> None
Makes sure that dlt type in column
in prepared table
is supported by the destination for a given file format
UnsupportedTypeMapper Objects
class UnsupportedTypeMapper(DataTypeMapper)
Type Mapper that can't map any type
DestinationCapabilitiesContext Objects
@configspec
class DestinationCapabilitiesContext(ContainerInjectableContext)
Injectable destination capabilities required for many Pipeline stages ie. normalize
loader_file_format_selector
Callable that adapts preferred_loader_file_format
and supported_loader_file_formats
at runtime.
recommended_file_size
Recommended file size in bytes when writing extract/load files
escape_identifier
Escapes table name, column name and other identifiers
escape_literal
Escapes string literal
casefold_identifier
Casing function applied by destination to represent case insensitive identifiers.
has_case_sensitive_identifiers
Tells if destination supports case sensitive identifiers
supports_clone_table
Destination supports CREATE TABLE ... CLONE ... statements
max_table_nesting
Allows a destination to overwrite max_table_nesting from source
max_parallel_load_jobs
The destination can set the maximum amount of parallel load jobs being executed
loader_parallelism_strategy
The destination can override the parallelism strategy
max_query_parameters
The maximum number of parameters that can be supplied in a single parametrized query
supports_native_boolean
The destination supports a native boolean type, otherwise bool columns are usually stored as integers
generates_case_sensitive_identifiers
def generates_case_sensitive_identifiers() -> bool
Tells if capabilities as currently adjusted, will generate case sensitive identifiers
merge_caps_file_formats
def merge_caps_file_formats(
destination: str, staging: str, dest_caps: DestinationCapabilitiesContext,
stage_caps: DestinationCapabilitiesContext
) -> Tuple[TLoaderFileFormat, Sequence[TLoaderFileFormat]]
Merges preferred and supported file formats from destination and staging. Returns new preferred file format and all possible formats.