common.destination.reference

Destination Objects

class Destination(ABC, Generic[TDestinationConfig, TDestinationClient])

[view_source]

A destination factory that can be partially pre-configured with credentials and other config params.

config_params

Explicit config params, overriding any injected or default values.

caps_params

Explicit capabilities params, overriding any default values for this destination

spec

@property
@abstractmethod
def spec() -> Type[TDestinationConfig]

[view_source]

A spec of destination configuration that also contains destination credentials

capabilities

def capabilities(
    config: Optional[TDestinationConfig] = None,
    naming: Optional[NamingConvention] = None
) -> DestinationCapabilitiesContext

[view_source]

Destination capabilities ie. supported loader file formats, identifier name lengths, naming conventions, escape function etc. Explicit caps arguments passed to the factory init and stored in caps_params are applied.

If config is provided, it is used to adjust the capabilities, otherwise the explicit config composed just of config_params passed to factory init is applied If naming is provided, the case sensitivity and case folding are adjusted.

destination_name

@property
def destination_name() -> str

[view_source]

The destination name will either be explicitly set while creating the destination or will be taken from the type

configured_name

@property
def configured_name() -> str

[view_source]

Configured destination name, None by default

client_class

@property
@abstractmethod
def client_class() -> Type[TDestinationClient]

[view_source]

A job client class responsible for starting and resuming load jobs

configuration

def configuration(initial_config: TDestinationConfig,
                  accept_partial: bool = False) -> TDestinationConfig

[view_source]

Get a fully resolved destination config from the initial config

client

def client(schema: Schema,
           initial_config: TDestinationConfig = None) -> TDestinationClient

[view_source]

Returns a configured instance of the destination's job client

adjust_capabilities

@classmethod
def adjust_capabilities(
        cls, caps: DestinationCapabilitiesContext, config: TDestinationConfig,
        naming: Optional[NamingConvention]) -> DestinationCapabilitiesContext

[view_source]

Adjust the capabilities to match the case sensitivity as requested by naming convention.

register

@classmethod
def register(cls) -> None

[view_source]

Registers this factory class under module.name of a Destination factory

from_reference

@classmethod
def from_reference(cls,
                   ref: TDestinationReferenceArg,
                   credentials: Optional[Any] = None,
                   destination_name: Optional[str] = None,
                   environment: Optional[str] = None,
                   **kwargs: Any) -> Optional[AnyDestination]

[view_source]

Instantiate destination from a string reference or one of supported forms. This methods obtains a destination factory and then instantiates it passing the arguments after ref

DestinationReference Objects

class DestinationReference()

[view_source]

A registry of destination factories with a set of method for finding and instantiating

DESTINATIONS

A registry of all the destination factories

normalize_type

@staticmethod
def normalize_type(destination_type: str) -> str

[view_source]

Normalizes destination type string into a canonical form. Assumes that type names without dots correspond to built in destinations.

register

@classmethod
def register(cls, factory: Type[AnyDestination_CO], ref: str) -> None

[view_source]

Registers factory class under ref. ref

to_fully_qualified_refs

@staticmethod
def to_fully_qualified_refs(ref: str) -> List[str]

[view_source]

Converts ref into fully qualified form, return one or more alternatives for shorthand notations. Run context is injected if needed. Following formats are recognized

• name NOTE: the last component of destination type serves as destination name if not explicitly specified

find

@classmethod
def find(
    cls,
    ref: str,
    raise_exec_errors: bool = False,
    import_missing_modules: bool = False
) -> Union[Type[AnyDestination], Callable[..., AnyDestination]]

[view_source]

Finds or auto-imports destination factory that can be further called in order to instantiate it The ref can be a destination name or import path pointing to a destination class (e.g. dlt.destinations.postgres) You can control auto-import behavior:

• raise_exec_errors - will re-raise code execution errors in imported modules

• import_missing_modules - will ignore missing dependencies during import by substituting them with dummy modules. this should be only used to manipulate local dev environment

  NOTE: find returns a factory class or a callable that will create factory instance (for custom destinations) use ensure_factory to extract factory class from callable TODO: synthesize a call on custom destination (destination) factory type so this distinction is not needed

ensure_factory

@classmethod
def ensure_factory(
    cls, ref_factory: Union[Type[AnyDestination], Callable[...,
                                                           AnyDestination]]
) -> Type[AnyDestination]

[view_source]

Extract factory type from a callable creating factory instance.

from_reference

@classmethod
def from_reference(cls,
                   ref: str,
                   credentials: Optional[Any] = None,
                   destination_name: Optional[str] = None,
                   environment: Optional[str] = None,
                   **kwargs: Any) -> Optional[AnyDestination]

[view_source]

Instantiate destination from str reference. The ref can be a destination name or import path pointing to a destination class (e.g. dlt.destinations.postgres) This methods obtains a destination factory and then instantiates it passing the arguments after ref