dlt.common.destination.reference
Destination Objects
class Destination(ABC, Generic[TDestinationConfig, TDestinationClient])
A destination factory that can be partially pre-configured with credentials and other config params.
config_params
Explicit config params, overriding any injected or default values.
caps_params
Explicit capabilities params, overriding any default values for this destination
spec
@property
@abstractmethod
def spec() -> Type[TDestinationConfig]
A spec of destination configuration that also contains destination credentials
capabilities
def capabilities(
config: Optional[TDestinationConfig] = None,
naming: Optional[NamingConvention] = None
) -> DestinationCapabilitiesContext
Destination capabilities ie. supported loader file formats, identifier name lengths, naming conventions, escape function etc.
Explicit caps arguments passed to the factory init and stored in caps_params are applied.
If config is provided, it is used to adjust the capabilities, otherwise the explicit config composed just of config_params passed
to factory init is applied
If naming is provided, the case sensitivity and case folding are adjusted.
destination_name
@property
def destination_name() -> str
The destination name will either be explicitly set while creating the destination or will be taken from the type
configured_name
@property
def configured_name() -> str
Configured destination name, None by default
client_class
@property
@abstractmethod
def client_class() -> Type[TDestinationClient]
A job client class responsible for starting and resuming load jobs
configuration
def configuration(initial_config: TDestinationConfig,
accept_partial: bool = False) -> TDestinationConfig
Get a fully resolved destination config from the initial config
client
def client(schema: Schema,
initial_config: TDestinationConfig = None) -> TDestinationClient
Returns a configured instance of the destination's job client
adjust_capabilities
@classmethod
def adjust_capabilities(
cls, caps: DestinationCapabilitiesContext, config: TDestinationConfig,
naming: Optional[NamingConvention]) -> DestinationCapabilitiesContext
Adjust the capabilities to match the case sensitivity as requested by naming convention.
register
@classmethod
def register(cls) -> None
Registers this factory class under module.name of a Destination factory
from_reference
@classmethod
def from_reference(cls,
ref: TDestinationReferenceArg,
credentials: Optional[Any] = None,
destination_name: Optional[str] = None,
environment: Optional[str] = None,
**kwargs: Any) -> Optional[AnyDestination]
Instantiate destination from a string reference or one of supported forms.
This methods obtains a destination factory and then instantiates it passing
the arguments after ref
DestinationReference Objects
class DestinationReference()
A registry of destination factories with a set of method for finding and instantiating
DESTINATIONS
A registry of all the destination factories
normalize_type
@staticmethod
def normalize_type(destination_type: str) -> str
Normalizes destination type string into a canonical form. Assumes that type names without dots correspond to built in destinations.
register
@classmethod
def register(cls, factory: Type[AnyDestination_CO], ref: str) -> None
Registers factory class under ref. ref
to_fully_qualified_refs
@staticmethod
def to_fully_qualified_refs(ref: str) -> List[str]
Converts ref into fully qualified form, return one or more alternatives for shorthand notations. Run context is injected if needed. Following formats are recognized
- name NOTE: the last component of destination type serves as destination name if not explicitly specified
find
@classmethod
def find(
cls,
ref: str,
raise_exec_errors: bool = False,
import_missing_modules: bool = False
) -> Union[Type[AnyDestination], Callable[..., AnyDestination]]
Finds or auto-imports destination factory that can be further called in order to instantiate it
The ref can be a destination name or import path pointing to a destination class (e.g. dlt.destinations.postgres)
You can control auto-import behavior:
raise_exec_errors- will re-raise code execution errors in imported modulesimport_missing_modules- will ignore missing dependencies during import by substituting them with dummy modules. this should be only used to manipulate local dev environment
NOTE: find returns a factory class or a callable that will create factory instance (for custom destinations)
use ensure_factory to extract factory class from callable
TODO: synthesize a call on custom destination (destination) factory type so this distinction is not
needed
ensure_factory
@classmethod
def ensure_factory(
cls, ref_factory: Union[Type[AnyDestination], Callable[...,
AnyDestination]]
) -> Type[AnyDestination]
Extract factory type from a callable creating factory instance.
from_reference
@classmethod
def from_reference(cls,
ref: str,
credentials: Optional[Any] = None,
destination_name: Optional[str] = None,
environment: Optional[str] = None,
**kwargs: Any) -> Optional[AnyDestination]
Instantiate destination from str reference.
The ref can be a destination name or import path pointing to a destination class (e.g. dlt.destinations.postgres)
This methods obtains a destination factory and then instantiates it passing the arguments after ref