extract.reference

SourceFactory Objects

class SourceFactory(ABC, Generic[TSourceFunParams, TDltSourceImpl])

[view_source]

__call__

def __call__(*args: TSourceFunParams.args,
             **kwargs: TSourceFunParams.kwargs) -> TDltSourceImpl

[view_source]

Makes dlt source

clone

@abstractmethod
def clone(*,
          name: str = None,
          section: str = None,
          max_table_nesting: int = None,
          root_key: bool = False,
          schema: Schema = None,
          schema_contract: TSchemaContract = None,
          spec: Type[BaseConfiguration] = None,
          parallelized: bool = None,
          _impl_cls: Type[TDltSourceImpl] = DltSource) -> Self

[view_source]

Overrides default decorator arguments that will be used to when DltSource instance and returns modified clone.

SourceReference Objects

class SourceReference()

[view_source]

SOURCES

A registry of all the decorated sources and resources discovered when importing modules

ref

A fully qualified reference: module.name to the factory instance

expand_shorthand_ref

@classmethod
def expand_shorthand_ref(cls, ref: str) -> List[str]

[view_source]

Converts ref into fully qualified form, return one or more alternatives for shorthand notations. Run context is injected if needed. Following formats are recognized

• section.name
• name

find

@classmethod
def find(cls,
         ref: str,
         _impl_sig: Callable[TSourceFunParams, Any] = None,
         _impl_cls: Type[TDltSourceImpl] = None) -> Any

[view_source]

Returns source factory from reference ref. Looks into registry or tries auto-import

Expands shorthand notation into section.name eg. "sql_database" is expanded into "dlt.sources.sql_database.sql_database".

from_reference

@classmethod
def from_reference(cls,
                   ref: str,
                   name: str = None,
                   section: str = None,
                   max_table_nesting: int = None,
                   root_key: bool = False,
                   schema: Schema = None,
                   schema_contract: TSchemaContract = None,
                   spec: Type[BaseConfiguration] = None,
                   parallelized: bool = None,
                   _impl_cls: Type[TDltSourceImpl] = None,
                   source_args: Tuple[Any, ...] = None,
                   source_kwargs: Dict[str, Any] = None) -> TDltSourceImpl

[view_source]

Find registered source factory or imports it, then instantiates the DltSource using passed args and kwargs. Passes additional arguments to clone of source factory