dlt.common.runtime.collector
NullCollector Objects
class NullCollector(Collector)
A default counter that does not count anything.
DictCollector Objects
class DictCollector(Collector)
A collector that just counts
LogCollector Objects
class LogCollector(Collector)
A Collector that shows progress by writing to a Python logger or a console
__init__
def __init__(log_period: float = 1.0,
logger: Union[logging.Logger, TextIO] = sys.stdout,
log_level: int = logging.INFO,
dump_system_stats: bool = True) -> None
Collector writing to a logger every log_period seconds. The logger can be a Python logger instance, text stream, or None that will attach dlt logger
Arguments:
log_periodfloat, optional - Time period in seconds between log updates. Defaults to 1.0.loggerlogging.Logger | TextIO, optional - Logger or text stream to write log messages to. Defaults to stdio.log_levelstr, optional - Log level for the logger. Defaults to INFO leveldump_system_statsbool, optional - Log memory and cpu usage. Defaults to True
maybe_log
def maybe_log() -> None
Check if should report and if so, call self.on_log
dump_counters
def dump_counters() -> None
Dump all counters to log using the shared formatting methods.
TqdmCollector Objects
class TqdmCollector(Collector)
A Collector that shows progress with tqdm progress bars
__init__
def __init__(single_bar: bool = False, **tqdm_kwargs: Any) -> None
A Collector that uses tqdm to display counters as progress bars. Set single_bar to True to show just the main progress bar. Pass any config to tqdm in kwargs
AliveCollector Objects
class AliveCollector(Collector)
A Collector that shows progress with alive-progress progress bars
__init__
def __init__(single_bar: bool = True, **alive_kwargs: Any) -> None
Collector that uses alive_progress to display counters as progress bars. Set single_bar to True to show just the main progress bar. Pass any config to alive_progress in kwargs
EnlightenCollector Objects
class EnlightenCollector(Collector)
A Collector that shows progress with enlighten progress and status bars that also allow for logging.
__init__
def __init__(single_bar: bool = False, **enlighten_kwargs: Any) -> None
Collector that uses Enlighten to display counters as progress bars. Set single_bar to True to show just the main progress bar. Pass any config to Enlighten in kwargs