common.runtime.collector
Collector Objects
class Collector(ABC)
update
@abstractmethod
def update(name: str,
inc: int = 1,
total: int = None,
message: str = None,
label: str = None) -> None
Creates or updates a counter
This function updates a counter name
with a value inc
. If counter does not exist, it is created with optional total value of total
.
Depending on implementation label
may be used to create nested counters and message to display additional information associated with a counter.
Arguments:
name
str - An unique name of a counter, displayable.inc
int, optional - Increase amount. Defaults to 1.total
int, optional - Maximum value of a counter. Defaults to None which means unbound counter.message
str, optional - Additional message attached to a counter. Defaults to None.label
str, optional - Creates nested counter for countername
. Defaults to None.
__call__
def __call__(step: str) -> TCollector
Syntactic sugar for nicer context managers
NullCollector Objects
class NullCollector(Collector)
A default counter that does not count anything.
DictCollector Objects
class DictCollector(Collector)
A collector that just counts
LogCollector Objects
class LogCollector(Collector)
A Collector that shows progress by writing to a Python logger or a console
__init__
def __init__(log_period: float = 1.0,
logger: Union[logging.Logger, TextIO] = sys.stdout,
log_level: int = logging.INFO,
dump_system_stats: bool = True) -> None
Collector writing to a logger
every log_period
seconds. The logger can be a Python logger instance, text stream, or None that will attach dlt
logger
Arguments:
log_period
float, optional - Time period in seconds between log updates. Defaults to 1.0.logger
logging.Logger | TextIO, optional - Logger or text stream to write log messages to. Defaults to stdio.log_level
str, optional - Log level for the logger. Defaults to INFO leveldump_system_stats
bool, optional - Log memory and cpu usage. Defaults to True
TqdmCollector Objects
class TqdmCollector(Collector)
A Collector that shows progress with tqdm
progress bars
__init__
def __init__(single_bar: bool = False, **tqdm_kwargs: Any) -> None
A Collector that uses tqdm to display counters as progress bars. Set single_bar
to True to show just the main progress bar. Pass any config to tqdm in kwargs
AliveCollector Objects
class AliveCollector(Collector)
A Collector that shows progress with alive-progress
progress bars
__init__
def __init__(single_bar: bool = True, **alive_kwargs: Any) -> None
Collector that uses alive_progress to display counters as progress bars. Set single_bar
to True to show just the main progress bar. Pass any config to alive_progress in kwargs
EnlightenCollector Objects
class EnlightenCollector(Collector)
A Collector that shows progress with enlighten
progress and status bars that also allow for logging.
__init__
def __init__(single_bar: bool = False, **enlighten_kwargs: Any) -> None
Collector that uses Enlighten to display counters as progress bars. Set single_bar
to True to show just the main progress bar. Pass any config to Enlighten in kwargs