common.data_writers.buffered
new_file_id
def new_file_id() -> str
Creates new file id which is globally unique within table_name scope
BufferedDataWriter Objects
class BufferedDataWriter(Generic[TWriter])
write_empty_file
def write_empty_file(columns: TTableSchemaColumns) -> DataWriterMetrics
Writes empty file: only header and footer without actual items. Closed the empty file and returns metrics. Mind that header and footer will be written.
import_file
def import_file(file_path: str,
metrics: DataWriterMetrics,
with_extension: str = None) -> DataWriterMetrics
Import a file from file_path
into items storage under a new file name. Does not check
the imported file format. Uses counts from metrics
as a base. Logically closes the imported file
The preferred import method is a hard link to avoid copying the data. If current filesystem does not support it, a regular copy is used.
Alternative extension may be provided via with_extension
so various file formats may be imported into the same folder.
close
def close(skip_flush: bool = False) -> None
Flushes the data, writes footer (skip_flush is True), collects metrics and closes the underlying file.
alternative_spec
@contextlib.contextmanager
def alternative_spec(spec: FileWriterSpec) -> Iterator[FileWriterSpec]
Temporarily changes the writer spec ie. for the moment file is rotated