sources._core_source_templates.filesystem_pipeline
stream_and_merge_csv
def stream_and_merge_csv() -> None
Demonstrates how to scan folder with csv files, load them in chunk and merge on date column with the previous load
read_custom_file_type_excel
def read_custom_file_type_excel() -> None
Here we create an extract pipeline using filesystem resource and read_csv transformer
copy_files_resource
def copy_files_resource(local_folder: str) -> None
Demonstrates how to copy files locally by adding a step to filesystem resource and the to load the download listing to db