jsonl - JSON delimited
JSON delimited is a file format that stores several
JSON documents in one file. The
documents are separated by a new line.
Additional data types are stored as follows:
dateas ISO strings;
decimalas text representation of decimal number;
binaryis base64 encoded string;
HexBytesis hex encoded string;
complexis serialized as a string.
This file format is compressed by default.
Used by default by: BigQuery, Snowflake, filesystem.
By setting the
loader_file_format argument to
jsonl in the run command, the pipeline will store
your data in the jsonl format to the destination:
info = pipeline.run(some_source(), loader_file_format="jsonl")