common.normalizers.json.helpers

Cached helper methods for all operations that are called often

get_table_nesting_level

@lru_cache(maxsize=None)
def get_table_nesting_level(schema: Schema,
                            table_name: str,
                            default_nesting: int = 1000) -> Optional[int]

[view_source]

gets table nesting level, will inherit from parent if not set

is_nested_type

@lru_cache(maxsize=None)
def is_nested_type(schema: Schema, table_name: str, field_name: str,
                   _r_lvl: int) -> bool

[view_source]

For those paths the nested objects should be left in place. Cache perf: max_nesting < _r_lvl: ~2x faster, full check 10x faster

get_nested_row_id_type

@lru_cache(maxsize=None)
def get_nested_row_id_type(schema: Schema,
                           table_name: str) -> Tuple[TRowIdType, bool]

[view_source]

Gets type of row id to be added to nested table and if linking information should be added

get_row_hash

def get_row_hash(row: Dict[str, Any],
                 subset: Optional[List[str]] = None) -> str

[view_source]

Returns hash of row.

Hash includes column names and values and is ordered by column name. Excludes dlt system columns. Can be used as deterministic row identifier.