common.normalizers.json.helpers

Cached helper methods for all operations that are called often

get_table_nesting_level

@lru_cache(maxsize=None)
def get_table_nesting_level(schema: Schema,
                            table_name: str,
                            default_nesting: int = 1000) -> Optional[int]

[view_source]

gets table nesting level, will inherit from parent if not set

is_nested_type

@lru_cache(maxsize=None)
def is_nested_type(schema: Schema, table_name: str, field_name: str,
                   _r_lvl: int) -> bool

[view_source]

For those paths the nested objects should be left in place. Cache perf: max_nesting < _r_lvl: ~2x faster, full check 10x faster

should_be_nested

@lru_cache(maxsize=None)
def should_be_nested(schema: Schema, table_name: str) -> bool

[view_source]

Tells if table should be nested or should be a root table. All tables created by the normalizer are nested, defined tables are checked.

get_row_hash

def get_row_hash(row: Dict[str, Any],
                 subset: Optional[List[str]] = None) -> str

[view_source]

Returns hash of row.

Hash includes column names and values and is ordered by column name. Excludes dlt system columns. Can be used as deterministic row identifier.