common.normalizers.json.relational

DataItemNormalizer Objects

class DataItemNormalizer(DataItemNormalizerBase[RelationalNormalizerConfig])

[view_source]

C_DLT_ID

unique id of current row

C_DLT_LOAD_ID

load id to identify records loaded together that ie. need to be processed

C_DLT_ROOT_ID

unique id of top level parent

C_DLT_PARENT_ID

unique id of parent row

C_DLT_LIST_IDX

position in the list of rows

C_VALUE

for lists of simple types

EMPTY_KEY_IDENTIFIER

replace empty keys with this

__init__

def __init__(schema: Schema) -> None

[view_source]

This item normalizer works with nested dictionaries. It flattens dictionaries and descends into lists. It yields row dictionaries at each nesting level.

get_row_hash

@staticmethod
def get_row_hash(row: Dict[str, Any],
                 subset: Optional[List[str]] = None) -> str

[view_source]

Returns hash of row.

Hash includes column names and values and is ordered by column name. Excludes dlt system columns. Can be used as deterministic row identifier.

extend_schema

def extend_schema() -> None

[view_source]

Extends Schema with normalizer-specific hints and settings.

This method is called by Schema when instance is created or restored from storage.

extend_table

def extend_table(table_name: str) -> None

[view_source]

If the table has a merge write disposition, add propagation info to normalizer

Called by Schema when new table is added to schema or table is updated with partial table. Table name should be normalized.