You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Often json document contain inconsistently typed elements that may occur 1 or more times. In case of 1 they are inserted directly as type T, if > 1 they are inserted as list of type T.
ie. first_name may be "Juan" or ["Juan", "Pablo"]
in that case DLT builds bot flattened representation and table representation for T and inserts in both place.
Fix the json normalizer such that if table is present in the schema, the table is always used.
in relational.py
ifisinstance(v, (dict, list)):
ifnot_is_complex_type(schema, table, child_name, __r_lvl):
# TODO: if schema contains table {table}__{child_name} then convert v into single element listifisinstance(v, dict):
# flatten the dict morenorm_row_dicts(v, __r_lvl+1, parent_name=child_name)
else:
# pass the list to out_rec_listout_rec_list[child_name] =v
Note: in case of automatic schema inference, all values until first list of T is detected will be added flattened. To assure consistency such tables should be added to the schema before first data item is normalized
The text was updated successfully, but these errors were encountered:
Often json document contain inconsistently typed elements that may occur 1 or more times. In case of 1 they are inserted directly as type T, if > 1 they are inserted as list of type T.
ie.
first_name
may be "Juan" or ["Juan", "Pablo"]in that case DLT builds bot flattened representation and table representation for T and inserts in both place.
Fix the json normalizer such that if table is present in the schema, the table is always used.
in
relational.py
Note: in case of automatic schema inference, all values until first list of T is detected will be added flattened. To assure consistency such tables should be added to the schema before first data item is normalized
The text was updated successfully, but these errors were encountered: