You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Goal
We want to mark certain string values to be normalized according to naming convention used later in the pipeline. Compare such feature to dlt.with_table where we mark whole dictionary to be routed to table with given type.
Motivation
Quite often some lineage metadata is loaded with the data itself. For example in case of pipedrive we load a table with definitions of user defined fields. Those fields are present in dlt schema and their names are normalized like any other column names.
Tasks
add normalize function in dlt.mark module. that will wrap a str in a string like object that will be serialized in typed_dump as a special type (like for example Decimal or datetime)
when deserialized in normalizer, a normalize_identifier method from the current schema context will be applied to the value
The text was updated successfully, but these errors were encountered:
Goal
We want to mark certain string values to be normalized according to naming convention used later in the pipeline. Compare such feature to
dlt.with_table
where we mark whole dictionary to be routed to table with given type.Motivation
Quite often some lineage metadata is loaded with the data itself. For example in case of
pipedrive
we load a table with definitions of user defined fields. Those fields are present indlt
schema and their names are normalized like any other column names.Tasks
normalize
function indlt.mark
module. that will wrap astr
in a string like object that will be serialized intyped_dump
as a special type (like for exampleDecimal
ordatetime
)normalize_identifier
method from the current schema context will be applied to the valueThe text was updated successfully, but these errors were encountered: