You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Background
extract and normalize stage use files to pass intermediary data (extracted and normalized files). the metadata on that files is stored in file name. it contains schema name, table name and other information
the os file name limit is ~255 characters (but may be much less on some NFSses etc.)
that impact our ability to create long table names on bigquery/duckdb even with #157
Tasks
We need to remove metadata information from file name or find a way to shorten the file name ie. by using hashes. File names should be human readable though.
see test: test_run_with_table_name_exceeding_path_length which must start passing
The text was updated successfully, but these errors were encountered:
Background
extract and normalize stage use files to pass intermediary data (extracted and normalized files). the metadata on that files is stored in file name. it contains schema name, table name and other information
the os file name limit is ~255 characters (but may be much less on some NFSses etc.)
that impact our ability to create long table names on bigquery/duckdb even with #157
Tasks
We need to remove metadata information from file name or find a way to shorten the file name ie. by using hashes. File names should be human readable though.
see test:
test_run_with_table_name_exceeding_path_length
which must start passingThe text was updated successfully, but these errors were encountered: