You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge?
TableProvider has a type which indicates which type of table it contains. This can be a view. Therefore it makes sense to encode materialised views using TableProvider and use TableScan.
Currently TableScan does force the use of one or no TableReference. Since a view can be derived from multiple tables having the same column names it can happen that two columns can only be distinguished by their respective TableReference.
Describe the solution you'd like
TableScan should contain unique table_name: TableReference for each column.
Describe alternatives you've considered
Even through TableScan contains a DFSchema which contains a list of TableReference, it is not possible to use it since some optimisation rules recreate the TableScan with the wrong schema.
We change the schema in TableProvider to DFSchema instead of Schema. I did not test if having multiple same columns in TableProvider does cause issues which would force this option. As in the execution only the column position is important, this seems unlikely.
In my current implementation I am inlining table names using .flat_name() and outline them again using a analyser rule. Since I can't use TableScan I am using a custom Extension which is resolved by the analyser too.
Leaving this to the user or creating some new alternative LogicalPlan node is also possible.
Is your feature request related to a problem or challenge?
TableProvider
has a type which indicates which type of table it contains. This can be a view. Therefore it makes sense to encode materialised views usingTableProvider
and useTableScan
.Currently
TableScan
does force the use of one or noTableReference
. Since a view can be derived from multiple tables having the same column names it can happen that two columns can only be distinguished by their respectiveTableReference
.Describe the solution you'd like
TableScan
should contain uniquetable_name: TableReference
for each column.Describe alternatives you've considered
Even through
TableScan
contains aDFSchema
which contains a list ofTableReference
, it is not possible to use it since some optimisation rules recreate theTableScan
with the wrong schema.TableProvider
toDFSchema
instead ofSchema
. I did not test if having multiple same columns inTableProvider
does cause issues which would force this option. As in the execution only the column position is important, this seems unlikely..flat_name()
and outline them again using a analyser rule. Since I can't useTableScan
I am using a custom Extension which is resolved by the analyser too.LogicalPlan
node is also possible.Additional context
This feature is slightly related to #14310.
The text was updated successfully, but these errors were encountered: