Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable TableScan to return multiple arbitrary table references #14358

Open
phisn opened this issue Jan 29, 2025 · 0 comments
Open

Enable TableScan to return multiple arbitrary table references #14358

phisn opened this issue Jan 29, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@phisn
Copy link
Contributor

phisn commented Jan 29, 2025

Is your feature request related to a problem or challenge?

TableProvider has a type which indicates which type of table it contains. This can be a view. Therefore it makes sense to encode materialised views using TableProvider and use TableScan.

Currently TableScan does force the use of one or no TableReference. Since a view can be derived from multiple tables having the same column names it can happen that two columns can only be distinguished by their respective TableReference.

Describe the solution you'd like

TableScan should contain unique table_name: TableReference for each column.

Describe alternatives you've considered

Even through TableScan contains a DFSchema which contains a list of TableReference, it is not possible to use it since some optimisation rules recreate the TableScan with the wrong schema.

  1. We change the schema in TableProvider to DFSchema instead of Schema. I did not test if having multiple same columns in TableProvider does cause issues which would force this option. As in the execution only the column position is important, this seems unlikely.
  2. In my current implementation I am inlining table names using .flat_name() and outline them again using a analyser rule. Since I can't use TableScan I am using a custom Extension which is resolved by the analyser too.
  3. Leaving this to the user or creating some new alternative LogicalPlan node is also possible.

Additional context

This feature is slightly related to #14310.

@phisn phisn added the enhancement New feature or request label Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant