Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: don't rewrite table names inside subquery when engine doesn't support it #29

Merged
merged 10 commits into from
Dec 30, 2024

Conversation

Sevenannn
Copy link

@Sevenannn Sevenannn commented Dec 25, 2024

🗣 Description

For certain query engine: dremio, it doesn't support fully qualified table path in the subquery, so for example, this subquery select * from db.schema.table where db.schema.table.col = 0 won't be supported. The subquery will be supported when subquery columns that are from tablescans within subquery only contains partial path / table name, for example select * from db.schema.table where table.col = 0. This PR implements changes to only use partial path within subquery for such engines.

  • Implement a function rewrite_subquery_use_partial_path to determine whether the subquery requires partial path through the executor name, currently only returns true for Dremio.
  • Use a bool subquery_uses_partial_path to track whether a executor engine needs a partial path for query, and a Option<HashSet<TableReference>> subquery_table_scans to track the table scans appearing in the subqueries in function rewrite_table_scans.
  • When a subquery column is from a table scan within subquery, i.e., its TableReference can be found in subquery_table_scans, and subquery_uses_partial_path is true, use partial path instead of fully qualified table path for rewriting the column.
  • Add test test_subquery_requires_partial_path to verify the changes.

🔨 Related Issues

🤔 Concerns

@Sevenannn Sevenannn changed the base branch from spiceai-43 to main December 26, 2024 19:11
@Sevenannn Sevenannn changed the base branch from main to spiceai-43 December 26, 2024 19:11
@Sevenannn Sevenannn changed the title Don't rewrite table names inside subquery when engine doesn't support it fix: Don't rewrite table names inside subquery when engine doesn't support it Dec 26, 2024
@Sevenannn Sevenannn changed the title fix: Don't rewrite table names inside subquery when engine doesn't support it fix: don't rewrite table names inside subquery when engine doesn't support it Dec 27, 2024
@Sevenannn Sevenannn marked this pull request as ready for review December 27, 2024 05:08
@Sevenannn Sevenannn requested a review from a team December 27, 2024 05:09
@Sevenannn Sevenannn merged commit 4c0ff79 into spiceai-43 Dec 30, 2024
17 checks passed
@Sevenannn Sevenannn deleted the qianqian/subquery-rewrite branch December 30, 2024 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants