[Enhancement] Eliminate non-required unnest computation #55431

satanson · 2025-01-24T10:03:38Z

Why I'm doing:

If table function result is not required to output, then it is not need to compute, instead, we just obtain the number of input param's element then duplicate the required output columns.

With this optimzation, some query can be accelerated significantly.

What I'm doing:

Fixes #issue

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

fe/fe-core/src/main/java/com/starrocks/planner/TableFunctionNode.java

be/src/exec/pipeline/table_function_operator.cpp

be/src/exprs/table_function/unnest.h

satanson · 2025-02-06T11:45:01Z

@Mergifyio rebase

mergify · 2025-02-06T11:45:26Z

rebase

✅ Branch has been successfully rebased

github-actions · 2025-02-06T13:34:32Z

[FE Incremental Coverage Report]

✅ pass : 11 / 11 (100.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/planner/TableFunctionNode.java	3	3	100.00%	[]
🔵	com/starrocks/sql/plan/PlanFragmentBuilder.java	8	8	100.00%	[]

Signed-off-by: satanson <[email protected]>

sonarqubecloud · 2025-02-07T03:02:29Z

Quality Gate failed

Failed conditions
B Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

github-actions · 2025-02-07T04:08:32Z

[BE Incremental Coverage Report]

❌ fail : 4 / 19 (21.05%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	be/src/exprs/table_function/unnest.h	0	7	00.00%	[51, 52, 61, 62, 68, 69, 70]
🔵	be/src/exec/pipeline/table_function_operator.cpp	3	10	30.00%	[99, 168, 169, 170, 233, 234, 235]
🔵	be/src/exprs/table_function/table_function.h	1	2	50.00%	[64]

github-actions · 2025-02-07T06:29:36Z

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2025-02-07T11:55:47Z

@Mergifyio backport branch-3.4

github-actions · 2025-02-07T11:55:48Z

@Mergifyio backport branch-3.3

github-actions · 2025-02-07T11:55:49Z

@Mergifyio backport branch-3.2

github-actions · 2025-02-07T11:55:51Z

@Mergifyio backport branch-3.0

mergify · 2025-02-07T11:55:53Z

backport branch-3.4

✅ Backports have been created

#55675 [Enhancement] Eliminate non-required unnest computation (backport #55431) has been created for branch branch-3.4 but encountered conflicts

mergify · 2025-02-07T11:55:54Z

backport branch-3.3

✅ Backports have been created

#55676 [Enhancement] Eliminate non-required unnest computation (backport #55431) has been created for branch branch-3.3 but encountered conflicts

mergify · 2025-02-07T11:55:55Z

backport branch-3.2

✅ Backports have been created

#55677 [Enhancement] Eliminate non-required unnest computation (backport #55431) has been created for branch branch-3.2 but encountered conflicts

mergify · 2025-02-07T11:55:58Z

backport branch-3.1

✅ Backports have been created

#55678 [Enhancement] Eliminate non-required unnest computation (backport #55431) has been created for branch branch-3.1 but encountered conflicts

mergify · 2025-02-07T11:56:00Z

backport branch-3.0

✅ Backports have been created

#55679 [Enhancement] Eliminate non-required unnest computation (backport #55431) has been created for branch branch-3.0 but encountered conflicts

Signed-off-by: satanson <[email protected]> (cherry picked from commit 4be4b5c) # Conflicts: # fe/fe-core/src/test/java/com/starrocks/sql/plan/TableFunctionTest.java

Signed-off-by: satanson <[email protected]> (cherry picked from commit 4be4b5c) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/planner/TableFunctionNode.java # fe/fe-core/src/test/java/com/starrocks/sql/plan/TableFunctionTest.java

Signed-off-by: satanson <[email protected]> (cherry picked from commit 4be4b5c) # Conflicts: # be/src/exprs/table_function/table_function.h # be/src/exprs/table_function/unnest.h # fe/fe-core/src/main/java/com/starrocks/planner/TableFunctionNode.java # fe/fe-core/src/test/java/com/starrocks/sql/plan/TableFunctionTest.java

Signed-off-by: satanson <[email protected]> (cherry picked from commit 4be4b5c) # Conflicts: # be/src/exec/pipeline/table_function_operator.cpp # be/src/exec/pipeline/table_function_operator.h # be/src/exprs/table_function/table_function.h # be/src/exprs/table_function/unnest.h # fe/fe-core/src/main/java/com/starrocks/planner/TableFunctionNode.java # fe/fe-core/src/main/java/com/starrocks/sql/plan/PlanFragmentBuilder.java # fe/fe-core/src/test/java/com/starrocks/sql/plan/TableFunctionTest.java

) Signed-off-by: satanson <[email protected]>

…rRocks#55431) Signed-off-by: satanson <[email protected]>

) (#55680) Signed-off-by: satanson <[email protected]>

) (#55681) Signed-off-by: satanson <[email protected]>

satanson requested a review from a team as a code owner January 24, 2025 10:03

wanpengfei-git added the PROTO-REVIEW label Jan 24, 2025

github-actions bot added 3.4 3.3 labels Jan 24, 2025

wanpengfei-git requested a review from a team January 24, 2025 10:04

github-actions bot added 3.2 3.1 3.0 labels Jan 24, 2025

mergify bot assigned satanson Jan 24, 2025

starrocks-cr bot reviewed Jan 24, 2025

View reviewed changes

fe/fe-core/src/main/java/com/starrocks/planner/TableFunctionNode.java Show resolved Hide resolved

starrocks-cr bot reviewed Jan 24, 2025

View reviewed changes

be/src/exec/pipeline/table_function_operator.cpp Show resolved Hide resolved

starrocks-cr bot reviewed Jan 24, 2025

View reviewed changes

be/src/exprs/table_function/unnest.h Show resolved Hide resolved

satanson force-pushed the eliminate_non_required_unest branch from 31a802d to 6c31b71 Compare February 6, 2025 11:45

[Enhancement] Eliminate non-required unnest computation

00ebcd7

Signed-off-by: satanson <[email protected]>

satanson force-pushed the eliminate_non_required_unest branch from 6c31b71 to 00ebcd7 Compare February 7, 2025 02:54

stdpain approved these changes Feb 7, 2025

View reviewed changes

ZiheLiu approved these changes Feb 7, 2025

View reviewed changes

wangruin approved these changes Feb 7, 2025

View reviewed changes

kangkaisen merged commit 4be4b5c into main Feb 7, 2025
46 of 52 checks passed

kangkaisen deleted the eliminate_non_required_unest branch February 7, 2025 11:55

github-actions bot removed the 3.4 label Feb 7, 2025

github-actions bot removed the 3.2 label Feb 7, 2025

github-actions bot removed 3.1 3.0 labels Feb 7, 2025

mergify bot mentioned this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431) #55675

Closed

18 tasks

This was referenced Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431) #55676

Closed

[Enhancement] Eliminate non-required unnest computation (backport #55431) #55677

Closed

mergify bot mentioned this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431) #55678

Closed

18 tasks

mergify bot mentioned this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431) #55679

Closed

42 tasks

satanson added a commit that referenced this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431

443264a

) Signed-off-by: satanson <[email protected]>

satanson mentioned this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431) #55680

Merged

24 tasks

satanson added a commit to satanson/starrocks that referenced this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport Sta…

c604f89

…rRocks#55431) Signed-off-by: satanson <[email protected]>

satanson mentioned this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431) #55681

Merged

24 tasks

github-actions bot added 3.4 3.3 3.0 labels Feb 7, 2025

kangkaisen pushed a commit that referenced this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431

b2e8f90

) (#55680) Signed-off-by: satanson <[email protected]>

github-actions bot added the 3.4-merged label Feb 7, 2025

kangkaisen pushed a commit that referenced this pull request Feb 7, 2025

[Enhancement] Eliminate non-required unnest computation (backport #55431

af66bbe

) (#55681) Signed-off-by: satanson <[email protected]>

github-actions bot added the 3.3-merged label Feb 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Enhancement] Eliminate non-required unnest computation #55431

[Enhancement] Eliminate non-required unnest computation #55431

satanson commented Jan 24, 2025 •

edited

Loading

satanson commented Feb 6, 2025

mergify bot commented Feb 6, 2025

github-actions bot commented Feb 6, 2025

sonarqubecloud bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

[Enhancement] Eliminate non-required unnest computation #55431

[Enhancement] Eliminate non-required unnest computation #55431

Conversation

satanson commented Jan 24, 2025 • edited Loading

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

satanson commented Feb 6, 2025

mergify bot commented Feb 6, 2025

✅ Branch has been successfully rebased

github-actions bot commented Feb 6, 2025

[FE Incremental Coverage Report]

file detail

sonarqubecloud bot commented Feb 7, 2025

Quality Gate failed

github-actions bot commented Feb 7, 2025

[BE Incremental Coverage Report]

file detail

github-actions bot commented Feb 7, 2025

[Java-Extensions Incremental Coverage Report]

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

github-actions bot commented Feb 7, 2025

mergify bot commented Feb 7, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Feb 7, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Feb 7, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Feb 7, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Feb 7, 2025 • edited Loading

✅ Backports have been created

satanson commented Jan 24, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading

mergify bot commented Feb 7, 2025 •

edited

Loading