Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] adjust translate function syntax structure to prevent parser performance rollback (backport #54830) #55514

Merged
merged 2 commits into from
Jan 31, 2025

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jan 31, 2025

Why I'm doing:

parser performance rollback is introduced by this patch #54185

parser performance will rollback when there are many function call in query.

trace times SELECT SUM(ResolutionWidth), SUM(ResolutionWidth + 1), SUM(ResolutionWidth + 2), SUM(ResolutionWidth + 3), SUM(ResolutionWidth + 4), SUM(ResolutionWidth + 5), SUM(ResolutionWidth + 6), SUM(ResolutionWidth + 7), SUM(ResolutionWidth + 8), SUM(ResolutionWidth + 9), SUM(ResolutionWidth + 10), SUM(ResolutionWidth + 11), SUM(ResolutionWidth + 12), SUM(ResolutionWidth + 13), SUM(ResolutionWidth + 14), SUM(ResolutionWidth + 15), SUM(ResolutionWidth + 16), SUM(ResolutionWidth + 17), SUM(ResolutionWidth + 18), SUM(ResolutionWidth + 19), SUM(ResolutionWidth + 20), SUM(ResolutionWidth + 21), SUM(ResolutionWidth + 22), SUM(ResolutionWidth + 23), SUM(ResolutionWidth
+ 24), SUM(ResolutionWidth + 25), SUM(ResolutionWidth + 26), SUM(ResolutionWidth + 27), SUM(ResolutionWidth + 28), SUM(ResolutionWidth + 29), SUM(ResolutionWidth + 30), SUM(ResolutionWidth + 31), SUM(ResolutionWidth + 32), SUM(ResolutionWidth + 33), SUM(ResolutionWidth +
34), SUM(ResolutionWidth + 35), SUM(ResolutionWidth + 36), SUM(ResolutionWidth + 37), SUM(ResolutionWidth + 38), SUM(ResolutionWidth + 39), SUM(ResolutionWidth + 40), SUM(ResolutionWidth + 41), SUM(ResolutionWidth + 42), SUM(ResolutionWidth + 43), SUM(ResolutionWidth + 44), SUM(ResolutionWidth + 45), SUM(ResolutionWidth + 46), SUM(ResolutionWidth + 47), SUM(ResolutionWidth + 48), SUM(ResolutionWidth + 49), SUM(ResolutionWidth + 50), SUM(ResolutionWidth + 51), SUM(ResolutionWidth + 52), SUM(ResolutionWidth + 53), SUM(ResolutionWidth + 54), SUM(ResolutionWidth + 55), SUM(ResolutionWidth + 56), SUM(ResolutionWidth + 57), SUM(ResolutionWidth + 58), SUM(ResolutionWidth + 59),
SUM(ResolutionWidth + 60), SUM(ResolutionWidth + 61), SUM(ResolutionWidth + 62), SUM(ResolutionWidth + 63), SUM(ResolutionWidth + 64), SUM(ResolutionWidth + 65), SUM(ResolutionWidth + 66), SUM(ResolutionWidth + 67), SUM(ResolutionWidth + 68), SUM(ResolutionWidth + 69), SUM(ResolutionWidth + 70), SUM(ResolutionWidth + 71), SUM(ResolutionWidth + 72), SUM(ResolutionWidth + 73), SUM(ResolutionWidth + 74), SUM(ResolutionWidth + 75), SUM(ResolutionWidth + 76), SUM(ResolutionWidth + 77), SUM(ResolutionWidth + 78), SUM(ResolutionWidth + 79), SUM(ResolutionWidth + 80), SUM(ResolutionWidth + 81), SUM(ResolutionWidth + 82), SUM(ResolutionWidth + 83), SUM(ResolutionWidth + 84), SUM(ResolutionWidth + 85), SUM(ResolutionWidth + 86), SUM(ResolutionWidth + 87), SUM(ResolutionWidth + 88), SUM(ResolutionWidth + 89) FROM hits;
+----------------------------------------------------+
| Explain String                                     |
+----------------------------------------------------+
|   0ms|-- Parser[1] 101ms                           |
| 101ms|-- Total[1] 12ms                             |
| 101ms|    -- Analyzer[1] 2ms                       |
| 101ms|        -- Lock[1] 0                         |
| 101ms|        -- AnalyzeDatabase[1] 0              |
| 101ms|        -- AnalyzeTemporaryTable[1] 0        |
| 101ms|        -- AnalyzeTable[1] 0                 |
| 104ms|    -- Transformer[1] 2ms                    |
| 107ms|    -- Optimizer[1] 5ms                      |
| 107ms|        -- MVPreprocess[1] 0                 |
| 107ms|            -- MVChooseCandidates[1] 0       |
| 107ms|            -- MVGenerateMvPlan[1] 0         |
| 107ms|            -- MVValidateMv[1] 0             |
| 107ms|            -- MVProcessWithView[1] 0        |
| 107ms|        -- MVTextRewrite[1] 0                |
| 107ms|        -- RuleBaseOptimize[1] 3ms           |
| 110ms|        -- CostBaseOptimize[1] 1ms           |
| 111ms|        -- PhysicalRewrite[1] 0              |
| 112ms|        -- DynamicRewrite[1] 0               |
| 112ms|        -- PlanValidate[1] 0                 |
| 112ms|            -- InputDependenciesChecker[1] 0 |
| 112ms|            -- TypeChecker[1] 0              |
| 112ms|            -- CTEUniqueChecker[1] 0         |
| 112ms|            -- ColumnReuseChecker[1] 0       |
| 112ms|    -- ExecPlanBuild[1] 1ms                  |
| Tracer Cost: 25us                                  |
+----------------------------------------------------+

the parser of this case will take about 100ms.

bad case:
image

normal case:
image

What I'm doing:

Translate statement conflicts with translate function, so stripping translate function from simple function prevents performance rollback.

adjust translate function syntax structure and remove getCharPositionInLine for translate to prevent parser performance rollback.

base: 100ms
optimized: 4ms

Fixes #issue
https://github.com/StarRocks/StarRocksTest/issues/8932#issuecomment-2572967332

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

…arser performance rollback (#54830)

Signed-off-by: stephen <[email protected]>
(cherry picked from commit c1bc8d3)

# Conflicts:
#	fe/fe-core/src/main/java/com/starrocks/sql/parser/StarRocks.g4
@mergify mergify bot added the conflicts label Jan 31, 2025
Copy link
Contributor Author

mergify bot commented Jan 31, 2025

Cherry-pick of c1bc8d3 has failed:

On branch mergify/bp/branch-3.3/pr-54830
Your branch is up to date with 'origin/branch-3.3'.

You are currently cherry-picking commit c1bc8d31d1.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   fe/fe-core/src/main/java/com/starrocks/sql/parser/AstBuilder.java
	modified:   fe/fe-core/src/main/java/com/starrocks/sql/parser/StarRocksLex.g4
	modified:   fe/fe-core/src/test/java/com/starrocks/sql/parser/ParserTest.java
	modified:   test/sql/test_agg/R/test_distinct_agg
	modified:   test/sql/test_agg/T/test_distinct_agg

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   fe/fe-core/src/main/java/com/starrocks/sql/parser/StarRocks.g4

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

Copy link
Contributor Author

mergify bot commented Jan 31, 2025

@mergify[bot]: Backport conflict, please reslove the conflict and resubmit the pr

Signed-off-by: yanz <[email protected]>
@dirtysalt dirtysalt reopened this Jan 31, 2025
@wanpengfei-git wanpengfei-git enabled auto-merge (squash) January 31, 2025 18:51
@wanpengfei-git wanpengfei-git merged commit 7848101 into branch-3.3 Jan 31, 2025
42 of 43 checks passed
@wanpengfei-git wanpengfei-git deleted the mergify/bp/branch-3.3/pr-54830 branch January 31, 2025 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants