[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection #54595

ZiheLiu · 2025-01-02T09:50:22Z

Why I'm doing:

Previous Logic:

evaluate_group_by_columns.
evaluate_input_columns
calculate _streaming_selection to indicate which rows hit HT.
output_chunk_by_streaming_with_selection (filter group_by_columns and input_columns by _streaming_selection)
output_chunk_by_streaming

Recent Logic:

evaluate_group_by_columns
calculate _streaming_selection to indicate which rows hit HT.
output_chunk_by_streaming_with_selection (filter group_by_columns by _streaming_selection)
output_chunk_by_streaming (evaluate input_columns here now)

This cause two issues.

In output_chunk_by_streaming, before calling evaluate_input_columns, the number of rows in group_by_columns and input_chunk differs. This is because group_by_columns is filtered by _streaming_selection, whereas input_chunk is not.
The result of evaluate_input_columns is not filtered by _streaming_selection.

*** Aborted at 1735817590 (unix time) try "date -d @1735817590" if you are using GNU date ***
PC: @     0x7ff6544c3387 __GI_raise
*** SIGABRT (@0x3e80000448c) received by PID 17548 (TID 0x7ff56c586700) from PID 17548; stack trace: ***
    @         0x1c6fcf47 google::(anonymous namespace)::HandleSignal(int, siginfo_t*, void*)
    @     0x7ff65695920b __pthread_once_slow
    @         0x1c6fc754 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
F20250102 19:33:10.237299 140692041484032 aggregator.cpp:951] Check failed: num_rows == _group_by_columns[i]->size() (3014 vs. 2357)
    @     0x7ff656962630 (/usr/lib64/libpthread-2.17.so+0xf62f)
    @     0x7ff6544c3387 __GI_raise
    @     0x7ff6544c4a78 __GI_abort
    @          0xd76fc0a starrocks::failure_function()
    @         0x1c6f154a google::LogMessage::Fail()
    @         0x1c6f2f84 google::LogMessageFatal::~LogMessageFatal()
    @          0xf5700ac starrocks::Aggregator::output_chunk_by_streaming(starrocks::Chunk*, std::shared_ptr<starrocks::Chunk>*)
    @          0xf5730bc starrocks::Aggregator::output_chunk_by_streaming_with_selection(starrocks::Chunk*, std::shared_ptr<starrocks::Chunk>*)
    @          0xf1612ae starrocks::pipeline::AggregateStreamingSinkOperator::_push_chunk_by_selective_preaggregation(std::shared_ptr<starrocks::Chunk> const&, unsigned long, bool)
    @          0xf163d30 starrocks::pipeline::AggregateStreamingSinkOperator::_push_chunk_by_auto(std::shared_ptr<starrocks::Chunk> const&, unsigned long)
    @          0xf15eba3 starrocks::pipeline::AggregateStreamingSinkOperator::push_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&)
    @          0xf0c306c starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int)
    @          0xfed09a3 starrocks::pipeline::GlobalDriverExecutor::_worker_thread()
    @          0xfecf1c6 starrocks::pipeline::GlobalDriverExecutor::initialize(int)::{lambda()#1}::operator()() const
    @          0xfed99ec void std::__invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::{lambda()#1}&>(std::__invoke_other, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::{lambda()#1}&)
    @          0xfed8dca std::enable_if<is_invocable_r_v<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::{lambda()#1}&>, void>::type std::__invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::{lambda()#1}&>(starrocks::pipeline::GlobalDriver@
    @          0xfed877f std::_Function_handler<void (), starrocks::pipeline::GlobalDriverExecutor::initialize(int)::{lambda()#1}>::_M_invoke(std::_Any_data const&)
    @          0xbdcd888 std::function<void ()>::operator()() const
    @          0xc595650 starrocks::FunctionRunnable::run()
    @          0xc592494 starrocks::ThreadPool::dispatch_thread()
    @          0xc5b1ae0 void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&)
    @          0xc5b03f3 std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&)
    @          0xc5aee80 void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>)
    @          0xc5ad442 void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>()
    @          0xc5ab4f4 void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&)
    @          0xc5a8f02 std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool:@
    @          0xc5a477f std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&)
    @          0xbdcd888 std::function<void ()>::operator()() const
    @          0xc5785c9 starrocks::Thread::supervise_thread(void*)

What I'm doing:

Fixes https://github.com/StarRocks/StarRocksTest/issues/9009

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

…with_selectoin Signed-off-by: zihe.liu <[email protected]>

be/src/exec/aggregator.cpp

Signed-off-by: zihe.liu <[email protected]>

github-actions · 2025-01-02T14:16:16Z

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2025-01-02T14:16:19Z

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2025-01-02T14:21:57Z

@Mergifyio backport branch-3.4

github-actions · 2025-01-02T14:21:58Z

@Mergifyio backport branch-3.3

github-actions · 2025-01-02T14:21:59Z

@Mergifyio backport branch-3.2

github-actions · 2025-01-02T14:22:00Z

@Mergifyio backport branch-3.1

github-actions · 2025-01-02T14:22:01Z

@Mergifyio backport branch-3.0

andyziye · 2025-01-02T14:22:15Z

@mergify backport branch-3.4.0-rc01

mergify · 2025-01-02T14:22:25Z

backport branch-3.4

✅ Backports have been created

#54614 [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) has been created for branch branch-3.4

mergify · 2025-01-02T14:22:27Z

backport branch-3.3

✅ Backports have been created

#54615 [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) has been created for branch branch-3.3 but encountered conflicts

mergify · 2025-01-02T14:22:28Z

backport branch-3.2

✅ Backports have been created

#54616 [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) has been created for branch branch-3.2 but encountered conflicts

mergify · 2025-01-02T14:22:30Z

backport branch-3.1

✅ Backports have been created

#54617 [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) has been created for branch branch-3.1 but encountered conflicts

mergify · 2025-01-02T14:22:32Z

backport branch-3.0

✅ Backports have been created

#54618 [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) has been created for branch branch-3.0 but encountered conflicts

mergify · 2025-01-02T14:22:33Z

backport branch-3.4.0-rc01

✅ Backports have been created

#54619 [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) has been created for branch branch-3.4.0-rc01

…with_selection (#54595) Signed-off-by: zihe.liu <[email protected]> (cherry picked from commit efe1dde)

…with_selection (#54595) Signed-off-by: zihe.liu <[email protected]> (cherry picked from commit efe1dde) # Conflicts: # be/src/exec/aggregator.h

…with_selection (#54595) Signed-off-by: zihe.liu <[email protected]> (cherry picked from commit efe1dde)

github-actions · 2025-01-02T14:26:44Z

[BE Incremental Coverage Report]

✅ pass : 12 / 12 (100.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	be/src/exec/aggregator.cpp	12	12	100.00%	[]

…with_selection (backport #54595) (#54619) Co-authored-by: zihe.liu <[email protected]>

…with_selection (StarRocks#54595) Signed-off-by: zihe.liu <[email protected]>

ZiheLiu · 2025-01-07T03:18:33Z

@mergify backport branch-3.3

mergify · 2025-01-07T03:18:38Z

backport branch-3.3

✅ Backports have been created

#54615 [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) has been created for branch branch-3.3

…with_selection (backport #54595) (#54614) Co-authored-by: zihe.liu <[email protected]>

…with_selection (backport #54595) (#54615) Signed-off-by: zihe.liu <[email protected]> Co-authored-by: zihe.liu <[email protected]>

ZiheLiu · 2025-01-26T02:07:40Z

ignore backport check 3.2.15

ZiheLiu · 2025-01-26T02:08:26Z

ignore backport check: 3.2.15

…with_selection (backport StarRocks#54595) (StarRocks#54615) Signed-off-by: zihe.liu <[email protected]> Co-authored-by: zihe.liu <[email protected]>

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_…

9fd1c4c

…with_selectoin Signed-off-by: zihe.liu <[email protected]>

ZiheLiu requested a review from a team as a code owner January 2, 2025 09:50

github-actions bot added the 3.4 label Jan 2, 2025

mergify bot assigned ZiheLiu Jan 2, 2025

github-actions bot added 3.3 3.2 3.1 3.0 labels Jan 2, 2025

starrocks-cr bot reviewed Jan 2, 2025

View reviewed changes

be/src/exec/aggregator.cpp Show resolved Hide resolved

ZiheLiu added 2 commits January 2, 2025 20:24

update comment

4107cf7

Signed-off-by: zihe.liu <[email protected]>

fix

62627d0

Signed-off-by: zihe.liu <[email protected]>

ZiheLiu changed the title ~~[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selectoin~~ [BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection Jan 2, 2025

kangkaisen approved these changes Jan 2, 2025

View reviewed changes

stdpain approved these changes Jan 2, 2025

View reviewed changes

andyziye merged commit efe1dde into StarRocks:main Jan 2, 2025
79 checks passed

github-actions bot removed the 3.4 label Jan 2, 2025

github-actions bot removed the 3.3 label Jan 2, 2025

github-actions bot removed the 3.2 label Jan 2, 2025

github-actions bot removed 3.1 3.0 labels Jan 2, 2025

mergify bot pushed a commit that referenced this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_…

a4b7705

…with_selection (#54595) Signed-off-by: zihe.liu <[email protected]> (cherry picked from commit efe1dde)

mergify bot mentioned this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) #54614

Merged

18 tasks

mergify bot mentioned this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) #54615

Merged

18 tasks

mergify bot mentioned this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) #54616

Closed

18 tasks

mergify bot mentioned this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) #54617

Closed

18 tasks

mergify bot mentioned this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) #54618

Closed

42 tasks

mergify bot pushed a commit that referenced this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_…

b5541dd

…with_selection (#54595) Signed-off-by: zihe.liu <[email protected]> (cherry picked from commit efe1dde)

mergify bot mentioned this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection (backport #54595) #54619

Merged

18 tasks

ZiheLiu added a commit that referenced this pull request Jan 2, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_…

9f4bbab

…with_selection (backport #54595) (#54619) Co-authored-by: zihe.liu <[email protected]>

maggie-zhu pushed a commit to maggie-zhu/starrocks that referenced this pull request Jan 6, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_…

26979b0

…with_selection (StarRocks#54595) Signed-off-by: zihe.liu <[email protected]>

wanpengfei-git pushed a commit that referenced this pull request Jan 7, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_…

f6783c3

…with_selection (backport #54595) (#54614) Co-authored-by: zihe.liu <[email protected]>

github-actions bot added the 3.4-merged label Jan 7, 2025

wanpengfei-git pushed a commit that referenced this pull request Jan 7, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_…

33f111b

…with_selection (backport #54595) (#54615) Signed-off-by: zihe.liu <[email protected]> Co-authored-by: zihe.liu <[email protected]>

github-actions bot added the 3.3-merged label Jan 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection #54595

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection #54595

ZiheLiu commented Jan 2, 2025 •

edited

Loading

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

andyziye commented Jan 2, 2025

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

github-actions bot commented Jan 2, 2025

ZiheLiu commented Jan 7, 2025

mergify bot commented Jan 7, 2025

ZiheLiu commented Jan 26, 2025

ZiheLiu commented Jan 26, 2025

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection #54595

[BugFix] Fix sequence of selection and input_chunk for streaming_agg_with_selection #54595

Conversation

ZiheLiu commented Jan 2, 2025 • edited Loading

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

github-actions bot commented Jan 2, 2025

[Java-Extensions Incremental Coverage Report]

github-actions bot commented Jan 2, 2025

[FE Incremental Coverage Report]

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

github-actions bot commented Jan 2, 2025

andyziye commented Jan 2, 2025

mergify bot commented Jan 2, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Jan 2, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Jan 2, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Jan 2, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Jan 2, 2025 • edited Loading

✅ Backports have been created

mergify bot commented Jan 2, 2025 • edited Loading

✅ Backports have been created

github-actions bot commented Jan 2, 2025

[BE Incremental Coverage Report]

file detail

ZiheLiu commented Jan 7, 2025

mergify bot commented Jan 7, 2025

✅ Backports have been created

ZiheLiu commented Jan 26, 2025

ZiheLiu commented Jan 26, 2025

ZiheLiu commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading

mergify bot commented Jan 2, 2025 •

edited

Loading