Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] fix upgrade_if_overflow ASAN crash (backport #56261) #56526

Open
wants to merge 1 commit into
base: branch-3.3
Choose a base branch
from
Open

[BugFix] fix upgrade_if_overflow ASAN crash (backport #56261) #56526

wants to merge 1 commit into from
+10 −8

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Mar 4, 2025 •
edited by wanpengfei-git
Loading

Why I'm doing:

  1. from the following stack, we can see find_first_overflow_point may access offsets[end], which is a invalid address.
  2. in [BugFix] fix array_map crash #55383, we call upgrade_if_overflow in BinaryColumnBase::replicate, which is a wrong decision, since other expr can not handle LargeBinaryColumn. So now we choose report error when binary column exceeds 4GB
start time: Sun Feb 23 15:56:52 CST 2025, server uptime:  15:56:52 up 2 days, 23:39,  0 users,  load average: 0.00, 0.01, 0.06
Ignored unknown config: default_rowset_type
=================================================================
==51323==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7fcbc20ba628 at pc 0x00000d951f42 bp 0x7fca8c8284a0 sp 0x7fca8c828498
READ of size 4 at 0x7fcbc20ba628 thread T308 (pip_exec_com)
    #0 0xd951f41 in starrocks::find_first_overflow_point(std::vector<unsigned int, starrocks::ColumnAllocator<unsigned int> > const&, unsigned long, unsigned long) be/src/column/binary_column.cpp:705
    #1 0xd95283b in starrocks::BinaryColumnBase<unsigned int>::upgrade_if_overflow() be/src/column/binary_column.cpp:730
    #2 0xd95c160 in starrocks::BinaryColumnBase<unsigned int>::replicate(std::vector<unsigned int, starrocks::ColumnAllocator<unsigned int> > const&) be/src/column/binary_column.cpp:155
    #3 0xd92c79b in starrocks::NullableColumn::replicate(std::vector<unsigned int, starrocks::ColumnAllocator<unsigned int> > const&) be/src/column/nullable_column.cpp:140
    #4 0x14be6ffa in starrocks::StatusOr<std::shared_ptr<starrocks::Column> > starrocks::ArrayMapExpr::evaluate_lambda_expr<false, true>(starrocks::ExprContext*, starrocks::Chunk*, std::vector<std::shared_ptr<starrocks::Column>, std::allocator<std::shared_ptr<starrocks::Column> > > const&, std::shared_ptr<starrocks::FixedLengthColumn<unsigned char> > const&) be/src/exprs/array_map_expr.cpp:193
    #5 0x14bda5aa in starrocks::ArrayMapExpr::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*) be/src/exprs/array_map_expr.cpp:390
    #6 0x12ef2e16 in starrocks::ExprContext::evaluate(starrocks::Expr*, starrocks::Chunk*, unsigned char*) be/src/exprs/expr_context.cpp:188
    #7 0x149530cc in starrocks::VectorizedFunctionCallExpr::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*) be/src/exprs/function_call_expr.cpp:190
    #8 0x12ef2e16 in starrocks::ExprContext::evaluate(starrocks::Expr*, starrocks::Chunk*, unsigned char*) be/src/exprs/expr_context.cpp:188
    #9 0x12ef25b1 in starrocks::ExprContext::evaluate(starrocks::Chunk*, unsigned char*) be/src/exprs/expr_context.cpp:164
    #10 0xecf0525 in starrocks::pipeline::ProjectOperator::push_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) be/src/exec/pipeline/project_operator.cpp:60
    #11 0xf024f0a in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) be/src/exec/pipeline/pipeline_driver.cpp:358
    #12 0xfe5ec68 in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() be/src/exec/pipeline/pipeline_driver_executor.cpp:158
    #13 0xfe5d2cf in operator() be/src/exec/pipeline/pipeline_driver_executor.cpp:58
    #14 0xfe6807d in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/invoke.h:61
    #15 0xfe6745b in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/invoke.h:111
    #16 0xfe66e10 in _M_invoke /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/std_function.h:290
    #17 0xbc8c1a3 in std::function<void ()>::operator()() const /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/std_function.h:591
    #18 0xc4a27ef in starrocks::FunctionRunnable::run() be/src/util/threadpool.cpp:59
    #19 0xc49d645 in starrocks::ThreadPool::dispatch_thread() be/src/util/threadpool.cpp:635
    #20 0xc4bf625 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) (/home/disk1/sr/be/lib/starrocks_be+0xc4bf625)
    #21 0xc4bdf38 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/invoke.h:96
    #22 0xc4bc9c5 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/functional:513
    #23 0xc4baf95 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/functional:598
    #24 0xc4b907d in void std::__invoke_impl<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/invoke.h:61
    #25 0xc4b6ad9 in std::enable_if<is_invocable_r_v<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&>(std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>&) (/home/disk1/sr/be/lib/starrocks_be+0xc4b6ad9)
    #26 0xc4b236e in std::_Function_handler<void (), std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) (/home/disk1/sr/be/lib/starrocks_be+0xc4b236e)
    #27 0xbc8c1a3 in std::function<void ()>::operator()() const /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/std_function.h:591
    #28 0xc483c9e in starrocks::Thread::supervise_thread(void*) be/src/util/thread.cpp:366
    #29 0xb957f11 in asan_thread_start ../../.././libsanitizer/asan/asan_interceptors.cpp:234
    #30 0x7fcbc8ff4ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4) (BuildId: e10cc8f2b932fc3daeda22f8dac5ebb969524e5b)
    #31 0x7fcbc6c25b0c in clone (/lib64/libc.so.6+0xfeb0c) (BuildId: 1a8fb61bb4614a483833d5334202ab50edda2a25)

0x7fcbc20ba628 is located 0 bytes after 364072-byte region [0x7fcbc2061800,0x7fcbc20ba628)
allocated by thread T308 (pip_exec_com) here:
    #0 0xb9ef527 in malloc ../../.././libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0xbd7c2df in starrocks::MemHookAllocator::alloc(unsigned long) be/src/runtime/memory/mem_hook_allocator.h:25
    #2 0xbd9c0a3 in starrocks::AllocatorFactory<starrocks::Allocator, starrocks::MemHookAllocator>::checked_alloc(unsigned long) be/src/runtime/memory/allocator.h:64
    #3 0xc17362e in starrocks::ColumnAllocator<unsigned int>::allocate(unsigned long) be/src/runtime/memory/column_allocator.h:49
    #4 0xc175d37 in std::allocator_traits<starrocks::ColumnAllocator<unsigned int> >::allocate(starrocks::ColumnAllocator<unsigned int>&, unsigned long) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/alloc_traits.h:334
    #5 0xc1729df in std::_Vector_base<unsigned int, starrocks::ColumnAllocator<unsigned int> >::_M_allocate(unsigned long) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/stl_vector.h:380
    #6 0xc188cfa in std::vector<unsigned int, starrocks::ColumnAllocator<unsigned int> >::_M_default_append(unsigned long) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/vector.tcc:834
    #7 0xc185d32 in std::vector<unsigned int, starrocks::ColumnAllocator<unsigned int> >::resize(unsigned long) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/stl_vector.h:1016
    #8 0xd95a740 in starrocks::BinaryColumnBase<unsigned int>::replicate(std::vector<unsigned int, starrocks::ColumnAllocator<unsigned int> > const&) be/src/column/binary_column.cpp:143
    #9 0xd92c79b in starrocks::NullableColumn::replicate(std::vector<unsigned int, starrocks::ColumnAllocator<unsigned int> > const&) be/src/column/nullable_column.cpp:140
    #10 0x14be6ffa in starrocks::StatusOr<std::shared_ptr<starrocks::Column> > starrocks::ArrayMapExpr::evaluate_lambda_expr<false, true>(starrocks::ExprContext*, starrocks::Chunk*, std::vector<std::shared_ptr<starrocks::Column>, std::allocator<std::shared_ptr<starrocks::Column> > > const&, std::shared_ptr<starrocks::FixedLengthColumn<unsigned char> > const&) be/src/exprs/array_map_expr.cpp:193
    #11 0x14bda5aa in starrocks::ArrayMapExpr::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*) be/src/exprs/array_map_expr.cpp:390
    #12 0x12ef2e16 in starrocks::ExprContext::evaluate(starrocks::Expr*, starrocks::Chunk*, unsigned char*) be/src/exprs/expr_context.cpp:188
    #13 0x149530cc in starrocks::VectorizedFunctionCallExpr::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*) be/src/exprs/function_call_expr.cpp:190
    #14 0x12ef2e16 in starrocks::ExprContext::evaluate(starrocks::Expr*, starrocks::Chunk*, unsigned char*) be/src/exprs/expr_context.cpp:188
    #15 0x12ef25b1 in starrocks::ExprContext::evaluate(starrocks::Chunk*, unsigned char*) be/src/exprs/expr_context.cpp:164
    #16 0xecf0525 in starrocks::pipeline::ProjectOperator::push_chunk(starrocks::RuntimeState*, std::shared_ptr<starrocks::Chunk> const&) be/src/exec/pipeline/project_operator.cpp:60
    #17 0xf024f0a in starrocks::pipeline::PipelineDriver::process(starrocks::RuntimeState*, int) be/src/exec/pipeline/pipeline_driver.cpp:358
    #18 0xfe5ec68 in starrocks::pipeline::GlobalDriverExecutor::_worker_thread() be/src/exec/pipeline/pipeline_driver_executor.cpp:158
    #19 0xfe5d2cf in operator() be/src/exec/pipeline/pipeline_driver_executor.cpp:58
    #20 0xfe6807d in __invoke_impl<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/invoke.h:61
    #21 0xfe6745b in __invoke_r<void, starrocks::pipeline::GlobalDriverExecutor::initialize(int)::<lambda()>&> /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/invoke.h:111
    #22 0xfe66e10 in _M_invoke /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/std_function.h:290
    #23 0xbc8c1a3 in std::function<void ()>::operator()() const /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/std_function.h:591
    #24 0xc4a27ef in starrocks::FunctionRunnable::run() be/src/util/threadpool.cpp:59
    #25 0xc49d645 in starrocks::ThreadPool::dispatch_thread() be/src/util/threadpool.cpp:635
    #26 0xc4bf625 in void std::__invoke_impl<void, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(std::__invoke_memfun_deref, void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) (/home/disk1/sr/be/lib/starrocks_be+0xc4bf625)
    #27 0xc4bdf38 in std::__invoke_result<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>::type std::__invoke<void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&>(void (starrocks::ThreadPool::*&)(), starrocks::ThreadPool*&) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/bits/invoke.h:96
    #28 0xc4bc9c5 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/functional:513
    #29 0xc4baf95 in void std::_Bind<void (starrocks::ThreadPool::*(starrocks::ThreadPool*))()>::operator()<, void>() /opt/rh/gcc-toolset-10/root/usr/include/c++/14.2.0/functional:598

Thread T308 (pip_exec_com) created by T0 here:
    #0 0xb9e74ed in pthread_create ../../.././libsanitizer/asan/asan_interceptors.cpp:245
    #1 0xc483183 in starrocks::Thread::start_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()> const&, unsigned long, scoped_refptr<starrocks::Thread>*) be/src/util/thread.cpp:321
    #2 0xc4a6d80 in starrocks::Status starrocks::Thread::create<void (starrocks::ThreadPool::*)(), starrocks::ThreadPool*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, void (starrocks::ThreadPool::* const&)(), starrocks::ThreadPool* const&, scoped_refptr<starrocks::Thread>*) be/src/util/thread.h:71
    #3 0xc49f094 in starrocks::ThreadPool::create_thread() be/src/util/threadpool.cpp:694
    #4 0xc499ca4 in starrocks::ThreadPool::do_submit(std::shared_ptr<starrocks::Runnable>, starrocks::ThreadPoolToken*, starrocks::ThreadPool::Priority) be/src/util/threadpool.cpp:457
    #5 0xc498838 in starrocks::ThreadPool::submit(std::shared_ptr<starrocks::Runnable>, starrocks::ThreadPool::Priority) be/src/util/threadpool.cpp:364
    #6 0xc498994 in starrocks::ThreadPool::submit_func(std::function<void ()>, starrocks::ThreadPool::Priority) be/src/util/threadpool.cpp:368
    #7 0xfe5d459 in starrocks::pipeline::GlobalDriverExecutor::initialize(int) be/src/exec/pipeline/pipeline_driver_executor.cpp:58
    #8 0xf2784a6 in starrocks::workgroup::PipelineExecutorSet::start() be/src/exec/workgroup/pipeline_executor_set.cpp:104
    #9 0xff22e43 in starrocks::workgroup::ExecutorsManager::start_shared_executors_unlocked() const be/src/exec/workgroup/pipeline_executor_set_manager.cpp:37
    #10 0xf28f62c in starrocks::workgroup::WorkGroupManager::start() be/src/exec/workgroup/work_group.cpp:637
    #11 0xbc6416d in starrocks::ExecEnv::init(std::vector<starrocks::StorePath, std::allocator<starrocks::StorePath> > const&, bool) be/src/runtime/exec_env.cpp:439
    #12 0xc230d0c in starrocks::start_be(std::vector<starrocks::StorePath, std::allocator<starrocks::StorePath> > const&, bool) be/src/service/service_be/starrocks_be.cpp:208
    #13 0xba48b6b in main be/src/service/starrocks_main.cpp:258

What I'm doing:

Fixes https://github.com/StarRocks/StarRocksTest/issues/9294

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
  • This is a backport pr

@before-Sunrise @mergify
[BugFix] fix upgrade_if_overflow ASAN crash (#56261)
0714711 
Signed-off-by: before-Sunrise <[email protected]>
(cherry picked from commit 19edb0e)
@mergify mergify bot mentioned this pull request Mar 4, 2025
Merged
24 tasks
@wanpengfei-git wanpengfei-git enabled auto-merge (squash) March 4, 2025 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@before-Sunrise before-Sunrise

Labels
automerge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@before-Sunrise