Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

double free or corruption (!prev) #5191

Open
amorenoz opened this issue Jan 17, 2025 · 1 comment
Open

double free or corruption (!prev) #5191

amorenoz opened this issue Jan 17, 2025 · 1 comment

Comments

@amorenoz
Copy link
Contributor

Hi, I'm hitting this issue with bcc's master branch.

double free or corruption (!prev)                                                                                                                                                                        
                                                                                                                                                                                                         
Program received signal SIGABRT, Aborted.
0x00007ffff767f0f4 in __pthread_kill_implementation () from /lib64/libc.so
(gdb) bt
#0  0x00007ffff767f0f4 in __pthread_kill_implementation () from /lib64/libc.so.6
#1  0x00007ffff7625fde in raise () from /lib64/libc.so.6
#2  0x00007ffff760d942 in abort () from /lib64/libc.so.6
#3  0x00007ffff760e7a7 in __libc_message_impl.cold () from /lib64/libc.so.6
#4  0x00007ffff7689265 in malloc_printerr () from /lib64/libc.so.6
#5  0x00007ffff768b44c in _int_free_merge_chunk () from /lib64/libc.so.6
#6  0x00007ffff768dfce in free () from /lib64/libc.so.6
#7  0x00007fffd9bdf25e in std::__new_allocator<long>::deallocate (this=0x555555845b98, __p=0x2b908d, __n=356881)
    at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/new_allocator.h:172
#8  std::allocator_traits<std::allocator<long> >::deallocate (__a=..., __p=0x2b908d, __n=356881) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/alloc_traits.h:513
#9  std::_Vector_base<long, std::allocator<long> >::_M_deallocate (this=0x555555845b98, __p=0x2b908d, __n=356881)
    at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_vector.h:389
#10 std::_Vector_base<long, std::allocator<long> >::~_Vector_base (this=0x555555845b98) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_vector.h:368
#11 std::vector<long, std::allocator<long> >::~vector (this=0x555555845b98) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_vector.h:738
#12 llvm::TensorSpec::~TensorSpec (this=0x555555845b70) at /usr/src/debug/llvm-19.1.5-1.fc41.x86_64/llvm/include/llvm/Analysis/TensorSpec.h:62
#13 std::_Destroy<llvm::TensorSpec> (__pointer=0x555555845b70) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_construct.h:151
#14 std::_Destroy_aux<false>::__destroy<llvm::TensorSpec*> (__first=0x555555845b70, __last=0x555555846700)
    at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_construct.h:163
#15 std::_Destroy<llvm::TensorSpec*> (__first=<optimized out>, __last=0x555555846700) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_construct.h:195
#16 std::_Destroy<llvm::TensorSpec*, llvm::TensorSpec> (__first=<optimized out>, __last=0x555555846700) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/alloc_traits.h:944
#17 std::vector<llvm::TensorSpec, std::allocator<llvm::TensorSpec> >::~vector (this=0x7fffe9297668 <llvm::FeatureMap>)
    at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_vector.h:735
#18 0x00007ffff7628701 in __run_exit_handlers () from /lib64/libc.so.6
#19 0x00007ffff76287ce in exit () from /lib64/libc.so.6
#20 0x00007ffff7a66538 in Py_Exit () from /lib64/libpython3.13.so.1.0
#21 0x00007ffff7a58dc9 in ?? () from /lib64/libpython3.13.so.1.0
#22 0x00007ffff7a58c19 in ?? () from /lib64/libpython3.13.so.1.0
#23 0x00007ffff785db96 in ?? () from /lib64/libpython3.13.so.1.0
#24 0x00007ffff7a58661 in ?? () from /lib64/libpython3.13.so.1.0
#25 0x00007ffff7a50c58 in Py_RunMain () from /lib64/libpython3.13.so.1.0
#26 0x00007ffff7a0a8dc in Py_BytesMain () from /lib64/libpython3.13.so.1.0
#27 0x00007ffff760f248 in __libc_start_call_main () from /lib64/libc.so.6
#28 0x00007ffff760f30b in __libc_start_main_impl () from /lib64/libc.so.6
#29 0x0000555555555095 in _start ()

My setup is Fedora 41 with llvm.x86_64 19.1.5-1.fc41, python 3.13.1 and bcc master (fea5b15). Steps to reproduce:

  1. Build bcc from source
  2. Run any tool, it doesn't even need to actually insert any ebpf. Running without permissions also causes the error. E.g:
$ python ./tools/cpudist.py 1 1
could not open bpf map: dist, error: Operation not permitted
Traceback (most recent call last):
  File "/home/amorenoz/src/bcc/./tools/cpudist.py", line 248, in <module>
    b = BPF(text=bpf_text, cflags=["-DMAX_PID=%d" % max_pid])
  File "/usr/lib/python3.13/site-packages/bcc-0.32.0+fea5b15d-py3.13.egg/bcc/__init__.py", line 505, in __init__
    raise Exception("Failed to compile BPF module %s" % (src_file or "<text>"))
Exception: Failed to compile BPF module <text>
double free or corruption (out)
Aborted (core dumped)

Not sure if this bug falls in bcc's side or llvm's but starting here as it's likely others will hit this issue.
@amorenoz
Copy link
Contributor Author

amorenoz commented Jan 17, 2025 •
edited
Loading

Another look at this (this time with --help) yields a slightly different error message but pointing to the same area.

$ gdb -ex run --args python ./tools/cpudist.py 1 1 --help
GNU gdb (Fedora Linux) 15.2-3.fc41
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.fedoraproject.org/>
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
Reading symbols from /home/amorenoz/.cache/debuginfod_client/59deab8a1f44002a2b5d17e12d969cfe9afdc2d8/debuginfo...
Starting program: /usr/bin/python ./tools/cpudist.py 1 1 --help
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
usage: cpudist.py [-h] [-O] [-T] [-m] [-P] [-L] [-p PID] [-I] [-e] [interval] [count]

Summarize on- and off-CPU time per task as a histogram.

positional arguments:
  interval            output interval, in seconds
  count               number of outputs

options:
  -h, --help          show this help message and exit
  -O, --offcpu        measure off-CPU time
  -T, --timestamp     include timestamp on output
  -m, --milliseconds  millisecond histogram
  -P, --pids          print a histogram per process ID
  -L, --tids          print a histogram per thread ID
  -p, --pid PID       trace this PID only
  -I, --include-idle  include CPU idle time
  -e, --extension     show extension summary (average/total/count)

examples:
    cpudist              # summarize on-CPU time as a histogram
    cpudist -O           # summarize off-CPU time as a histogram
    cpudist 1 10         # print 1 second summaries, 10 times
    cpudist -mT 1        # 1s summaries, milliseconds, and timestamps
    cpudist -P           # show each PID separately
    cpudist -p 185       # trace PID 185 only
    cpudist -I           # include CPU idle time
    cpudist -e           # show extension summary (average/total/count)

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff768dfc1 in arena_for_chunk (ptr=0x5555556b4890) at /usr/src/debug/glibc-2.40-17.fc41.x86_64/malloc/arena.c:153
153       return chunk_main_arena (ptr) ? &main_arena : heap_for_ptr (ptr)->ar_ptr;
(gdb) bt
#0  0x00007ffff768dfc1 in arena_for_chunk (ptr=0x5555556b4890) at /usr/src/debug/glibc-2.40-17.fc41.x86_64/malloc/arena.c:153
#1  arena_for_chunk (ptr=0x5555556b4890) at /usr/src/debug/glibc-2.40-17.fc41.x86_64/malloc/arena.c:151
#2  __GI___libc_free (mem=0x5555556b48a0) at malloc.c:3397
#3  0x00007fffd9bdf275 in std::__new_allocator<char>::deallocate (this=0x5555556b5870, __p=0x7ffff77f6ac0 <main_arena> "", __n=1) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/new_allocator.h:172
#4  std::allocator_traits<std::allocator<char> >::deallocate (__a=..., __p=0x7ffff77f6ac0 <main_arena> "", __n=1) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/alloc_traits.h:513
#5  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy (this=0x5555556b5870, __size=<optimized out>) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/basic_string.h:294
#6  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=0x5555556b5870) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/basic_string.h:288
#7  std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=0x5555556b5870) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/basic_string.h:809
#8  llvm::TensorSpec::~TensorSpec (this=0x5555556b5870) at /usr/src/debug/llvm-19.1.5-1.fc41.x86_64/llvm/include/llvm/Analysis/TensorSpec.h:62
#9  std::_Destroy<llvm::TensorSpec> (__pointer=0x5555556b5870) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_construct.h:151
#10 std::_Destroy_aux<false>::__destroy<llvm::TensorSpec*> (__first=0x5555556b5870, __last=0x5555556b6450) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_construct.h:163
#11 std::_Destroy<llvm::TensorSpec*> (__first=<optimized out>, __last=0x5555556b6450) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_construct.h:195
#12 std::_Destroy<llvm::TensorSpec*, llvm::TensorSpec> (__first=<optimized out>, __last=0x5555556b6450) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/alloc_traits.h:944
#13 std::vector<llvm::TensorSpec, std::allocator<llvm::TensorSpec> >::~vector (this=0x7fffe9297668 <llvm::FeatureMap>) at /usr/bin/../lib/gcc/x86_64-redhat-linux/14/../../../../include/c++/14/bits/stl_vector.h:735
#14 0x00007ffff7628701 in __run_exit_handlers (status=status@entry=0, listp=0x7ffff77f6680 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
#15 0x00007ffff76287ce in __GI_exit (status=status@entry=0) at exit.c:138
#16 0x00007ffff7a66538 in Py_Exit (sts=0) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Python/pylifecycle.c:3412
#17 0x00007ffff7a58dc9 in handle_system_exit () at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Python/pythonrun.c:649
#18 0x00007ffff7a58c19 in _PyErr_PrintEx (tstate=0x7ffff7d3a870 <_PyRuntime+283024>, set_sys_last_vars=1) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Python/pythonrun.c:658
#19 0x00007ffff785db96 in PyErr_PrintEx (set_sys_last_vars=1) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Python/pythonrun.c:735
#20 PyErr_Print () at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Python/pythonrun.c:741
#21 _PyRun_SimpleFileObject (fp=fp@entry=0x555555570240, filename=filename@entry=0x7fffe9667690, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffd7f8) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Python/pythonrun.c:496
#22 0x00007ffff7a58661 in _PyRun_AnyFileObject (fp=fp@entry=0x555555570240, filename=filename@entry=0x7fffe9667690, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffd7f8) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Python/pythonrun.c:77
#23 0x00007ffff7a50c58 in pymain_run_file_obj (program_name=0x7fffe9636eb0, filename=0x7fffe9667690, skip_source_first_line=0) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Modules/main.c:409
#24 pymain_run_file (config=0x7ffff7d0cf68 <_PyRuntime+96392>) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Modules/main.c:428
#25 pymain_run_python (exitcode=0x7fffffffd7ec) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Modules/main.c:696
#26 Py_RunMain () at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Modules/main.c:775
#27 0x00007ffff7a0a8dc in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/python3.13-3.13.1-2.fc41.x86_64/Modules/main.c:829
#28 0x00007ffff760f248 in __libc_start_call_main (main=main@entry=0x555555555160 <main>, argc=argc@entry=5, argv=argv@entry=0x7fffffffda58) at ../sysdeps/nptl/libc_start_call_main.h:58
#29 0x00007ffff760f30b in __libc_start_main_impl (main=0x555555555160 <main>, argc=5, argv=0x7fffffffda58, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffda48) at ../csu/libc-start.c:360
#30 0x0000555555555095 in _start ()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@amorenoz