[VPP-1742] VPP Crash when doing packet trace dump

[vvalderrv]

Description

VPP recycles the graph node when the interface is deleted. Most tx graph nodes have format_tx_function. tapcli does not. If tapcli happens to recycle a tx graph node which has format_tx_function, problem will arise when showing the trace buffer which was collected for the interface that tapcli recycles. So this is what I did.

- turn on trace for the graph node that has format_tx_function. In this case, I select vhost-user interface. - send traffic through the graph node's tx interface. - delete the interface that we turned trace on. The trace buffer still has the trace packets for the deleted interface in the system. - create a tapcli interface. It recycles the just deleted interface's graph node index. - show trace crashes on the format trace function due to tapcli does not have the format_tx_trace function. The reason tapcli is called to display the trace buffer because it recycles vhost-user's tx graph node index.

vpp# trace add vhost-user-input 100

trace add vhost-user-input 100

send traffic through the vhost-user interface

vpp# delete vhost VirtualEthernet0/0/0

delete vhost VirtualEthernet0/0/0

vpp# tap connect tap0

tap connect tap0

tapcli-0

vpp# sh trace

sh trace

Thread 1 "vpp_main" received signal SIGSEGV, Segmentation fault.

0x0000000000000000 in ?? ()

(gdb) set pagination off

(gdb) bt

#0 0x0000000000000000 in ?? ()

#1 0x00007ffff6830eb9 in do_percent (va=0x7fffb60ca988, fmt=0x7ffff6d3cb46 "%U", _s=) at /home/sluong/vpp1901/vpp/src/vppinfra/format.c:373

#2 va_format (s=0x7fffb6dfb62c '' <repeats 19 times>, " Start of thread 0 vpp_main ", '' <repeats 19 times>, "\nNo packets in trace buffer\n", '' <repeats 19 times>, " Start of thread 1 vpp_wk_0 ", '' <repeats 19 times>, "\nNo packets in trace buffer\n", '-' <repeats 12 times>..., fmt=, va=va@entry=0x7fffb60ca988) at /home/sluong/vpp1901/vpp/src/vppinfra/format.c:404

#3 0x00007ffff68308d7 in format (s=, fmt=fmt@entry=0x7ffff6d3cb43 "\n %U") at /home/sluong/vpp1901/vpp/src/vppinfra/format.c:423

#4 0x00007ffff6d1fd35 in format_vlib_trace (s=, va=) at /home/sluong/vpp1901/vpp/src/vlib/trace.c:162

#5 0x00007ffff6830eb9 in do_percent (va=0x7fffb60cab78, fmt=0x7ffff6d3cb94 "%U\n\n", _s=) at /home/sluong/vpp1901/vpp/src/vppinfra/format.c:373

#6 va_format (s=0x7fffb614553c ":00:17:319002: ethernet-input\n frame: flags 0x1, hw-if-index 2, sw-if-index 2\n IP4: 62:54:00:00:04:01 -> 52:54:00:00:04:01\n00:00:17:319005: l2-input\n l2-input: sw_if_index 2 dst 52:54:00:00:04:01 s"..., fmt=, va=va@entry=0x7fffb60cab78) at /home/sluong/vpp1901/vpp/src/vppinfra/format.c:404

#7 0x00007ffff68308d7 in format (s=, fmt=fmt@entry=0x7ffff6d3cb8a "Packet %d\n%U\n\n") at /home/sluong/vpp1901/vpp/src/vppinfra/format.c:423

#8 0x00007ffff6d22ff9 in cli_show_trace_buffer (vm=0x7ffff6f4b3c0 <vlib_global_main>, input=, cmd=) at /home/sluong/vpp1901/vpp/src/vlib/trace.c:297

#9 0x00007ffff6ced315 in vlib_cli_dispatch_sub_commands (vm=vm@entry=0x7ffff6f4b3c0 <vlib_global_main>, cm=cm@entry=0x7ffff6f4b5c0 <vlib_global_main+512>, input=input@entry=0x7fffb60caf60, parent_command_index=) at /home/sluong/vpp1901/vpp/src/vlib/cli.c:644

#10 0x00007ffff6ced6da in vlib_cli_dispatch_sub_commands (vm=vm@entry=0x7ffff6f4b3c0 <vlib_global_main>, cm=cm@entry=0x7ffff6f4b5c0 <vlib_global_main+512>, input=input@entry=0x7fffb60caf60, parent_command_index=parent_command_index@entry=0) at /home/sluong/vpp1901/vpp/src/vlib/cli.c:605

#11 0x00007ffff6ceda90 in vlib_cli_input (vm=0x7ffff6f4b3c0 <vlib_global_main>, input=input@entry=0x7fffb60caf60, function=function@entry=0x7ffff6d2bf80 <unix_vlib_cli_output>, function_arg=function_arg@entry=0) at /home/sluong/vpp1901/vpp/src/vlib/cli.c:744

#12 0x00007ffff6d2dee5 in unix_cli_process_input (cm=0x7ffff6f4b140 <unix_cli_main>, cli_file_index=0) at /home/sluong/vpp1901/vpp/src/vlib/unix/cli.c:2420

#13 unix_cli_process (vm=0x7ffff6f4b3c0 <vlib_global_main>, rt=0x7fffb60ba000, f=) at /home/sluong/vpp1901/vpp/src/vlib/unix/cli.c:2536

#14 0x00007ffff6d01636 in vlib_process_bootstrap (_a=) at /home/sluong/vpp1901/vpp/src/vlib/main.c:1440

#15 0x00007ffff683a18c in clib_calljmp () from /home/sluong/vpp1901/vpp/build-root/install-vpp-native/vpp/lib/libvppinfra.so.19.01.3

#16 0x00007fffb5bffb20 in ?? ()

#17 0x00007ffff6d07379 in vlib_process_startup (f=0x0, p=0x7fffb60ba000, vm=0x7ffff6f4b3c0 <vlib_global_main>) at /home/sluong/vpp1901/vpp/src/vlib/main.c:1462

#18 dispatch_process (vm=0x7ffff6f4b3c0 <vlib_global_main>, p=0x7fffb60ba000, last_time_stamp=0, f=0x0) at /home/sluong/vpp1901/vpp/src/vlib/main.c:1507

Assignee

steven luong

Reporter

steven luong

Comments

No comments.

Original issue: https://jira.fd.io/browse/VPP-1742