Skip to content

Commit

Permalink
estat backend-io script complains about missing blk_start_request kpr…
Browse files Browse the repository at this point in the history 
…obecloses delphix#45
  • Loading branch information
@mr-t-73
mr-t-73 authored Sep 20, 2021
1 parent acde52d commit 25b8d99
Showing 1 changed file with 0 additions and 1 deletion.
1 change: 0 additions & 1 deletion bpf/estat/backend-io.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ typedef struct {

BPF_HASH(io_base_data, u64, io_data_t);

// @@ kprobe|blk_start_request|disk_io_start
// @@ kprobe|blk_mq_start_request|disk_io_start
int
disk_io_start(struct pt_regs *ctx, struct request *reqp)
Expand Down

1 comment on commit 25b8d99

@mr-t-73
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduce the issue:

^Croot@ip-10-110-230-146:/usr/share/performance-diagnostics/bpf/estat# estat backend-io -m 5
WARNING: kprobe: blk_start_request - not found
09/20/21 - 05:04:13 UTC

 Tracing enabled... Hit Ctrl-C to end.

Function call not available in the kernel:

# cat /proc/kallsyms |grep blk_start_request
#

Not available as a configured kprobe either:

# cat /sys/kernel/debug/kprobes/list |grep blk
ffffffffab0c4640  k  blk_account_io_completion+0x0    [FTRACE]
ffffffffab0ccb00  k  blk_mq_start_request+0x0    [FTRACE]

So, its a part of disk_io_start (the disco ;-) in the script backend-io.c:

vi /usr/share/performance-diagnostics/bpf/estat/backend-io.c 
...
 32 // @@ kprobe|blk_start_request|disk_io_start
 33 // @@ kprobe|blk_mq_start_request|disk_io_start
 34 int
 35 disk_io_start(struct pt_regs *ctx, struct request *reqp)
 36 {

Not present anywhere in the ubuntu source:

$ git grep blk_start_request
$

but mq is:

$ git grep blk_mq_start_request
arch/um/drivers/ubd_kern.c:     blk_mq_start_request(req);
block/blk-mq.c: * blk_mq_start_request - Start processing a request
block/blk-mq.c:void blk_mq_start_request(struct request *rq)
...
include/linux/blk-mq.h:void blk_mq_start_request(struct request *rq);

The Multi-Queue Block I/O Queuing Mechanism (blk-mq) was introduced in Oct 2013 with commit:

320ae51feed5c2f13664aa05a76bec198967e04d

blk-mq deprecated blk_start_request()
blk_start_request() was removed in Nov 2018 with:

$ git show f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6
commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6
Author: Jens Axboe <[email protected]>
Date:   Thu Nov 1 16:36:27 2018 -0600

    scsi: kill off the legacy IO path
    
    This removes the legacy (non-mq) IO path for SCSI.

Summary:

  • The classic I/O scheduler was replaced in Oct 2013.
  • Calls to blk_start_request() were replaced by blk_mq_start_request().
  • This probe should be removed.
  • Tested with probe removed. No error, I/O throughput / latency values look good:
# estat backend-io -m 60
09/20/21 - 07:14:44 UTC

 Tracing enabled... Hit Ctrl-C to end.
   microseconds                                                   write 
value range                 count ------------- Distribution ------------- 
[300, 400)                     25 |@                                       
[400, 500)                   1340 |@                                       
[500, 600)                   3756 |@                                       
[600, 700)                   4606 |@                                       
[700, 800)                   6442 |@@                                      
[800, 900)                  10238 |@@                                      
[900, 1000)                  9253 |@@                                      
[1000, 2000)                42302 |@@@@@@@@@                               
[2000, 3000)                44178 |@@@@@@@@@                               
[3000, 4000)                57406 |@@@@@@@@@@@@                            
[4000, 5000)                14508 |@@@                                     
[5000, 6000)                 7366 |@@                                      
[6000, 7000)                 3655 |@                                       
[7000, 8000)                 1038 |@                                       
[8000, 9000)                  300 |@                                       
[9000, 10000)                 180 |@                                       
[10000, 20000)                608 |@                                       
[20000, 30000)                416 |@                                       
[30000, 40000)                128 |@                                       
[40000, 50000)                 14 |@                                       

                                       iops(/s)  avg latency(us)       stddev(us)  throughput(k/s)
write                                      3462             2703          4085849            25772


                                       iops(/s)  throughput(k/s)
total                                      3462            25773


09/20/21 - 07:15:44 UTC

Please sign in to comment.