Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider increasing stack sizes under KMSAN #97

Open
ramosian-glider opened this issue Sep 27, 2024 · 0 comments
Open

Consider increasing stack sizes under KMSAN #97

ramosian-glider opened this issue Sep 27, 2024 · 0 comments

Comments

@ramosian-glider
Copy link
Member

As reported at https://lore.kernel.org/lkml/[email protected]/T/, KMSAN sometimes smashes the kernel stack.

Previously we used to increase stack sizes in KMSAN builds: 060de96, but later it was deemed unnecessary, because the stack usage was not that big as compared to KASAN.

Trying to assess the impact of KMSAN on stack frame sizes, I took two kernel images built with and without CONFIG_KMSAN=y containing roughly 300K functions, and grepped them for sub.*rsp, assuming that the value subtracted from $rsp at function prologue is the stack frame size.
I filtered out some corner cases (functions with duplicate names, functions that only had sub.*rsp in one of the two builds, functions with more than one sub.*rsp instruction) and ended up with a list of 45415 functions with their stack frame sizes in KMSAN and non-KMSAN builds: bloat-kmsan-nokmsan.txt

aa_af_perm 0x100 0xd0
aa_alloc_profile 0x78 0x10
aa_audit_file 0xd0 0x98
aa_audit_perm_mask 0x90 0x70

From this list it was possible to calculate the relative bloat: bloat-percentage.txt

(first column is the line number in the sorted file)

    1 2650.00% dsp_bf_encrypt 0x1b8 0x10
    2 2550.00% camellia_setup_tail 0x1a8 0x10
    3 2000.00% writecache_status 0x150 0x10 
    4 2000.00% __blkcg_rstat_flush 0x150 0x10
    5 1750.00% trace_raw_output_btrfs__writepage 0x128 0x10
    6 1600.00% kcm_seq_show 0x110 0x10
    7 1600.00% ceph_read_dir 0x110 0x10 
...
32745 102.38% btrfs_log_inode 0x2a8 0x150 
32746 102.13% btrfs_fiemap 0x2f8 0x178
32747 100.00% zswap_decompress 0xc0 0x60
32748 100.00% ZSTD_freeDDict 0x30 0x18
32749 100.00% ZSTD_estimateCDictSize 0x40 0x20
...
34579 100.00% aa_profile_label_perm 0xa0 0x50
34580 100.00% aa_perm_mask_to_str 0x30 0x18
34581 98.70% bch2_fs_ioctl 0x4c8 0x268
34582 97.37% bch2_gc_gens 0x258 0x130
...
44672 1.92% nldev_set_sys_set_doit 0x350 0x340
44673 1.64% ext4_normal_submit_inode_data_buffers 0x1f0 0x1e8
44674 0.00% ZSTD_flushStream 0x18 0x18
44675 0.00% ZSTD_estimateDStreamSize_fromFrame 0x28 0x28
44676 0.00% zstd_cstream_workspace_bound 0x20 0x20
...
45387 0.00% acpi_dev_for_each_child 0x10 0x10
45388 0.00% aa_info_message 0x88 0x88
45389 -6.67% mptcp_crypto_hmac_sha 0xe0 0xf0
45390 -11.11% set_track_prepare 0x80 0x90
45391 -12.50% ethtool_get_ringparam 0x38 0x40
...
45410 -50.00% __x64_sys_utime32 0x10 0x20
45411 -50.00% __x64_sys_utime 0x10 0x20
45412 -50.00% intel_irq_remapping_deactivate 0x10 0x20
45413 -50.00% __ia32_sys_utime32 0x10 0x20
45414 -50.00% __ia32_sys_utime 0x10 0x20
45415 -62.50% __populate_section_memmap 0x18 0x40

So 44673 out of 45415 interesting functions (98.4%) had their stack frames increased, and for 34580 (76.1%) their stack usage increased by >= 100%.

This suggests we should probably double the stacks in KMSAN builds and see if that helps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant