Memory Leak Fix: Implementing jemalloc Allocator

[heemankv]

Memory Leak Fix: Implementing jemalloc Allocator

Issue

Memory leak identified during multiple SNOS job runs, causing cumulative memory growth over time. Memory usage increases with each job completion, even after tasks finish.

Root Cause Analysis

• Multiple tokio spawned tasks lead to memory fragmentation
• System allocator (malloc) requests new memory segments instead of reusing fragmented spaces
• Memory not being efficiently released back to OS

Solution

( See attached screenshots for reference )
Implemented jemalloc as global allocator with aggressive memory release configuration:

#[global_allocator]
static A: jemallocator::Jemalloc = jemallocator::Jemalloc;

Environment Configuration

JEMALLOC_SYS_WITH_MALLOC_CONF="background_thread:true,narenas:1,tcache:false,dirty_decay_ms:0,muzzy_decay_ms:0,abort_conf:true"

Configuration breakdown:

• background_thread:true: Enables background memory management
• narenas:1: Single arena to reduce fragmentation
• tcache:false: Disables thread caching for immediate memory release
• dirty_decay_ms:0: Immediate release of unused dirty pages
• muzzy_decay_ms:0: Immediate release of muzzy pages
• abort_conf:true: Fails fast on configuration errors

Testing

• Memory usage monitored across multiple job runs
• No cumulative memory growth observed with jemalloc implementation
• System stability maintained with aggressive memory release settings

Dependencies Added

[dependencies]
jemallocator = "0.5.4"

Empirical Analysis

Current allocation, leading to memory leaks.

After implementing Jemallocator

Note

While the exact mechanism of the fix requires further investigation, the implementation has effectively resolved the memory leak issue in production environment.
This PR description is enhanced using AI.

[coveralls]

coverage: 67.086%. remained the same
when pulling 4629784 on fix/memory-leak
into 9000464 on main.

[heemankv]

Self review done, lgtm!