Snmalloc v1 maintain oct 10 2021 (#161)

* bump module Signed-off-by: Schrodinger ZHU Yifan <[email protected]> * deprecate cache-friendly Signed-off-by: Schrodinger ZHU Yifan <[email protected]> * bump version and use exposed alloc_zeroed Signed-off-by: Schrodinger ZHU Yifan <[email protected]> * update change log Signed-off-by: Schrodinger ZHU Yifan <[email protected]>
SchrodingerZhu · Oct 11, 2021 · 4292a34 · 4292a34
1 parent 8457da5
commit 4292a34
Show file tree

Hide file tree

Showing 8 changed files with 77 additions and 32 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,10 @@
 ## Changelog
 
+### 0.2.28
+- Deprecation of `cache-friendly`
+- Use exposed `alloc_zeroed` from `snmalloc`
+- **upstream** changes of remote communication, corruption detection and compilation flag detection.
+
 ### 0.2.27
 
 - Reduction of libc dependency

diff --git a/Cargo.toml b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "snmalloc-rs"
-version = "0.2.27"
+version = "0.2.28"
 authors = ["schrodingerzhu <[email protected]>"]
 edition = "2018"
 license = "MIT"
@@ -16,7 +16,7 @@ readme = "README.md"
 members = ["snmalloc-sys" ]
 
 [dependencies]
-snmalloc-sys = {version = "0.2.27", path = "snmalloc-sys", default-features = false }
+snmalloc-sys = {version = "0.2.28", path = "snmalloc-sys", default-features = false }
 
 [features]
 default = ["snmalloc-sys/build_cmake","1mib"]

diff --git a/README.md b/README.md
@@ -1,32 +1,39 @@
 # snmalloc-rs
-**Notice: MinGW Build is broken and may not be fixed in a near future. See [this PR](https://github.com/microsoft/snmalloc/pull/217) in the upstream.**
+
+**Notice: MinGW Build is broken and may not be fixed in a near future.
+See [this PR](https://github.com/microsoft/snmalloc/pull/217) in the upstream.**
 
 MSVC/MinGW/Linux/MacOS: [![Actions Status](https://github.com/schrodingerzhu/snmalloc-rs/workflows/Rust/badge.svg)](https://github.com/schrodingerzhu/snmalloc-rs/actions)
 
 FreeBSD: [![Build Status](https://api.cirrus-ci.com/github/SchrodingerZhu/snmalloc-rs.svg)](https://cirrus-ci.com/github/SchrodingerZhu/snmalloc-rs)
 
-`snmalloc-rs` provides a wrapper for [`microsoft/snmalloc`](https://github.com/microsoft/snmalloc) to make it usable as a global allocator for rust.
-snmalloc is a research allocator. Its key design features are:
+`snmalloc-rs` provides a wrapper for [`microsoft/snmalloc`](https://github.com/microsoft/snmalloc) to make it usable as
+a global allocator for rust. snmalloc is a research allocator. Its key design features are:
 
 - Memory that is freed by the same thread that allocated it does not require any synchronising operations.
-- Freeing memory in a different thread to initially allocated it, does not take any locks and instead uses a novel message passing scheme to return the memory to the original allocator, where it is recycled.
+- Freeing memory in a different thread to initially allocated it, does not take any locks and instead uses a novel
+  message passing scheme to return the memory to the original allocator, where it is recycled.
 - The allocator uses large ranges of pages to reduce the amount of meta-data required.
 
-Some old benchmark results are available in the [`snmalloc` paper](https://github.com/microsoft/snmalloc/blob/master/snmalloc.pdf). Some recent benchmark results are listed at 
-[bench_suite](https://github.com/SchrodingerZhu/bench_suite).
-There are three features defined in this crate:
+Some old benchmark results are available in
+the [`snmalloc` paper](https://github.com/microsoft/snmalloc/blob/master/snmalloc.pdf). Some recent benchmark results
+are listed at
+[bench_suite](https://github.com/SchrodingerZhu/bench_suite). There are three features defined in this crate:
 
 - `debug`: Enable the `Debug` mode in `snmalloc`.
 - `1mib`: Use the `1mib` chunk configuration. From `0.2.17`, this is set as a default feature
 - `16mib`: Use the `16mib` chunk configuration.
-- `cache-friendly`: Make the allocator more cache friendly (setting `CACHE_FRIENDLY_OFFSET` to `64` in building the library).
-- `native-cpu`: Optimize `snmalloc` for the native CPU of the host machine. (this is not a default behavior since `0.2.14`)
+- `cache-friendly`: Make the allocator more cache friendly (setting `CACHE_FRIENDLY_OFFSET` to `64` in building the
+  library).
+  (**deprecated since 0.2.28**)
+- `native-cpu`: Optimize `snmalloc` for the native CPU of the host machine. (this is not a default behavior
+  since `0.2.14`)
 - `qemu`: Workaround `madvise` problem of QEMU environment
 - `stats`: Enable statistics
 - `local_dynamic_tls`: Workaround cannot allocate memory in static tls block
 - `build_cc`: Use of cc crate instead of cmake (cmake still default) as builder (more platform agnostic)
 - `usecxx20`: Enable C++20 standard if available
-- `win8compat`: Improve compatibility for old Windows platforms (removing usages of `VirtualAlloc2` and other new APIs) 
+- `win8compat`: Improve compatibility for old Windows platforms (removing usages of `VirtualAlloc2` and other new APIs)
 
 **To get the crates compiled, you need to choose either `1mib` or `16mib` to determine the chunk configuration**
 
@@ -47,27 +54,36 @@ static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
 
 ## For MinGW Users
 
-`mingw` version is only tested on nightly branch with MSYS environment. We are using dynamic linking method. 
-Hence, please make sure the following libs are in your `PATH`:
+`mingw` version is only tested on nightly branch with MSYS environment. We are using dynamic linking method. Hence,
+please make sure the following libs are in your `PATH`:
 
 - `winpthread`
 - `atomic`
 - `stdc++`
-- `gcc_s` 
+- `gcc_s`
 
-**Notice:** since version `0.2.12`, we no longer require you to provide additional environment variables for `mingw` target.
+**Notice:** since version `0.2.12`, we no longer require you to provide additional environment variables for `mingw`
+target.
 
 ## For Android Cross-Compilation
 
 - `ANDROID_NDK` must be provided as an environment variable
 - `ANDROID_PLATFORM` can be passed as an optional environment variable
 - `ANDROID_ABI` used by CMake is detected automatically
 - feature `android-lld` can be used to set the linker of `snmalloc` to `lld`
-- feature `android-shared-std` can be used to set the STL library of `snmalloc` to `c++_shared` (it uses `c++_static` by default)
+- feature `android-shared-std` can be used to set the STL library of `snmalloc` to `c++_shared` (it uses `c++_static` by
+  default)
 
 ## Changelog
 
+### 0.2.28
+
+- Deprecation of `cache-friendly`
+- Use exposed `alloc_zeroed` from `snmalloc`
+- **upstream** changes of remote communication, corruption detection and compilation flag detection.
+
 ### 0.2.27
+
 - Reduction of libc dependency
 - **upstream** Windows 7 and windows 8 compatibility added
 - **upstream** Option to use C++20 standards if available
@@ -80,16 +96,11 @@ Hence, please make sure the following libs are in your `PATH`:
 - option of cc crate as build feature, only c compiler needed, no cmake required
 - Addition of dynamic local TLS option
 
-
 ### 0.2.25
 
 - **upstream** Apple M1 support
 - **upstream** Building adjust
-- non-allocation tracking functions 
-
-### 0.2.24
+- non-allocation tracking functions
 
-- **upstream** update to use a more efficient power of 2 check
-- fix msvc support w/ crt-static
 
 for older versions, see [CHANGELOG](CHANGELOG.md) 
diff --git a/snmalloc-sys/Cargo.toml b/snmalloc-sys/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "snmalloc-sys"
-version = "0.2.27"
+version = "0.2.28"
 authors = ["schrodingerzhu <[email protected]>"]
 edition = "2018"
 license = "MIT"

diff --git a/snmalloc-sys/build.rs b/snmalloc-sys/build.rs
@@ -117,7 +117,10 @@ fn main() {
     }
 
     if cfg!(feature = "cache-friendly") {
-        build.define("CACHE_FRIENDLY_OFFSET", "64");
+        eprintln!("cache-friendly feature flag is deprecated and no longer has any effect. \
+            it may be removed in a future release");
+        // The following code no longer works
+        // build.define("CACHE_FRIENDLY_OFFSET", "64");
     }
 
     build.compile(target);
@@ -247,14 +250,13 @@ fn main() {
         cfg = cfg.define("SNMALLOC_QEMU_WORKAROUND", "ON")
     }
 
-    let mut dst = if cfg!(feature = "cache-friendly") {
-        cfg.define("CACHE_FRIENDLY_OFFSET", "64")
-            .build_target(target)
-            .build()
-    } else {
-        cfg.build_target(target).build()
+    if cfg!(feature = "cache-friendly") {
+        eprintln!("cache-friendly feature flag is deprecated and no longer has any effect. \
+            it may be removed in a future release");
     };
 
+    let mut dst = cfg.build_target(target).build();
+
     dst.push("./build");
 
     println!("cargo:rustc-link-lib={}", target);

diff --git a/snmalloc-sys/snmalloc b/snmalloc-sys/snmalloc
diff --git a/snmalloc-sys/src/lib.rs b/snmalloc-sys/src/lib.rs
@@ -3,6 +3,13 @@
 
 use {core::ffi::c_void, core::usize};
 
+#[cfg_attr(
+feature = "cache-friendly",
+deprecated(
+since = "0.2.28",
+note = "Crate `snmalloc-sys` enables cache-friendly feature flag, \
+                which is deprecated and no longer has any effect. \
+                It may be removed in a future release"))]
 extern "C" {
     /// Allocate the memory with the given alignment and size.
     /// On success, it returns a pointer pointing to the required memory address.
@@ -20,6 +27,9 @@ extern "C" {
     /// The program may be forced to abort if the constrains are not full-filled.
     pub fn rust_dealloc(ptr: *mut c_void, alignment: usize, size: usize) -> c_void;
 
+    /// Behaves like rust_alloc, but also ensures that the contents are set to zero before being returned.
+    pub fn rust_alloc_zeroed(alignment: usize, size: usize) -> *mut c_void;
+
     /// Re-allocate the memory at the given address with the given alignment and size.
     /// On success, it returns a pointer pointing to the required memory address.
     /// The memory content within the `new_size` will remains the same as previous.
@@ -68,6 +78,15 @@ extern "C" {
 mod tests {
     use super::*;
 
+    #[test]
+    fn it_zero_allocs_correctly() {
+        let ptr = unsafe { rust_alloc_zeroed(8, 1024) } as *mut u8 as *mut [u8; 1024];
+        unsafe {
+            assert!((*ptr).iter().all(|x| *x == 0));
+        };
+        unsafe { rust_dealloc(ptr as *mut c_void, 8, 1024) };
+    }
+
     #[test]
     fn it_frees_memory_malloc() {
         let ptr = unsafe { rust_alloc(8, 8) } as *mut u8;
@@ -77,6 +96,7 @@ mod tests {
         };
         unsafe { rust_dealloc(ptr as *mut c_void, 8, 8) };
     }
+
     #[test]
     fn it_frees_memory_sn_malloc() {
         let ptr = unsafe { sn_malloc(8) } as *mut u8;

diff --git a/src/lib.rs b/src/lib.rs
@@ -44,6 +44,13 @@ unsafe impl GlobalAlloc for SnMalloc {
         ffi::rust_alloc(layout.align(), layout.size()) as _
     }
 
+
+    /// Behaves like alloc, but also ensures that the contents are set to zero before being returned.
+    #[inline(always)]
+    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
+        ffi::rust_alloc_zeroed(layout.align(), layout.size()) as _
+    }
+
     /// De-allocate the memory at the given address with the given alignment and size.
     /// The client must assure the following things:
     /// - the memory is acquired using the same allocator and the pointer points to the start position.
+214 −0		.github/workflows/main.yml
+26 −27		CMakeLists.txt
+1 −1		README.md
+0 −355		azure-pipelines.yml
+13 −0		ci/Toolchain.cmake
+0 −7		ci/linux_arm64
+0 −7		ci/linux_armhf
+0 −17		ci/linux_x64
+0 −7		ci/linux_x86
+0 −19		ci/scripts/build.sh
+0 −14		ci/scripts/test.sh
+11 −5		src/ds/aba.h
+1 −0		src/ds/bits.h
+22 −0		src/ds/defines.h
+15 −0		src/ds/helpers.h
+2 −2		src/ds/ptrwrap.h
+46 −200		src/mem/alloc.h
+2 −1		src/mem/allocslab.h
+8 −2		src/mem/entropy.h
+1 −3		src/mem/freelist.h
+2 −2		src/mem/globalalloc.h
+8 −10		src/mem/metaslab.h
+1 −0		src/mem/pagemap.h
+185 −6		src/mem/remoteallocator.h
+0 −36		src/mem/sizeclass.h
+0 −21		src/mem/sizeclasstable.h
+10 −8		src/mem/threadalloc.h
+7 −0		src/override/rust.cc
+10 −5		src/pal/pal.h
+238 −44		src/pal/pal_apple.h
+1 −1		src/pal/pal_bsd_aligned.h
+8 −0		src/pal/pal_concept.h
+1 −1		src/pal/pal_consts.h
+15 −16		src/pal/pal_haiku.h
+8 −0		src/pal/pal_linux.h
+3 −17		src/pal/pal_netbsd.h
+10 −2		src/pal/pal_posix.h
+1 −1		src/pal/pal_windows.h
+3 −2		src/test/func/pagemap/pagemap.cc