Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GHA main: Add Alpine Linux job, to CI-test musl libc #20741

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ jobs:
os: ubuntu-22.04
host_dmd: gdmd-9
disable_debug_for_dmd_unittests: true # no `-debug` - host frontend too old
- job_name: Alpine 3.21 x64, LDC
os: ubuntu-latest
container_image: alpine:3.21
host_dmd: ldmd2
# macOS
- job_name: macOS 13 x64, DMD (latest)
os: macos-13
Expand All @@ -74,6 +78,7 @@ jobs:
model: 32
name: ${{ matrix.job_name }}
runs-on: ${{ matrix.os }}
container: ${{ matrix.container_image }}
timeout-minutes: 40
env:
# for ci/run.sh:
Expand All @@ -90,6 +95,11 @@ jobs:
run:
shell: bash
steps:
- name: 'Alpine container: Pre-install bash, git and sudo'
if: startsWith(matrix.container_image, 'alpine')
shell: sh
run: apk add bash git sudo

- uses: actions/checkout@v4
with:
fetch-depth: 50
Expand Down Expand Up @@ -149,20 +159,20 @@ jobs:
- name: Test dmd
run: ci/run.sh test_dmd
- name: Test druntime
if: '!matrix.coverage'
if: '!matrix.coverage && (success() || failure())'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was this needed and how does this help? (seems to add && true which is a no-op?)

Copy link
Contributor Author

@kinke kinke Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It runs all of the test steps (if all steps before were successful IIRC), doesn't abort after the 1st failing test step. I.e., you see all failures and don 't have to fix the compiler testsuite before seeing results for druntime and Phobos etc.

run: ci/run.sh test_druntime
- name: 'Windows x86: Add 32-bit libcurl.dll to PATH (required for Phobos unittests)'
if: runner.os == 'Windows' && env.MODEL == '32' && !matrix.coverage
if: runner.os == 'Windows' && env.MODEL == '32' && !matrix.coverage && (success() || failure())
run: |
# LDC
echo "$(dirname "$(which $DC)")/../lib32" >> $GITHUB_PATH
# DMD
echo "$(dirname "$(which $DC)")/../bin" >> $GITHUB_PATH
- name: Test phobos
if: '!matrix.coverage'
if: '!matrix.coverage && (success() || failure())'
run: ci/run.sh test_phobos
- name: Test self-compile
if: '!matrix.coverage' # already re-built with enabled coverage
if: '!matrix.coverage && (success() || failure())' # already re-built with enabled coverage
run: ENABLE_RELEASE=0 ci/run.sh rebuild
- name: Upload coverage report
if: matrix.coverage
Expand Down
28 changes: 17 additions & 11 deletions ci/cirrusci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,24 @@ if [ ! -z ${HOST_DC+x} ] ; then HOST_DMD=${HOST_DC}; fi
if [ -z ${HOST_DMD+x} ] ; then echo "Variable 'HOST_DMD' needs to be set."; exit 1; fi

if [ "$OS_NAME" == "linux" ]; then
export DEBIAN_FRONTEND=noninteractive
packages="git-core make g++ gdb gnupg curl libcurl4 tzdata zip unzip xz-utils llvm valgrind libc6-dbg"
if [ "$MODEL" == "32" ]; then
dpkg --add-architecture i386
packages="$packages g++-multilib libcurl4:i386 libc6-dbg:i386"
if type -P apk &>/dev/null; then
# Alpine
apk add git make g++ ldc \
bash grep coreutils diffutils curl gdb linux-headers dub
else
export DEBIAN_FRONTEND=noninteractive
packages="git-core make g++ gdb gnupg curl libcurl4 tzdata zip unzip xz-utils llvm valgrind libc6-dbg"
if [ "$MODEL" == "32" ]; then
dpkg --add-architecture i386
packages="$packages g++-multilib libcurl4:i386 libc6-dbg:i386"
fi
if [ "${HOST_DMD:0:4}" == "gdmd" ]; then
# ci/run.sh uses `sudo add-apt-repository ...` to add a PPA repo
packages="$packages sudo software-properties-common"
fi
apt-get -q update
apt-get install -yq $packages
fi
if [ "${HOST_DMD:0:4}" == "gdmd" ]; then
# ci/run.sh uses `sudo add-apt-repository ...` to add a PPA repo
packages="$packages sudo software-properties-common"
fi
apt-get -q update
apt-get install -yq $packages
elif [ "$OS_NAME" == "osx" ]; then
# upgrade GNU make
brew install make
Expand Down
10 changes: 10 additions & 0 deletions ci/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,11 @@ test_dmd() {
local args=(ARGS="-O -inline -release")
fi

if type -P apk &>/dev/null; then
# Alpine: no TLS variables support with gdb, https://gitlab.alpinelinux.org/alpine/aports/-/issues/11154
rm compiler/test/runnable/gdb4181.d
fi

$build_path/dmd -g -i -Icompiler/test -release compiler/test/run.d -ofgenerated/run
generated/run -j$N --environment MODEL=$MODEL HOST_DMD=$build_path/dmd "${args[@]}"
}
Expand Down Expand Up @@ -262,6 +267,11 @@ install_host_compiler() {
echo "export DMD=gdmd-$gdc_version" > ~/dlang/gdc-$gdc_version/activate
echo "deactivate(){ echo;}" >> ~/dlang/gdc-$gdc_version/activate
fi
elif type -P apk &>/dev/null; then
# fake install script and create a fake 'activate' script
mkdir -p ~/dlang/$HOST_DMD
echo "export DMD=$HOST_DMD" > ~/dlang/$HOST_DMD/activate
echo "deactivate(){ echo;}" >> ~/dlang/$HOST_DMD/activate
else
local install_sh="install.sh"
download_install_sh "$install_sh"
Expand Down
21 changes: 18 additions & 3 deletions druntime/test/exceptions/Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
TESTS=stderr_msg unittest_assert invalid_memory_operation unknown_gc static_dtor \
ifeq ($(OS),linux)
# FIXME: detect musl libc robustly; just checking Alpine Linux' apk tool for now
ifeq (1,$(shell which apk &>/dev/null && echo 1))
IS_MUSL:=1
endif
JohanEngelen marked this conversation as resolved.
Show resolved Hide resolved
endif

TESTS=stderr_msg unittest_assert invalid_memory_operation static_dtor \
future_message refcounted rt_trap_exceptions_drt catch_in_finally \
message_with_null

# FIXME: segfaults with musl libc
ifneq ($(IS_MUSL),1)
TESTS += unknown_gc
endif

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was unexpected, works with LDC (and its druntime v2.110 though). The test looks as harmless as it gets.

# fails on 32 bit linux
ifneq ($(OS),linux)
TESTS += assert_fail
Expand All @@ -12,8 +24,11 @@ SED:=sed
GDB:=gdb

ifeq ($(OS),linux)
TESTS+=line_trace line_trace_21656 long_backtrace_trunc rt_trap_exceptions cpp_demangle \
memoryerror_null_read memoryerror_null_write memoryerror_null_call memoryerror_stackoverflow
TESTS+=line_trace line_trace_21656 long_backtrace_trunc rt_trap_exceptions cpp_demangle
# registerMemoryAssertHandler requires glibc
ifneq ($(IS_MUSL),1)
TESTS+=memoryerror_null_read memoryerror_null_write memoryerror_null_call memoryerror_stackoverflow
endif
line_trace_dflags:=-L--export-dynamic
endif

Expand Down
8 changes: 8 additions & 0 deletions druntime/test/importc_compare/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
TESTS := importc_compare

# FIXME: fails on Alpine v3.21 with conflicting struct declarations in the C headers:
# /usr/include/asm-generic/fcntl.h(195): Error: struct `importc_includes.flock` conflicts with struct `importc_includes.flock` at /usr/include/fcntl.h(24)
ifeq ($(OS),linux)
ifeq (1,$(shell which apk &>/dev/null && echo 1))
TESTS :=
JohanEngelen marked this conversation as resolved.
Show resolved Hide resolved
endif
endif

include ../common.mak

extra_dflags += -d
11 changes: 7 additions & 4 deletions druntime/test/shared/src/finalize.d
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ extern (C) alias SetFinalizeCounter = void function(shared(size_t*));

void main(string[] args)
{
import utils : dllExt, loadSym;
import utils : dllExt, isDlcloseNoop, loadSym;

auto name = args[0] ~ '\0';
const pathlen = strrchr(name.ptr, '/') - name.ptr + 1;
Expand All @@ -44,7 +44,7 @@ void main(string[] args)
auto nf1 = new NoFinalize;
auto nf2 = new NoFinalizeBig;

shared size_t finalizeCounter;
static shared size_t finalizeCounter;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must be a global (not on the stack), as due to the dlclose no-op, the finalizers might run after main() ends, causing a segfault.

SetFinalizeCounter setFinalizeCounter;
loadSym(h, setFinalizeCounter, "setFinalizeCounter");
setFinalizeCounter(&finalizeCounter);
Expand All @@ -57,8 +57,11 @@ void main(string[] args)
auto r = Runtime.unloadLibrary(h);
if (!r)
assert(0);
if (finalizeCounter != 4)
assert(0);
static if (!isDlcloseNoop)
{
if (finalizeCounter != 4)
assert(0);
}
if (nf1._finalizeCounter)
assert(0);
if (nf2._finalizeCounter)
Expand Down
35 changes: 19 additions & 16 deletions druntime/test/shared/src/load.d
Original file line number Diff line number Diff line change
Expand Up @@ -130,26 +130,29 @@ void main(string[] args)
{
auto name = args[0] ~ '\0';
const pathlen = strrchr(name.ptr, '/') - name.ptr + 1;
import utils : dllExt;
import utils : dllExt, isDlcloseNoop;
name = name[0 .. pathlen] ~ "lib." ~ dllExt;

runTests(name);

// lib is no longer resident
name ~= '\0';
version (Windows)
static if (!isDlcloseNoop)
{
import core.sys.windows.winbase;
assert(!GetModuleHandleA(name.ptr));
// lib is no longer resident
name ~= '\0';
version (Windows)
{
import core.sys.windows.winbase;
assert(!GetModuleHandleA(name.ptr));
}
else
{
import core.sys.posix.dlfcn : dlopen, RTLD_LAZY;
assert(dlopen(name.ptr, RTLD_LAZY | RTLD_NOLOAD) is null);
}
name = name[0 .. $-1];

auto thr = new Thread({runTests(name);});
thr.start();
thr.join();
}
else
{
import core.sys.posix.dlfcn : dlopen, RTLD_LAZY;
assert(dlopen(name.ptr, RTLD_LAZY | RTLD_NOLOAD) is null);
}
name = name[0 .. $-1];

auto thr = new Thread({runTests(name);});
thr.start();
thr.join();
}
15 changes: 4 additions & 11 deletions druntime/test/shared/src/load_13414.d
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ void runTest(string name)
auto h = Runtime.loadLibrary(name);
assert(h !is null);

import utils : loadSym;
import utils : isDlcloseNoop, loadSym;
void function()* pLibStaticDtorHook, pLibSharedStaticDtorHook;
loadSym(h, pLibStaticDtorHook, "_D9lib_1341414staticDtorHookOPFZv");
loadSym(h, pLibSharedStaticDtorHook, "_D9lib_1341420sharedStaticDtorHookOPFZv");
Expand All @@ -20,19 +20,12 @@ void runTest(string name)
*pLibSharedStaticDtorHook = &sharedStaticDtorHook;

const unloaded = Runtime.unloadLibrary(h);
version (CRuntime_Musl)
{
// On Musl, unloadLibrary is a no-op because dlclose is a no-op
assert(!unloaded);
assert(tlsDtor == 0);
assert(unloaded);
assert(tlsDtor == 1);
static if (isDlcloseNoop)
assert(dtor == 0);
}
else
{
assert(unloaded);
assert(tlsDtor == 1);
assert(dtor == 1);
}
}

void main(string[] args)
Expand Down
19 changes: 18 additions & 1 deletion druntime/test/shared/src/utils.di
Original file line number Diff line number Diff line change
@@ -1,12 +1,29 @@
module utils;

version (OSX)
version = Darwin;
else version (iOS)
version = Darwin;
else version (TVOS)
version = Darwin;
else version (WatchOS)
version = Darwin;

version (Windows)
enum dllExt = "dll";
else version (darwin)
else version (Darwin)
enum dllExt = "dylib";
else
enum dllExt = "so";

// on some platforms, dlclose() is a no-op
version (Darwin)
enum isDlcloseNoop = true; // since macOS ~10.12.6 if shared lib uses TLS: https://github.com/rust-lang/rust/issues/28794#issuecomment-368693049
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hasn't been a problem for DMD - simply because it doesn't support shared druntime/Phobos on Darwin, and these tests are skipped. This matters for LDC though, which has tweaked these tests.

else version (CRuntime_Musl)
enum isDlcloseNoop = true; // https://wiki.musl-libc.org/functional-differences-from-glibc.html
else
enum isDlcloseNoop = false;

void loadSym(T)(void* handle, ref T val, const char* mangle)
{
version (Windows)
Expand Down
Loading