Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to v1.6.1 #18

Open
wants to merge 304 commits into
base: main-numpymeson
Choose a base branch
from
Open

Conversation

MichaIng
Copy link

@MichaIng MichaIng commented Jan 20, 2025

which fixes an issue I was facing when doing some install test on the new GitHub Actions ARM runners: installs of Python projects which depend on NumPy via pip fail, when doing this in a 32-bit armv6l/armv7l container with aarch64 host, or even just when compiling for 32-bit ARM targets on 64-bit system. While older meson detected it as "native build", as the host system can natively execute target binaries, it still required an emulation wrapper. meson 1.6.1 fixes this: mesonbuild#13841

Does the meson fork still contain changes which are not upstreamed? Else, probably a strict dependency can be defined in https://github.com/numpy/numpy/blob/main/pyproject.toml and the submodule dropped instead, if it has no other reason? It was quite confusing for me to see above error with Version: 1.5.2 stated, which pip verbose output clearly showed 1.6.1 being pulled and successfully installed meson-python dependency 😄.

There were some merge conflicts, where I took the upstream variant:

CONFLICT (content): Merge conflict in man/meson.1
CONFLICT (content): Merge conflict in mesonbuild/backend/ninjabackend.py
CONFLICT (content): Merge conflict in mesonbuild/compilers/detect.py
CONFLICT (content): Merge conflict in mesonbuild/coredata.py
CONFLICT (content): Merge conflict in mesonbuild/mdist.py
CONFLICT (content): Merge conflict in libgcrypt/test.json
CONFLICT (content): Merge conflict in test cases/frameworks/24 libgcrypt/test.json

If this may be broken intended changes in the fork, better redo the merge from upstream, and take this as a request for the update 😉.

Of course it would be also possible to cherry-pick the fix I am looking for, which is pretty small: https://patch-diff.githubusercontent.com/raw/mesonbuild/meson/pull/13930.diff

LegStrong and others added 30 commits July 22, 2024 00:16
The existing "*-linux-gnu*" works on the Debian multiarch model for
Linux architectures; tweak it to be "*-gnu*" so it works also for Hurd
architectures.

This makes the cmake-based search of dependencies work on Debian
GNU/Hurd.
In commit c9aa4af we added a refresh
call to git to catch cases where checking for uncommitted changes would
misfire. Unfortunately, that refresh performs a write operation, which
in turn misfires on readonly media. We don't actually care about the
return value of the refresh, since its purpose is solely to make the
next command more accurate -- so ignore it.

Fixes: c9aa4af
Fixes: mesonbuild#13461
We have permission to use that file. Fixes mesonbuild#13442.
Duplicated -delete_rpath arguments will cause macOS's install_name_tool
failed.
A GCC update broke this and is being investigated

Bug: mesonbuild#13465
Due to reasons, Arch has chosen to split out the glib2 package into a
glib2-devel package containing a small handful of python programs. The
references to these programs are contained in the main glib2 package, so
meson fails a lot in CI with e.g.

```
test cases/frameworks/7 gnome/gdbus/meson.build:1:18: ERROR: Dependency 'gio-2.0' tool variable 'gdbus_codegen' contains erroneous value: '/usr/bin/gdbus-codegen'

This is a distributor issue -- please report it to your gio-2.0 provider.
```
…th gdb

makepkg can do this, if when building packages from source you enable
debug. This is apparently being shipped in the /etc/makepkg.conf in
docker containers, which means building AUR packages now requires
installing debugedit, and then bloating your container with
/usr/src/debug. We really do not want that.

Reconfigure so that we do not, in fact, need that.
This causes us to not count the spaces between arguments, thereby
undercounting the number of elements. This is extra important because we
previously double counted all actual characters, covering this issue up.

Fixes: cf0fecf ("backend/ninja: Fix bug in NinjaRule.length_estimate")
At an OS level, Unix-like OSes usually have very large or even
unlimited sized command line limits. In practice, however, many
applications do not handle this (intentionally or otherwise). Notably
Wine has the same limits Windows does, 32,768 characters. Because we
previously double counted most characters, we papered over most
situations that we would need an RSP file on Unix-like OSes with Wine.

To fix this issue I have set the command line limit to 32k, this is
still a massive command line to pass without an RSP file, and will only
cause the use of an RSP file where it is not strictly necessary in a
small number of cases, but will fix Wine applications. Projects who wish
to not use an RSP file can still set the MESON_RSP_THRESHOLD environment
variable to a very large number instead.

Fixes: mesonbuild#13414
Fixes: cf0fecf ("backend/ninja: Fix bug in NinjaRule.length_estimate")
URLError is a subclass of OSError and intermittent server errors can
manifest as OSError while reading instead of a URLError while
establishing a connection, which will cause the fallback url to be
ignored:

```
Looking for a fallback subproject for the dependency gudev-1.0
Downloading libgudev source from https://gitlab.gnome.org/GNOME/libgudev/-/archive/238/libgudev-238.tar.bz2
HTTP Error 404: Not Found
WARNING: failed to download with error: could not get https://gitlab.gnome.org/GNOME/libgudev/-/archive/238/libgudev-238.tar.bz2 is the internet available?. Trying after a delay...
HTTP Error 404: Not Found
WARNING: failed to download with error: could not get https://gitlab.gnome.org/GNOME/libgudev/-/archive/238/libgudev-238.tar.bz2 is the internet available?. Trying after a delay...
HTTP Error 404: Not Found
WARNING: failed to download with error: could not get https://gitlab.gnome.org/GNOME/libgudev/-/archive/238/libgudev-238.tar.bz2 is the internet available?. Trying after a delay...
WARNING: failed to download with error: The read operation timed out. Trying after a delay...
WARNING: failed to download with error: The read operation timed out. Trying after a delay...
ERROR: Unhandled python OSError. This is probably not a Meson bug, but an issue with your build environment.
```
This is a bit of a hack, since the rule is added outside of the
`__init__` method, and that's probably bad. But at least we can get some
additional help by telling type checkers what it will be
On Windows, the output read from the stream has '\r\n', which in .txt,
.json and console logger (when captured to a file) translates to '\n\n'.

This results in every log line being separated by an empty line.
ninjabackend.py goes down from 322 type errors to 253, mainly by marking
functions as returning None.
It's always true because the func is always a real, truthy func object.

In the cmake case, the logic seems to be broken because if a path is not
a file, then that includes the case where it does not exist. It also
clearly meant "or" instead of "and".

What actually ended up happening was that this check never fired at all.
Because "if not func and not ..." would always fail, because "not func"
is always false. Maybe we don't need this logic at all...
We have a function that wraps two others because it first checks whether
the input is static or shared. By the same token, it has to return types
valid for either. We already *know* that we are a shared library, so we
can and should use the real function directly, which is both a
micro-optimization of function call overhead, and fixes a mypy
"union-attr" error.
… hasattr

If it exists, we want to know it has the right type when we use it in
ninjabackend.py.
smcv and others added 26 commits December 15, 2024 17:04
In Debian testing/unstable, there are wrappers available for various
GObject-Introspection tools during cross-builds, using qemu internally.

Signed-off-by: Simon McVittie <[email protected]>
(cherry picked from commit 851d866)
Recent versions of the architecture-properties package provide a
cross-exe-wrapper package containing
${DEB_HOST_GNU_TYPE}-cross-exe-wrapper, which is currently a wrapper
around qemu-user but could use different emulators on each architecture
if it becomes necessary in the future.

Signed-off-by: Simon McVittie <[email protected]>
(cherry picked from commit 86f9fe4)
This is functionally equivalent to the logic used to locate the cross
exe_wrapper, but puts it below the "Compilers" heading rather than
"Other binaries".

Signed-off-by: Simon McVittie <[email protected]>
(cherry picked from commit f0a86b2)
This adds support for parsing the new `autolib` member
(rust-lang/cargo#14591).

This is a quick and easy fix, suitable for backport. A larger more
involved fix will be to stop splatting out the cargo dictionaries, and
instead pass the fields we know about 1 by 1, and warning the user if we
come across unknown fields.

Fixes: mesonbuild#13826
(cherry picked from commit b131b2d)
Add a check before using an `appleframeworks` to make sure it was found.
This fixes an exception in meson while encountering an optional dep for
the target.

(cherry picked from commit 273894d)
`--html-nested` was added to gcovr in version 6.0 so passing
it to versions before this will result in gcovr complaining that
it doesn't recognise the argument.

Added a version check to ensure that we pass a recognised argument
for versions before 6.0

Fixes mesonbuild#13781

(cherry picked from commit 4c7226e)
If the same source is provided by multiple dependencies it was added
multiple times, as `added_sources` was only guarding against duplicates
within the same source list. This was not a problem with ninja, but it
triggers multiple sanity checks within xcode backend while attempting to
create multiple ids for the same file.

Rename `added_sources` to `seen_sources` as per reviewers request

(cherry picked from commit 7b10f48)
Using a rustup-based toolchain fails the "rust/2 sharedlib" test for me:

./prog: error while loading shared libraries: libstd-211931512faabf29.so: cannot open shared object file: No such file or directory

This happens because recent rustup places the standard library under
SYSROOT/lib/rustlib/TARGET/lib.  Retrieve the right directory using
"--print target-libdir".  This also provides a more accurate version
for rustc installed in /usr.

Before:
  $ echo $(/usr/bin/rustc --print sysroot)/lib
  /usr/lib

After:
  $ /usr/bin/rustc --print target-libdir
  /usr/lib/rustlib/x86_64-unknown-linux-gnu/lib

While at it, cache the value to avoid repeated process invocation.

Signed-off-by: Paolo Bonzini <[email protected]>
(cherry picked from commit 9f3f88f)
It is possible to run a container or chroot with one ABI on a CPU and
kernel that would normally have a different ABI, most commonly by
running a 32-bit container on a 64-bit CPU and kernel. When we do a
native build in such an environment, the build and host architectures
are both equal to the architecture of the container, and it is safe to
assume that we can run executables from that architecture, because if
we could not, we wouldn't be running Python successfully.

Until now, we have been handling this by adding explicit special
cases in `machine_info_can_run()` for each known-good combination of
the detected CPU and the host architecture: every x86_64 can run x86
binaries, and every mips64 is assumed to be able to run 32-bit mips
binaries. However, the equivalent would not be true on ARM systems: *most*
aarch64 CPUs can run arm binaries, but not all (according to Wikipedia,
ARM Cortex-A34 is an example of a purely 64-bit CPU that cannot execute
32-bit instructions).

Instead, assume that if we are doing a native build (not a cross build),
by definition we can run build-architecture executables, and since the
host architecture is equal to the build architecture during a native
build, this implies that we can run host-architecture executables too.

This makes the behaviour of `need_exe_wrapper()` consistent with
`meson.can_run_host_binaries()`, which in turn avoids `Compiler.run()`
failing with error message "Can not run test applications
in this cross environment" during native builds even though
`meson.can_run_host_binaries()` has previously succeeded.

Resolves: mesonbuild#13841
Signed-off-by: Simon McVittie <[email protected]>
(cherry picked from commit fd309ff)
Co-authored-by: Xavier Claessens <[email protected]>
(cherry picked from commit d8ea5c4)
We don't need an extra process in the process tree (specifically the
`meson devenv` process itself). Aside for the redundancy, it's actively
problematic if you abort a program in the devenv by using CTRL+C, as
meson itself will then emit a traceback (KeyboardInterrupt) which is
quite ugly.

(cherry picked from commit 1985ad1)
The `-show` switch was added into the cmake-generated scripts in
HDFGroup/hdf5@6fa09bd and backported to 1.14.5.

(cherry picked from commit 33376ef)
Signed-off-by: Paolo Bonzini <[email protected]>
(cherry picked from commit 8578a99)
Do the same as pytorch/test-infra#5959 and download
an old nodejs to keep Ubuntu Bionic working.

Bug: actions/checkout#1809
Bug: actions/runner#3373
(cherry picked from commit 54cab09)
In old versions of Vulkan SDK, VK_SDK_PATH environment variable was used instead
of VULKAN_SDK.

This patch check both environment variables is fallback mode.

(cherry picked from commit bcb6052)
1.6.1

Fixes: mesonbuild#13841

Signed-off-by: MichaIng <[email protected]>
@eli-schwartz
Copy link

Does the meson fork still contain changes which are not upstreamed? Else, probably a strict dependency can be defined in https://github.com/numpy/numpy/blob/main/pyproject.toml and the submodule dropped instead, if it has no other reason? It was quite confusing for me to see above error with Version: 1.5.2 stated, which pip verbose output clearly showed 1.6.1 being pulled and successfully installed meson-python dependency 😄.

The critical detail here is mesonbuild#11307 I guess.

@rgommers
Copy link
Member

The critical detail here is mesonbuild#11307 I guess.

Yes, and also mesonbuild#10921, which I really really should finish off soon.

@MichaIng quick question: did you do a merge with the main-numpymeson branch that preserves those two PRs?

@MichaIng
Copy link
Author

Yes, based on current main-numpymeson branch, I added the upstream remote, did git merge 1.6.1, and resolved the 7 conflicts with the upstream version. So as those two PRs do not change any of the conflicted files, they should be intact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.