Troubleshooting building the binary package on Ubuntu 23.10

[parselings]

Hello !

I am currently trying to compile Ungoogled Chromium on Ubuntu 23.10. And the build failed despite following the instructions.

What I did

sudo apt install -y devscripts equivs

git clone https://github.com/ungoogled-software/ungoogled-chromium-debian.git
cd ungoogled-chromium-debian
git submodule update --init --recursive

debian/rules setup

sudo mk-build-deps -i debian/control
rm ungoogled-chromium-build-deps_*

dpkg-buildpackage -b -uc

The error occured on the last command.

The error I see

[A LOT OF LOGS]

[8535/57298] CXX obj/third_party/ruy/ruy_instrumentation/instrumentation.o
FAILED: obj/third_party/ruy/ruy_instrumentation/instrumentation.o 
clang++-16 -MMD -MF obj/third_party/ruy/ruy_instrumentation/instrumentation.o.d -DRUY_PROFILER -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -DUSE_OZONE=1 -DOFFICIAL_BUILD -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FORTIFY_SOURCE=2 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -D_GNU_SOURCE -DCR_CLANG_REVISION=\"llvmorg-18-init-9505-g10664813-1\" -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -I../.. -Igen -I../../third_party/ruy/src -Wall -Wextra -Wimplicit-fallthrough -Wextra-semi -Wunreachable-code-aggressive -Wthread-safety -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wenum-compare-conditional -Wno-ignored-pragma-optimize -Wno-deprecated-builtins -Wno-bitfield-constant-conversion -Wno-deprecated-this-capture -Wno-invalid-offsetof -Wno-vla-extension -Wshadow -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing -fstack-protector -fno-unwind-tables -fno-asynchronous-unwind-tables -fPIC -pthread -fcolor-diagnostics -fmerge-all-constants -fcrash-diagnostics-dir=../../tools/clang/crashreports -mllvm -instcombine-lower-dbg-declare=0 -ffp-contract=off -m64 -msse3 -ffile-compilation-dir=. -no-canonical-prefixes -ftrivial-auto-var-init=pattern -O2 -fdata-sections -ffunction-sections -fno-unique-section-names -fno-math-errno -fno-omit-frame-pointer -gdwarf-4 -g1 -gdwarf-aranges -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -std=c++20 -Wno-trigraphs -gsimple-template-names -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -Wdate-time -D_FORTIFY_SOURCE=2 -O2 -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wno-unknown-warning-option -Wno-deprecated-declarations -c ../../third_party/ruy/src/ruy/profiler/instrumentation.cc -o obj/third_party/ruy/ruy_instrumentation/instrumentation.o
In file included from ../../third_party/ruy/src/ruy/profiler/instrumentation.cc:16:
../../third_party/ruy/src/ruy/profiler/instrumentation.h:55:8: error: no type named 'string' in namespace 'std'
  std::string Formatted() const;
  ~~~~~^
../../third_party/ruy/src/ruy/profiler/instrumentation.h:147:7: error: use of undeclared identifier 'abort'
      abort();
      ^
../../third_party/ruy/src/ruy/profiler/instrumentation.cc:34:12: error: no member named 'string' in namespace 'std'
  if (std::string(format_) != std::string(other.format_)) {
      ~~~~~^
../../third_party/ruy/src/ruy/profiler/instrumentation.cc:34:36: error: no member named 'string' in namespace 'std'
  if (std::string(format_) != std::string(other.format_)) {
                              ~~~~~^
../../third_party/ruy/src/ruy/profiler/instrumentation.cc:48:6: error: no type named 'string' in namespace 'std'
std::string Label::Formatted() const {
~~~~~^
../../third_party/ruy/src/ruy/profiler/instrumentation.cc:52:12: error: cannot initialize return object of type 'int' with an lvalue of type 'const char *const'
    return format_;
           ^~~~~~~
../../third_party/ruy/src/ruy/profiler/instrumentation.cc:63:5: error: use of undeclared identifier 'abort'
    abort();
    ^
../../third_party/ruy/src/ruy/profiler/instrumentation.cc:65:10: error: cannot initialize return object of type 'int' with an lvalue of type 'char[256]'
  return buf;
         ^~~
8 errors generated.
[8566/57298] ACTION //third_party/dav1d:dav1d_asm_action(//build/toolchain/linux/unbundle:default)
ninja: build stopped: subcommand failed.
make[1]: *** [debian/rules:143 : override_dh_auto_build] Erreur 1
make[1] : on quitte le répertoire « /home/BLABLABLA/Ungoogled/ungoogled-chromium-debian »
make: *** [debian/rules:109 : binary] Erreur 2
dpkg-buildpackage: erreur: le sous-processus fakeroot debian/rules binary a retourné l’état de sortie 2

My environment

$ clang++-16 -v
Ubuntu clang version 16.0.6 (15)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/13
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-linux-gnu/13/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 13.2.0-4ubuntu3' --with-bugurl=file:///usr/share/doc/gcc-13/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-13 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/libexec --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-offload-defaulted --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 13.2.0 (Ubuntu 13.2.0-4ubuntu3)

I have libstdc++6and libstdc++-13-dev installed (I guessed it was used over libc++).

Can you please help me seeing what is going wrong here ?

Thanks !

[PF4Public]

Can you please help me seeing what is going wrong here ?

Chromium is attempting to build itself against libstdc++, but it is know to be a problem since around version 119. Better luck might be with use_custom_libcxx=true. Perhaps @iskunk Had already made that possible?

[iskunk]

FWIW, Debian builds with use_custom_libcxx=false. I think this is more a matter of needing to build in a minimal environment, where only the build-dependencies are installed.

@parselings, are you conversant with (Docker) containers, or Debian chroot builds? With something like Chromium, building directly on your regular system isn't ideal---not only because of the tons of dependencies needed, but also because having additional things installed can break the build, as you suspected.

That said, you still might not get a successful build, because this repo primarily targets Debian proper. Ubuntu is Debian-derived, but there are minor differences that can lead to build failures with non-obvious causes.

If it helps, I do have chromium and ungoogled-chromium packages (source and binary) supporting anything from jammy to noble up on this PPA. If you do not wish to use the binary packages, then you may want to study the sources. These are modified versions of the corresponding chromium source package from Debian sid/unstable, and the big .orig.tar.xz upstream-source tarball is exactly the same, so you need only look at the .debian.tar.xz file for the salient differences. Of particular interest may be the patches under debian/patches/xtradeb/, which are the ones I added for Ubuntu.

[parselings]

Thank you both for your quick answers !

@PF4Public I tried to build the project with the option use_custom_libcxx=true. Sadly, another error occurred as @iskunk suspected.

I have no experience with docker nor chroot in the case of building a project. But I think it's an excellent excuse to learn one of them (along with looking at the patches for Ubuntu, @iskunk ).

I suggest keeping this ticket open until I have a bit of documentation written and ready to share.

[thedeadliestcatch]

@iskunk Do you have a Dockerfile file or CI config to share?

Do you keep an open/public repo here with your patches?

I've been looking around but surprisingly nobody seems to offer a Dockerfile to have a standardized build environment.

[iskunk]

Hello @thedeadliestcatch,

My modifications to Chromium to get it building on Ubuntu can be found in this repository. Run the chromium.sh script on the debian/ subdirectory of the source package.

What are you looking for in the way of a "standardized" build container? Debian and Ubuntu have official images on Docker Hub, and with a few exceptions, the only necessary preparation beyond those images is just the normal dev environment setup (e.g. installing dpkg-dev). And don't forget the infrastructure they provide for hermetic package builds, like sbuild and schroot.

Is there a use case you have in mind that is not served by these resources?

[thedeadliestcatch]

@iskunk Thank you for the quick response time.

Yes, I'm interested in setting up a CI for your packages (Gitlab CI based or similar, can do with just about any system and adapt).

I was mostly referring to ready-made Dockerfiles that are purpose built for building and producing semi-reproducible builds, although I am fine with just regular builds. Not just a base image, after all the magic happens in the RUN/entrypoint sequence..

I assumed someone must have already done this out of convenience.

[iskunk]

What are the goals of setting up this CI? There are a few things you should know:

1. Up-to-date builds of ungoogled-chromium for Ubuntu jammy and later are already available via the XtraDeb apps PPA. These make use of both the xtradeb-convert scripts, and the conversion framework in this repo under convert/. If you just need binaries for your own use, then these might fit the bill.
2. Chromium is a fairly heavy build. If you're committing to build it regularly, whether on a third-party CI host or your own system, that's going to be quite a bit of resources getting used.
3. There isn't much point in having a Dockerfile specific to building Chromium, since the bits that are Chromium-specific---namely, the build dependencies---can change from one version to the next. When that happens, the image you've previously built is no longer useful.

The Dockerfile that I use for doing test builds of Chromium (and other XtraDeb packages) has the following changes on top of the base image:

• Use my on-site APT cache;
• Install dpkg-dev, plus some convenient packages (like less, nano, and time);
• Set up an unprivileged user to perform the build;
• Set up ccache to make repeated builds less painful.

Note that there is nothing particular to Chromium. And outside of dpkg-dev, that's all my own customization, not requirements as such.

I assumed someone must have already done this out of convenience.

That's what the base images are for :-] The additional setup needed to actually build Chromium is not substantial enough to justify packing it up into its own image---you're better off just including those steps in the script that drives the Chromium build. And if you want an image that is customized to your taste, as I did, then you'll need to write your own Dockerfile anyway.

[thedeadliestcatch]

@iskunk This is for a corporate environment where policy explicitly forbids using third-party packages whose provenance and supply chain cannot be guaranteed. There are plenty of resources available, including dedicated systems building codebases just as large as Chromium, if not larger.

Building Chromium without caching on a high-end i9 CPU doesn't take that long actually (benchmarked recently).

I do think your efforts are great and invaluable for folks to build on, but in this specific case it isn't really my call.

Re Dockerfile, I have built something similar for repro kernel builds, if you ever decide to release/put up the Dockerfiles let me know. The portablelinux version already is docker-based.

Do you do it direct from git on a tag trigger or use the Debian source packages? The former seems like the easiest route, IMO, at least for CI.

[iskunk]

Okay, so this comes down to organizational policy. If you have corporate-level resources at your disposal, then yes, building it will not be a problem.

The XtraDeb builds (chromium and ungoogled-chromium) are based on modifications to the Debian chromium source package. (I do not work with the Google upstream source at all.)

If you'd like to recreate the pipeline that creates those packages, I would suggest setting up a Jenkins job to automatically (1) check for new versions of the chromium package in Debian unstable; (2) download it, unpack, run the conversion framework in this repo; (3) apply the xtradeb-convert changes; (4) build it in an Ubuntu image. That is more or less my process, albeit based on e-mail notifications and Launchpad hosted builds.

The Dockerfile I use may see eventual release, but it won't be for the purpose of providing the build environment, because that is trivially covered by the base images. Rather, it would be for convenience features beyond that, such as making it easy to switch between Debian/Ubuntu releases and architectures. Trust me when I say, if you're planning to run through this whole process on your own, setting up the build environment is the easy part.

Let me know if you run into any issues preparing the build, and I'll be happy to help.

[PF4Public]

There are plenty of resources available, including dedicated systems building codebases just as large as Chromium, if not larger.

I wonder if you consider publishing your binaries at https://github.com/ungoogled-software/ungoogled-chromium-binaries

[thedeadliestcatch]

Why would you trust our binaries? If our build process is compromised (or anybody else's), it's a done deal.
That is the main concern for most hobbyist privacy-oriented projects: in the end, it's just too convenient to target the developer or small group of developers.

I personally advocate building things from source. Introducing malicious logic/backdoors in VCS repos, especially git, is far more complex than tampering with binaries. Of course we can go into the rabbit hole of compromised toolchains (compiler plugin based backdoors, etc), but at some point you have to reach a balance between security considerations and pragmatism. We have a fairly consistent and well reviewed process to build sources, including building untrusted sources that might actually be malicious, storing artifacts and quite a bit of telemetry from the process (network traffic, syscall logs, etc).

So far, the Windows builds seem to be "OK", as they are Github action-based and you can more or less review the entire pipeline. Still, there is no guarantee.

Okay, so this comes down to organizational policy. If you have corporate-level resources at your disposal, then yes, building it will not be a problem.

@iskunk Yes, this is no issue. For a weekly build it's negligible cost.

The XtraDeb builds (chromium and ungoogled-chromium) are based on modifications to the Debian chromium source package. (I do not work with the Google upstream source at all.)

I have reviewed the convert part of the repo and it seems pretty straightforward.

Let me know if you run into any issues preparing the build, and I'll be happy to help.

Much appreciated. I will probably put together a Dockerfile with ccache for Debian today. An offtopic question: I've come across the Vanadium repo, how much crossover exists between their patches and plain ungoogled-chromium?

[iskunk]

I wonder if you consider publishing your binaries at https://github.com/ungoogled-software/ungoogled-chromium-binaries

Why would you trust our binaries? ... I personally advocate building things from source.

Supply-chain attacks are certainly a threat, but it is neither practical, nor desirable, for every user (or user org) of a software package to build their own binaries, especially for a package as large and regularly updated as a modern browser. The long-term solution for the "trusting trust" problem is to have multiple, transparent, semi-trusted builds that validate each other by agreeing on the build outputs---a situation enabled by reproducible builds.

This is still a work-in-progress for the software industry generally, and it is certainly so for this project---but that is the eventual goal. The binaries you build may not be the ones officially posted. But ensuring that the build environments are matched so that the outputs likewise reflect each other, and posting that validation publicly in some form, would be a useful exercise.

I'm still focusing on the problem of providing Debian binaries at all, but do hope to get there eventually.

An offtopic question: I've come across the Vanadium repo, how much crossover exists between their patches and plain ungoogled-chromium?

Vanadium, the GrapheneOS remix of Chromium?

Ungoogled-chromium doesn't use any of their patches at present, but given that the projects are similarly aligned, I see no reason why we couldn't cherry-pick a few. I don't know if anyone here has looked at them.