Clarification needed on plans to support D-in-D & IntelliJ on Macs (incl arm64)

[szab100]

Hi,

This FR is about the following two known limitations:

⚠️ Docker-in-Docker depends on the environment you use run-gp in. It does not work yet on MacOS and when run-gp is used from within a Gitpod workspace.
⚠️ JetBrains Gateway support also depends on the environment run-gp is used in. It is known NOT to work on arm64 MacOS.

Can you please help us better understand the reasons of these limitations, as well as future plans to overcome / fix them?

Update: I ran a project with "run-gp" and I see all processes are running with QEMU x86-64, which makes things slower and I guess the IntelliJ agent is incompatible. I also saw the Docker-in-Docker issue, docker cli in the workspace cannot connect to the host's docker socket.

Asking since these limitations affect almost 100% of our use-cases:

• all our devs are on Macs, with a fast increasing portion using arm64
• the vast majority of our devs using IntelliJ

[szab100]

@csweichel @svenefftinge

Since my initial post, I ran a few tests and been searching for viable workarounds, like running paravirtualized arm64 or emulated amd64 (desktop) VMs on MacOS M1/2 hosts with more or less success: the emulated amd64 is fully compatible, but is quite slow.. In general we should not need to virtualize a complete desktop OS to achieve the goals, but if so, only arm64 Linux is a viable option.

BTW, it seems JetBrains GW offers native arm64 support with their latest EAP releases (which should work within both Linux or Mac containers).

Finally, I found project Lima, which looks very promising:

A simple, flexible CLI (brew install lima) providing small, efficient managed VM(s) to run either containerd / docker / podman that can simply be used by the corresponding CLIs from the host. It provides very flexible profile-based configuration, as well as intel-on-arm / arm-on-arm, arm-on-intel, etc variations, rootful containers, etc.. So, I believe both D-in-D & IntelliJ GW should work fine. The main question is performance, in case we still need intel-on-arm (for JB GW), but even in this case Lima seems to offer a Fast mode that can potentially make this a viable option (incl. "cross-building" images onto the various archs).

This would also provide a cost-effective solution, eg. no need to purchase costly Docker Desktop user licenses.

[szab100]

Today I played around a bit with Lima and made some tests.

The multi-arch setup runs perfectly on my M1 Pro Mac (tried both docker [below] & the containerd "Fast-Mode"):

$ brew install lima
$ limactl start https://gist.githubusercontent.com/fuzmish/a1130fd2b41a54c44974e6cfe8447069/raw/4f9213cf6c1e940b699ce9286757cdc70a16493e/docker.yaml
$ docker context create lima-docker --docker "host=unix:///Users/sfruhwald/.lima/docker/sock/docker.sock"
$ docker context use lima-docker-rootful

$ docker run --rm --platform linux/arm64 --entrypoint "/usr/bin/uname" ubuntu -a
Linux 0890e34462a7 5.15.0-47-generic #51-Ubuntu SMP Fri Aug 12 08:18:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

$ docker run --rm --platform linux/amd64 --entrypoint "/usr/bin/uname" ubuntu -a
Linux 08186fabf2be 5.15.0-47-generic #51-Ubuntu SMP Fri Aug 12 08:18:32 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

However, running sysbench cpu tests has shown that their "Fast-mode" isn't any faster (in terms of what's running inside the amd64 container). I went a bit further and cloned one of our massive (~5GB) Java projects with 100s of Maven sub-modules and tried building it. Git cloning speed was about 3x slower (but seemed OK), but unfortunately, running the Maven java build was unbearably slow, so the arm64 ==> amd64 software emulation via QEMU seems to be unusably slow.

The only truly viable option is to build native arm64 Gitpod images (incl. supervisor and JetBrains & VS Code workspace agents) and run that with Lima (so that D-in-D & rootful containers are supported & no Docker Desktop license is needed).

I tested performance using an arm64 ubuntu container with lima with the same massive Java project and the Maven build was fast, on par with the current Gitpod experience in the cloud! Even without JetBrains IDEs, the experience is much better than how run-gp currently runs on Mac with VS Code, since Docker Desktop is still running each processes through QEMU with x86_64 emulation ==> everything is quite slow. Using the docker-context of lima by the docker CLI (as seen above) made run-gp talk to the dockerd in lima, but it was unable to start the container, due to the workspace image built as amd64, instead of arm64.

[csweichel]

But it was unable to start the container, due to the workspace image built as amd64, instead of arm64

Is a mixed mode possible in lima? I.e. what if we ran an x86 supervisor + IDE, but the rest ran as arm64 because we're using an arm64 workspace image? run-gp does not enforce a particular platform during the build.