Cargo workspace / Monorepo?

[ifsheldon]

Hi! I see now the cosmic-epoch repo is actually a portal with submodules to individual related repos. I wonder if this can be done in a monorepo style with Cargo workspace. I understand this would be a big change, but there're a few reasons:

• Reduce redundant compilation: most related project relies on iced and libcosmic so we now have to build them and their deps again and again, which increase compile time and total compilation storage needed.
• Consistent Rust compiler requirement: I saw different rust toolchain requirements in different projects, so to compile and install the DE, we need to install multiple rust toolchains of different versions. I don't know why, but I think one rustc of a specific version should be sufficient.

Maybe a Cargo.toml with workspece configs should be enough? The idea is that related projects still have their own repos (issues, PRs, etc) and cosmic-epoch repo still links them as submodules, but a Cargo.toml with workspece configs should trick Cargo to consider the submodules as workspace members when we compile them in the directory of cosmic-epoch.

[mmstick]

You don't need different versions of the Rust compiler, and that would break the packaging CI which relies on git repositories for their source tarballs, and for triggering package builds on git commits. Not to mention how incredibly CPU-intense that would be to rebuild the world for every little commit, and how difficult that'd make reviews.

[ifsheldon]

I don't think you have thought through the solutions.

The idea is that related projects still have their own repos (issues, PRs, etc) and cosmic-epoch repo still links them as submodules, but a Cargo.toml with workspece configs should trick Cargo to consider the submodules as workspace members when we compile them in the directory of cosmic-epoch.

This trick will largely keep most things unchanged. At best, just adding one Cargo.toml to this repo will be sufficient I guess.

You don't need different versions of the Rust compiler

You can just try. This was what happened on my machine. Just only install the latest stable rustc (1.84.0) and follow the instructions.

git clone --recurse-submodules https://github.com/pop-os/cosmic-epoch
cd cosmic-epoch
just sysext

The first rust cargo would download is 1.80.1, which is specified in some rust-toolchain.toml file, which is fair enough. Then cargo would download other versions (e.g., 1.80.0, 1.82.0, etc). Maybe this was caused by deeper deps, I don't know, but I didn't have to lie about it.

As for CI, there are real cases of complex Cargo monorepo, say zed and wasmtime, which have ~50 crates and ~10 binaries in one cargo workspace. They manage to do this. I have contributed to wasmtime, the CI doesn't have to rebuild the world for every little commit. cargo is not stupid, it will compile whatever crate/binary you ask it to compile, even in a workspace, and will not rebuild the world.

The reason why I put this in a discussion, instead of filing an issue/feature request out of cold, is I'd like to discuss, not to push anything, but it just prematurely got closed.

[mmstick]

You do not need to use this repository to build packages, and you do not need to use different compilers. Our build system uses one compiler and compiles every package with it. zed and wasmtime are very small projects compared to COSMIC.

[Drakulix]

Cargo doesn't support nested workspaces (yet?) and few of our projects already use a cargo workspace. So supporting this would need major restructuring, that we don't want to engage with at this point.

[ifsheldon]

Cargo doesn't support nested workspaces

No, unfortunately... rust-lang/cargo#5042

few of our projects already use a cargo workspace. So supporting this would need major restructuring, that we don't want to engage with at this point.

OK, that's a fair one.

When I have some time, I will try to do some brute force restructure and see how much it improves compile times (i.e., total compile time and individual one). I am pretty sure that dozens of crates get recompiled dozens of times.