Support path dependencies for Cargo.toml patches

[P1n3appl3]

This makes cargo_bazel respect the [patch] section in the manifest when using crates_vendor. This change was originally proposed in #1902, and this PR is a cleaned up version of that.

The added example and docs section show the resulting BUILD files and directory structure, but here's a high level description:

• The call to cargo vendor will no longer download crates which are patched out
• BUILD files are generated by reading the cargo manifest from the patched crate. They're placed in the vendor dir alongside the unpatched crates.
• These generated BUILD files reference the sources of the patched crate through a filegroup, which is the only "manual" work needed to expose the patched crate to the targets in the vendored dir.

That last one was the main point of discussion when this change was initially proposed (see this comment). There are several options here, such as having cargo_bazel generate BUILD files outside the vendor dir, or having the user manually write BUILD files for each patch. We still believe that the compromise of manually writing BUILD files that expose filegroups and letting crates_vendor still generate BUILD files in the normal vendor dir is ideal for a couple of reasons:

• If you wanted to manually write BUILD files to patch out vendored crates with your own, that use case is already supported via the override-targets annotation.
• Even if it'd require the least manual effort, letting cargo_bazel generate files outside of the vendor dir is highly surprising for users, and it'd establish a brittle 2-way dependency between the generated BUILD files inside and outside the vendor dir. If either were ever moved it'd require some nontrivial fixup.
• The BUILD file that exposes filegroups can be the same for all patched crates. See this change we made on fuchsia where we symlinked that BUILD file to all of our patches.

Fuchsia's adoption of bazel for rust is still blocked on this feature. We're happy to make changes or add tests that the maintainers deem necessary so that we can get this landed and avoid having to maintain these patches on top of upstream. We're successfully using this change now, and besides the integration test added in the example, we'll make sure that this feature continues working as we update rules_rust regularly.

[illicitonion]

Hi @P1n3appl3! Sorry for not picking up your work here earlier. I recently added quite a similar change in #3025 which is intended to make path dependencies and [patch]es work for crate_universe rules - I haven't actually tried it out with vendor, but would be interested in your experience trying it out - it currently uses a slightly different approach for the BUILD files (it makes a remote repository which copies in the source files, and uses the existing cargo-bazel BUILD file generation to generate a BUILD file in that remote repository. (This is likely also responsible for the merge conflicts you're running into already with the new PR - sorry!)

Let me know what you think and how you'd like to proceed - I'm not particularly attached to what I merged (other than the tests, the tests are useful! Including #3063 which adds tests that live edits to the source code behind the path dependency work).

[jayzhuang]

Hey @illicitonion, I tried running crate_universe from 0.56.0, which I think include your changes, on Fuchsia's setup, and it didn't work for me. Can you extend your patch path support for vendor as well? It'll greatly help our project.

I saw the following error when generating build files:

Error: Failed to get crate from cache for Package { name: Name("askama"), version: Version { major: 0, minor: 12, patch: 1 }, source: Some(SourceId { url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("github.com")), port: None, path: "/rust-lang/crates.io-index", query: None, fragment: None }, kind: Registry, precise: Some("locked"), name: None }), checksum: Some(Sha256(b79091df18a97caea757e28cd2d5fda49c6cd4bd01ddffd7ff01ace0c0ad2c28)), dependencies: [Dependency { name: Name("askama_derive"), version: Version { major: 0, minor: 12, patch: 5 }, source: None }, Dependency { name: Name("askama_escape"), version: Version { major: 0, minor: 10, patch: 3 }, source: Some(SourceId { url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("github.com")), port: None, path: "/rust-lang/crates.io-index", query: None, fragment: None }, kind: Registry, precise: Some("locked"), name: None }) }], replace: None }

Caused by:
    No such file or directory (os error 2)

For askama, we have the following entry in our Cargo.toml (this is the full file if it helps):

askama = { version = "0.12.1", default-features = false, features = ["config"] }

This is the crate_vendor Bazel target I'm using:

crates_vendor(
    name = "crates_vendor",
    bazel = "//:bazel",
    buildifier = "//:buildifier",
    cargo_lockfile = "//:third_party/rust_crates/Cargo.lock",
    manifests = ["//:third_party/rust_crates/Cargo.toml"],
    mode = "local",
    render_config = render_config(
        generate_rules_license_metadata = True,
    ),
    supported_platform_triples = [
        "x86_64-unknown-linux-gnu",
    ],
    vendor_path = "third_party/rust_crates/vendor",
)