Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ClusterMap benchmarks #856

Merged
merged 8 commits into from
Feb 26, 2024
Merged

ClusterMap benchmarks #856

merged 8 commits into from
Feb 26, 2024

Conversation

Jake-Shadle
Copy link
Collaborator

During investigation of #837 (which fortunately? ended with the original issue not being observed) I added several benchmarks.

While getting familiar with the code, I noticed an easy win of Locality using 3 unnecessary heap allocations, now it uses 0, and no longer heap allocates during parsing. This also simplifies how it is serialized to/from json which gives slight speedups on both ends since it's not serializing as many fields. Really, locality is essentially an opaque identifier as far as the codebase is concerned, and it could probably be even more simplified to a single hash value, but that can be done separately, this was mainly just to see numbers change in the benchmarks.

@XAMPPRocky XAMPPRocky enabled auto-merge (squash) November 17, 2023 17:50
@XAMPPRocky
Copy link
Collaborator

XAMPPRocky commented Nov 17, 2023

We just need to add an exception in the cargo-deny for xxhash's BSL-1.0 (Boost Software Licence), that is allowed by Google and we already do the work of including licences in distribution because it's a requirement for Apache. https://opensource.google/documentation/reference/thirdparty/licenses#notice

(which fortunately? ended with the original issue not being observed)

I can observe it now with #855, it was just lost compared to massive cloning we were doing in that.

@markmandel
Copy link
Contributor

We just need to add an exception in the cargo-deny for xxhash's BSL-1.0 (Boost Software Licence), that is allowed by Google and we already do the work of including licences in distribution because it's a requirement for Apache. https://opensource.google/documentation/reference/thirdparty/licenses#notice

Just making a note for myself - if you add the exception, this will add this notice to our licence.html document - but I do need to do the work to include it in our release .zip (which is easy enough to do) -- I'll write a ticket for this now so I remember to do it before the next release. The licence.html file is already in the Docker image.

@Jake-Shadle
Copy link
Collaborator Author

Is all of that necessary? I can just take it out, I was only using it for testing purposes, and it's a dev only dependency so not part of any release artifacts.

@markmandel
Copy link
Contributor

Is all of that necessary? I can just take it out, I was only using it for testing purposes, and it's a dev only dependency so not part of any release artifacts.

We already do 95% of the work already to stay in compliance with OSS licencing - so it's not a big step to add this extra piece. We should probably ship the licence.html anyway in the zip to be honest - so it's work I'll do before our next release anyway whether you want to keep the dependency or not. Up to you 👍🏻

@XAMPPRocky
Copy link
Collaborator

Sorry, I was eager to bench with these improvements, so I pushed a commit to add the exception 😄

@XAMPPRocky XAMPPRocky force-pushed the clustermap-benchmarks branch from b3baaca to cea5c98 Compare February 26, 2024 08:44
@XAMPPRocky XAMPPRocky merged commit 1a7d3ab into main Feb 26, 2024
5 checks passed
@quilkin-bot
Copy link
Collaborator

Build Succeeded 🥳

Build Id: 15b0d34e-f7b0-400f-8d69-95ce92deaa2a

The following development images have been built, and will exist for the next 30 days:

To build this version:

git fetch [email protected]:googleforgames/quilkin.git pull/856/head:pr_856 && git checkout pr_856
cargo build

@Jake-Shadle Jake-Shadle deleted the clustermap-benchmarks branch February 26, 2024 09:36
@markmandel markmandel added kind/feature New feature or request area/performance Anything to do with Quilkin being slow, or making it go faster. area/build-tools Development tooling. labels Mar 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build-tools Development tooling. area/performance Anything to do with Quilkin being slow, or making it go faster. kind/feature New feature or request size/l
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants