Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: identifier completion benchmark #6796

Merged
merged 1 commit into from
Jan 27, 2025

Conversation

mhuisi
Copy link
Contributor

@mhuisi mhuisi commented Jan 27, 2025

Adds a basic identifier completion benchmark so that bugs like the one in #6794 are caught earlier.

@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Jan 27, 2025
@leanprover-community-bot
Copy link
Collaborator

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 3aea0fd810f404f930aa95f9666cf60940bcf64e --onto 69a73a18fbfa1fc045bfbf1c4cf93b155d4c9387. (2025-01-27 18:15:57)

@mhuisi
Copy link
Contributor Author

mhuisi commented Jan 27, 2025

!bench

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit e86681f.
There were significant changes against commit 3aea0fd:

  Benchmark                   Metric          Change
  ============================================================
+ big_omega.lean              branch-misses    -4.9% (-17.9 σ)
+ big_omega.lean MT           branch-misses    -5.1% (-10.9 σ)
+ bv_decide_inequality.lean   branch-misses    -1.2% (-25.2 σ)
+ bv_decide_mul               branch-misses    -3.1% (-44.7 σ)
+ bv_decide_realworld         branch-misses    -1.4% (-13.4 σ)
+ ilean roundtrip             branch-misses    -3.9% (-11.2 σ)
+ simp_arith1                 branch-misses    -3.2% (-23.0 σ)

@mhuisi mhuisi added this pull request to the merge queue Jan 27, 2025
Merged via the queue into leanprover:master with commit 0160aa1 Jan 27, 2025
14 checks passed
luisacicolini pushed a commit to opencompl/lean4 that referenced this pull request Jan 28, 2025
Adds a basic identifier completion benchmark so that bugs like the one
in leanprover#6794 are caught earlier.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants