Add benchmarks #58
Merged
Add benchmarks #58
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #58 +/- ##
=======================================
Coverage 70.62% 70.62%
=======================================
Files 12 12
Lines 429 429
=======================================
Hits 303 303
Misses 126 126 ☔ View full report in Codecov by Sentry. |
|
BenchmarkCI failure is normal, as they don't exist on master yet. |
|
Updated output: julia> median(results["Jacobian"]["Global"])
4-element BenchmarkTools.BenchmarkGroup:
tags: []
"sparse_matmul" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"sparsity=0.05" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(59.542 μs)
"Set{UInt64}" => TrialEstimate(258.125 μs)
"DuplicateVector{UInt64}" => TrialEstimate(108.500 μs)
"SortedVector{UInt64}" => TrialEstimate(90.000 μs)
"sparsity=0.01" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(11.500 μs)
"Set{UInt64}" => TrialEstimate(17.083 μs)
"DuplicateVector{UInt64}" => TrialEstimate(10.792 μs)
"SortedVector{UInt64}" => TrialEstimate(11.333 μs)
"sparsity=0.25" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(268.375 μs)
"Set{UInt64}" => TrialEstimate(2.485 ms)
"DuplicateVector{UInt64}" => TrialEstimate(128.338 ms)
"SortedVector{UInt64}" => TrialEstimate(688.458 μs)
"sparsity=0.1" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(101.605 μs)
"Set{UInt64}" => TrialEstimate(834.375 μs)
"DuplicateVector{UInt64}" => TrialEstimate(1.207 ms)
"SortedVector{UInt64}" => TrialEstimate(236.208 μs)
"conv" => 2-element BenchmarkTools.BenchmarkGroup:
tags: []
"size=28x28x3x1" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(11.736 ms)
"Set{UInt64}" => TrialEstimate(63.068 ms)
"DuplicateVector{UInt64}" => TrialEstimate(8.667 ms)
"SortedVector{UInt64}" => TrialEstimate(24.388 ms)
"size=128x128x3x1" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(1.977 s)
"Set{UInt64}" => TrialEstimate(1.552 s)
"DuplicateVector{UInt64}" => TrialEstimate(265.054 ms)
"SortedVector{UInt64}" => TrialEstimate(627.820 ms)
"brusselator_ode_solve" => 3-element BenchmarkTools.BenchmarkGroup:
tags: []
"N=6" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(83.771 μs)
"Set{UInt64}" => TrialEstimate(333.958 μs)
"DuplicateVector{UInt64}" => TrialEstimate(116.458 μs)
"SortedVector{UInt64}" => TrialEstimate(128.542 μs)
"N=100" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(135.492 ms)
"Set{UInt64}" => TrialEstimate(179.689 ms)
"DuplicateVector{UInt64}" => TrialEstimate(41.716 ms)
"SortedVector{UInt64}" => TrialEstimate(40.989 ms)
"N=24" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(1.531 ms)
"Set{UInt64}" => TrialEstimate(7.411 ms)
"DuplicateVector{UInt64}" => TrialEstimate(1.786 ms)
"SortedVector{UInt64}" => TrialEstimate(1.718 ms)
"brusselator" => 3-element BenchmarkTools.BenchmarkGroup:
tags: []
"N=6" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(32.083 μs)
"Set{UInt64}" => TrialEstimate(70.437 μs)
"DuplicateVector{UInt64}" => TrialEstimate(31.125 μs)
"SortedVector{UInt64}" => TrialEstimate(37.584 μs)
"N=100" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(34.251 ms)
"Set{UInt64}" => TrialEstimate(32.541 ms)
"DuplicateVector{UInt64}" => TrialEstimate(8.799 ms)
"SortedVector{UInt64}" => TrialEstimate(10.505 ms)
"N=24" => 4-element BenchmarkTools.BenchmarkGroup:
tags: []
"BitSet" => TrialEstimate(574.229 μs)
"Set{UInt64}" => TrialEstimate(1.176 ms)
"DuplicateVector{UInt64}" => TrialEstimate(488.000 μs)
"SortedVector{UInt64}" => TrialEstimate(545.958 μs) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #54.
Some results from running the benchmark suite: