Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor code to match theory #59

Merged
merged 46 commits into from
May 14, 2024
Merged

Refactor code to match theory #59

merged 46 commits into from
May 14, 2024

Conversation

adrhill
Copy link
Owner

@adrhill adrhill commented May 10, 2024

Will close #56, #57.

@adrhill adrhill added the run benchmark Run benchmarks in CI label May 10, 2024
@adrhill
Copy link
Owner Author

adrhill commented May 10, 2024

Up to 8ed3f4d, this just refactors types. Running intermediate benchmarks due to BenchmarkCI not supporting ARM CPUs.

@codecov-commenter
Copy link

codecov-commenter commented May 10, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 61.27820% with 103 lines in your changes are missing coverage. Please review.

Project coverage is 70.04%. Comparing base (5a6771f) to head (72df223).

Files Patch % Lines
src/operators.jl 33.33% 70 Missing ⚠️
src/overload_gradient.jl 45.83% 13 Missing ⚠️
src/tracers.jl 71.79% 11 Missing ⚠️
src/overload_hessian.jl 93.18% 3 Missing ⚠️
src/conversion.jl 66.66% 2 Missing ⚠️
src/pattern.jl 88.88% 2 Missing ⚠️
src/settypes/duplicatevector.jl 77.77% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main      #59      +/-   ##
==========================================
- Coverage   70.62%   70.04%   -0.59%     
==========================================
  Files          12       12              
  Lines         429      424       -5     
==========================================
- Hits          303      297       -6     
- Misses        126      127       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@adrhill adrhill mentioned this pull request May 10, 2024
Closed
2 tasks
@github-actions GitHub Actions
Copy link

Benchmark result

Judge result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmarks:
    • Target: 10 May 2024 - 18:24
    • Baseline: 10 May 2024 - 18:28
  • Package commits:
    • Target: bd4683
    • Baseline: e7342b
  • Julia commits:
    • Target: 0b4590
    • Baseline: 0b4590
  • Julia command flags:
    • Target: None
    • Baseline: None
  • Environment variables:
    • Target: None
    • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 0.78 (5%) ✅ 1.01 (1%)
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 0.92 (5%) ✅ 1.02 (1%) ❌
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 1.04 (5%) 1.03 (1%) ❌
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 1.01 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 1.07 (5%) ❌ 1.04 (1%) ❌
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 1.07 (5%) ❌ 1.00 (1%)
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 1.02 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 1.10 (5%) ❌ 1.01 (1%)
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 1.02 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 1.14 (5%) ❌ 1.05 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 1.03 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 1.05 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 1.05 (5%) ❌ 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 1.02 (5%) 1.05 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 1.04 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 1.07 (5%) ❌ 1.08 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 1.02 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 1.02 (5%) 1.05 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 1.04 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.01 (5%) 1.01 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 0.95 (5%) ✅ 1.12 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 1.28 (5%) ❌ 1.23 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 1.00 (5%) 1.07 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 1.21 (5%) ❌ 1.17 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 1.21 (5%) ❌ 1.12 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 1.11 (5%) ❌ 1.23 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 1.10 (5%) ❌ 1.07 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 1.14 (5%) ❌ 1.15 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 0.96 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 1.15 (5%) ❌ 1.11 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 1.00 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 1.10 (5%) ❌ 1.07 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 1.11 (5%) ❌ 1.05 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 1.20 (5%) ❌ 1.29 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.00 (5%) 1.03 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 1.22 (5%) ❌ 1.19 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 1.02 (5%) 0.99 (1%) ✅

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Target

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3201 MHz       1923 s          0 s        194 s       6613 s          0 s
       #2  2909 MHz       2056 s          0 s        175 s       6433 s          0 s
       #3  2445 MHz       2794 s          0 s        195 s       5633 s          0 s
       #4  3243 MHz       2040 s          0 s        187 s       6491 s          0 s
  Memory: 15.606491088867188 GB (13320.5 MB free)
  Uptime: 890.77 sec
  Load Avg:  1.01  1.1  0.83
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2445 MHz       2490 s          0 s        223 s       8428 s          0 s
       #2  3162 MHz       3759 s          0 s        222 s       7098 s          0 s
       #3  3274 MHz       2829 s          0 s        210 s       7992 s          0 s
       #4  3243 MHz       2096 s          0 s        210 s       8821 s          0 s
  Memory: 15.606491088867188 GB (13754.44140625 MB free)
  Uptime: 1132.37 sec
  Load Avg:  1.0  1.04  0.88
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 10 May 2024 - 18:24
  • Package commit: bd4683
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 45.229 ms (5%) 5.683 ms 148.11 MiB (1%) 446018
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 15.184 ms (5%) 30.66 MiB (1%) 260057
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 42.414 ms (5%) 71.25 MiB (1%) 680060
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 16.947 ms (5%) 35.39 MiB (1%) 320057
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 932.344 μs (5%) 2.22 MiB (1%) 24570
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 843.637 μs (5%) 1.80 MiB (1%) 15017
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.181 ms (5%) 4.14 MiB (1%) 39212
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 932.722 μs (5%) 2.07 MiB (1%) 18473
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 50.905 μs (5%) 112.62 KiB (1%) 1326
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 51.026 μs (5%) 115.94 KiB (1%) 965
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 114.765 μs (5%) 265.89 KiB (1%) 2480
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 59.722 μs (5%) 133.38 KiB (1%) 1181
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 226.893 ms (5%) 55.309 ms 804.86 MiB (1%) 1245764
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 77.980 ms (5%) 8.342 ms 154.56 MiB (1%) 580171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 349.585 ms (5%) 69.825 ms 364.07 MiB (1%) 2500179
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 63.546 ms (5%) 5.159 ms 130.91 MiB (1%) 800171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.581 ms (5%) 6.55 MiB (1%) 68583
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 3.076 ms (5%) 8.92 MiB (1%) 33556
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.716 ms (5%) 20.99 MiB (1%) 144156
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 2.974 ms (5%) 7.56 MiB (1%) 46228
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 139.992 μs (5%) 289.73 KiB (1%) 3305
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 186.720 μs (5%) 579.80 KiB (1%) 2223
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 717.511 μs (5%) 1.32 MiB (1%) 9143
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 199.734 μs (5%) 492.61 KiB (1%) 3015
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.853 s (5%) 341.867 ms 19.16 GiB (1%) 16415375
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 549.174 ms (5%) 63.058 ms 1.27 GiB (1%) 7310405
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 3.769 s (5%) 736.769 ms 4.67 GiB (1%) 34510815
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 1.001 s (5%) 96.383 ms 2.19 GiB (1%) 11765773
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 23.965 ms (5%) 2.088 ms 75.57 MiB (1%) 599866
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 15.500 ms (5%) 51.13 MiB (1%) 274462
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 106.681 ms (5%) 6.807 ms 182.40 MiB (1%) 1295950
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 33.741 ms (5%) 1.961 ms 86.51 MiB (1%) 441358
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 17.002 μs (5%) 34.70 KiB (1%) 427
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 17.653 μs (5%) 41.78 KiB (1%) 427
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 27.662 μs (5%) 73.36 KiB (1%) 717
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 15.008 μs (5%) 27.64 KiB (1%) 313
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 83.696 μs (5%) 173.08 KiB (1%) 1345
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 125.565 μs (5%) 240.21 KiB (1%) 1011
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 283.841 μs (5%) 486.91 KiB (1%) 3166
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 122.349 μs (5%) 246.91 KiB (1%) 1219
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 158.456 μs (5%) 361.81 KiB (1%) 2533
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 1.375 ms (5%) 5.18 MiB (1%) 1703
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.282 ms (5%) 2.23 MiB (1%) 8582
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 305.882 μs (5%) 974.98 KiB (1%) 2413
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 329.567 μs (5%) 622.59 KiB (1%) 6155
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 229.941 ms (5%) 7.076 ms 1.08 GiB (1%) 4874
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.270 ms (5%) 8.27 MiB (1%) 25274
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 823.580 μs (5%) 3.09 MiB (1%) 6097

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3201 MHz       1923 s          0 s        194 s       6613 s          0 s
       #2  2909 MHz       2056 s          0 s        175 s       6433 s          0 s
       #3  2445 MHz       2794 s          0 s        195 s       5633 s          0 s
       #4  3243 MHz       2040 s          0 s        187 s       6491 s          0 s
  Memory: 15.606491088867188 GB (13320.5 MB free)
  Uptime: 890.77 sec
  Load Avg:  1.01  1.1  0.83
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 10 May 2024 - 18:28
  • Package commit: e7342b
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 58.189 ms (5%) 15.260 ms 146.89 MiB (1%) 426016
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 14.705 ms (5%) 30.66 MiB (1%) 260056
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 46.016 ms (5%) 70.03 MiB (1%) 660056
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 16.340 ms (5%) 35.39 MiB (1%) 320056
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 899.492 μs (5%) 2.15 MiB (1%) 23416
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 809.474 μs (5%) 1.80 MiB (1%) 15016
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.164 ms (5%) 4.07 MiB (1%) 38056
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 902.248 μs (5%) 2.07 MiB (1%) 18472
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 47.580 μs (5%) 108.00 KiB (1%) 1252
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 47.820 μs (5%) 115.88 KiB (1%) 964
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 113.022 μs (5%) 261.00 KiB (1%) 2404
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 57.238 μs (5%) 133.31 KiB (1%) 1180
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 206.867 ms (5%) 52.340 ms 798.75 MiB (1%) 1145762
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 76.726 ms (5%) 7.468 ms 152.12 MiB (1%) 540171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 307.342 ms (5%) 49.108 ms 347.59 MiB (1%) 2320171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 61.907 ms (5%) 6.557 ms 128.47 MiB (1%) 760171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.451 ms (5%) 6.20 MiB (1%) 62821
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 2.927 ms (5%) 8.78 MiB (1%) 31252
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.522 ms (5%) 20.04 MiB (1%) 133780
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 2.854 ms (5%) 7.42 MiB (1%) 43924
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 130.825 μs (5%) 267.05 KiB (1%) 2943
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 183.093 μs (5%) 570.80 KiB (1%) 2079
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 700.761 μs (5%) 1.26 MiB (1%) 8487
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 191.187 μs (5%) 483.61 KiB (1%) 2871
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.841 s (5%) 317.334 ms 18.88 GiB (1%) 11691915
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 578.266 ms (5%) 96.372 ms 1.13 GiB (1%) 4973251
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 2.948 s (5%) 454.559 ms 3.80 GiB (1%) 25113047
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 1.002 s (5%) 112.872 ms 2.05 GiB (1%) 9428619
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 19.859 ms (5%) 1.746 ms 64.74 MiB (1%) 422406
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 12.828 ms (5%) 45.78 MiB (1%) 186908
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 95.899 ms (5%) 5.566 ms 148.85 MiB (1%) 943382
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 30.677 ms (5%) 1.978 ms 81.16 MiB (1%) 353804
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 14.888 μs (5%) 30.20 KiB (1%) 355
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 18.294 μs (5%) 41.12 KiB (1%) 417
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 24.155 μs (5%) 65.94 KiB (1%) 623
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 15.019 μs (5%) 26.62 KiB (1%) 297
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 75.972 μs (5%) 162.47 KiB (1%) 1253
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 113.512 μs (5%) 228.61 KiB (1%) 1007
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 276.988 μs (5%) 488.96 KiB (1%) 3176
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 101.831 μs (5%) 191.50 KiB (1%) 1227
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 163.847 μs (5%) 359.33 KiB (1%) 2459
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 1.422 ms (5%) 5.20 MiB (1%) 1695
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.277 ms (5%) 2.17 MiB (1%) 8437
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 293.469 μs (5%) 975.53 KiB (1%) 2381
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 330.137 μs (5%) 618.22 KiB (1%) 6085
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 188.462 ms (5%) 3.751 ms 929.58 MiB (1%) 4729
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.201 ms (5%) 8.36 MiB (1%) 25304
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 792.832 μs (5%) 3.09 MiB (1%) 6065

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2445 MHz       2490 s          0 s        223 s       8428 s          0 s
       #2  3162 MHz       3759 s          0 s        222 s       7098 s          0 s
       #3  3274 MHz       2829 s          0 s        210 s       7992 s          0 s
       #4  3243 MHz       2096 s          0 s        210 s       8821 s          0 s
  Memory: 15.606491088867188 GB (13754.44140625 MB free)
  Uptime: 1132.37 sec
  Load Avg:  1.0  1.04  0.88
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.86
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected
Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft

@adrhill adrhill added the run benchmark Run benchmarks in CI label May 13, 2024
@adrhill adrhill removed the run benchmark Run benchmarks in CI label May 13, 2024
@github-actions GitHub Actions
Copy link

Benchmark result

Judge result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmarks:
    • Target: 13 May 2024 - 17:08
    • Baseline: 13 May 2024 - 17:11
  • Package commits:
    • Target: 497488
    • Baseline: e7342b
  • Julia commits:
    • Target: 0b4590
    • Baseline: 0b4590
  • Julia command flags:
    • Target: None
    • Baseline: None
  • Environment variables:
    • Target: None
    • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 4.78 (5%) ❌ 2.46 (1%) ❌
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 1.06 (5%) ❌ 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 0.96 (5%) 0.98 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 4.44 (5%) ❌ 2.44 (1%) ❌
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 1.02 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 0.98 (5%) 0.97 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 4.53 (5%) ❌ 2.43 (1%) ❌
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 1.01 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 1.18 (5%) ❌ 1.00 (1%)
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 5.73 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 1.09 (5%) ❌ 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 0.66 (5%) ✅ 0.62 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 1.03 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 6.13 (5%) ❌ 3.53 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 1.04 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 0.65 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 1.04 (5%) 1.03 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 5.96 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 1.02 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 0.69 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.00 (5%) 1.01 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 2.50 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 1.13 (5%) ❌ 1.23 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 0.73 (5%) ✅ 0.59 (1%) ✅
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 1.22 (5%) ❌ 1.16 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 3.03 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 1.17 (5%) ❌ 1.22 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 0.68 (5%) ✅ 0.60 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 1.10 (5%) ❌ 1.07 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 0.80 (5%) ✅ 0.54 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 1.08 (5%) ❌ 1.10 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 1.05 (5%) ❌ 0.98 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 0.91 (5%) ✅ 0.76 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 7.04 (5%) ❌ 4.92 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 1.06 (5%) ❌ 1.00 (1%)
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 0.73 (5%) ✅ 0.68 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 1.00 (5%) 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 3.89 (5%) ❌ 2.49 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 0.99 (5%) 0.98 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 0.74 (5%) ✅ 0.68 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 1.02 (5%) 0.97 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 1.44 (5%) ❌ 0.67 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 0.75 (5%) ✅ 0.63 (1%) ✅

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Target

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2445 MHz       1985 s          0 s        182 s       6423 s          0 s
       #2  2710 MHz       2836 s          0 s        186 s       5575 s          0 s
       #3  3312 MHz       2098 s          0 s        186 s       6297 s          0 s
       #4  3242 MHz       1904 s          0 s        184 s       6500 s          0 s
  Memory: 15.606494903564453 GB (13366.60546875 MB free)
  Uptime: 863.27 sec
  Load Avg:  1.02  1.06  0.77
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2445 MHz       2454 s          0 s        219 s       8275 s          0 s
       #2  2445 MHz       3597 s          0 s        227 s       7131 s          0 s
       #3  3258 MHz       2681 s          0 s        213 s       8044 s          0 s
       #4  3243 MHz       2397 s          0 s        210 s       8338 s          0 s
  Memory: 15.606494903564453 GB (13761.09375 MB free)
  Uptime: 1099.51 sec
  Load Avg:  1.04  1.04  0.84
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 13 May 2024 - 17:8
  • Package commit: 497488
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 45.544 ms (5%) 5.975 ms 145.98 MiB (1%) 426018
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 69.520 ms (5%) 5.033 ms 75.53 MiB (1%) 740057
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 44.721 ms (5%) 69.12 MiB (1%) 660060
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 10.748 ms (5%) 23.19 MiB (1%) 200057
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 875.570 μs (5%) 2.10 MiB (1%) 23418
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 3.596 ms (5%) 4.38 MiB (1%) 42665
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.180 ms (5%) 4.01 MiB (1%) 38060
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 590.776 μs (5%) 1.37 MiB (1%) 11561
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 47.409 μs (5%) 104.75 KiB (1%) 1254
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 219.541 μs (5%) 281.31 KiB (1%) 2693
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 113.844 μs (5%) 258.02 KiB (1%) 2408
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 37.640 μs (5%) 88.38 KiB (1%) 749
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 225.583 ms (5%) 62.178 ms 800.89 MiB (1%) 1225764
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 387.169 ms (5%) 32.887 ms 531.22 MiB (1%) 3100171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 332.726 ms (5%) 57.738 ms 360.10 MiB (1%) 2480179
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 38.568 ms (5%) 3.234 ms 80.25 MiB (1%) 460171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.515 ms (5%) 6.32 MiB (1%) 67431
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 17.979 ms (5%) 31.01 MiB (1%) 178708
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.758 ms (5%) 20.76 MiB (1%) 143004
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 1.814 ms (5%) 4.64 MiB (1%) 26644
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 136.996 μs (5%) 275.11 KiB (1%) 3233
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 1.098 ms (5%) 1.95 MiB (1%) 11295
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 722.274 μs (5%) 1.31 MiB (1%) 9071
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 130.454 μs (5%) 305.86 KiB (1%) 1791
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.840 s (5%) 355.207 ms 19.15 GiB (1%) 16384623
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 1.197 s (5%) 62.326 ms 1.20 GiB (1%) 7109869
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 3.365 s (5%) 573.330 ms 4.65 GiB (1%) 34480063
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 644.449 ms (5%) 61.279 ms 1.20 GiB (1%) 7159021
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 24.097 ms (5%) 1.806 ms 74.83 MiB (1%) 598714
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 39.701 ms (5%) 48.51 MiB (1%) 267454
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 113.358 ms (5%) 8.480 ms 181.66 MiB (1%) 1294798
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 20.350 ms (5%) 48.65 MiB (1%) 269806
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 15.700 μs (5%) 31.45 KiB (1%) 375
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 13.505 μs (5%) 22.53 KiB (1%) 231
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 25.728 μs (5%) 70.23 KiB (1%) 667
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 14.367 μs (5%) 25.58 KiB (1%) 280
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 75.351 μs (5%) 126.42 KiB (1%) 1275
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 877.207 μs (5%) 1.21 MiB (1%) 7410
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 281.599 μs (5%) 475.80 KiB (1%) 3152
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 81.934 μs (5%) 165.73 KiB (1%) 753
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 157.686 μs (5%) 339.59 KiB (1%) 2469
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 5.414 ms (5%) 12.08 MiB (1%) 25427
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.286 ms (5%) 2.21 MiB (1%) 8580
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 228.008 μs (5%) 656.36 KiB (1%) 1346
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 335.239 μs (5%) 601.00 KiB (1%) 6145
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 302.557 ms (5%) 168.369 ms 693.12 MiB (1%) 29817
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.254 ms (5%) 8.27 MiB (1%) 25206
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 596.400 μs (5%) 1.95 MiB (1%) 3149

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2445 MHz       1985 s          0 s        182 s       6423 s          0 s
       #2  2710 MHz       2836 s          0 s        186 s       5575 s          0 s
       #3  3312 MHz       2098 s          0 s        186 s       6297 s          0 s
       #4  3242 MHz       1904 s          0 s        184 s       6500 s          0 s
  Memory: 15.606494903564453 GB (13366.60546875 MB free)
  Uptime: 863.27 sec
  Load Avg:  1.02  1.06  0.77
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 13 May 2024 - 17:11
  • Package commit: e7342b
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 45.760 ms (5%) 5.491 ms 146.89 MiB (1%) 426016
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 14.550 ms (5%) 30.66 MiB (1%) 260056
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 42.314 ms (5%) 70.03 MiB (1%) 660056
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 15.930 ms (5%) 35.39 MiB (1%) 320056
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 912.208 μs (5%) 2.15 MiB (1%) 23416
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 808.976 μs (5%) 1.80 MiB (1%) 15016
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.142 ms (5%) 4.07 MiB (1%) 38056
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 880.528 μs (5%) 2.07 MiB (1%) 18472
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 48.601 μs (5%) 108.00 KiB (1%) 1252
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 48.481 μs (5%) 115.88 KiB (1%) 964
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 112.832 μs (5%) 261.00 KiB (1%) 2404
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 56.315 μs (5%) 133.31 KiB (1%) 1180
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 191.876 ms (5%) 45.321 ms 798.75 MiB (1%) 1145762
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 67.515 ms (5%) 5.684 ms 152.12 MiB (1%) 540171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 304.093 ms (5%) 57.260 ms 347.59 MiB (1%) 2320171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 58.784 ms (5%) 4.954 ms 128.47 MiB (1%) 760171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.452 ms (5%) 6.20 MiB (1%) 62821
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 2.931 ms (5%) 8.78 MiB (1%) 31252
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.318 ms (5%) 20.04 MiB (1%) 133780
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 2.799 ms (5%) 7.42 MiB (1%) 43924
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 131.466 μs (5%) 267.05 KiB (1%) 2943
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 184.185 μs (5%) 570.80 KiB (1%) 2079
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 705.781 μs (5%) 1.26 MiB (1%) 8487
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 188.443 μs (5%) 483.61 KiB (1%) 2871
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.849 s (5%) 377.314 ms 18.88 GiB (1%) 11691915
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 477.883 ms (5%) 73.756 ms 1.13 GiB (1%) 4973251
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 2.971 s (5%) 474.787 ms 3.80 GiB (1%) 25113047
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 879.586 ms (5%) 81.224 ms 2.05 GiB (1%) 9428619
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 19.792 ms (5%) 1.675 ms 64.74 MiB (1%) 422406
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 13.107 ms (5%) 45.78 MiB (1%) 186908
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 97.302 ms (5%) 5.587 ms 148.85 MiB (1%) 943382
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 30.103 ms (5%) 1.585 ms 81.16 MiB (1%) 353804
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 14.237 μs (5%) 29.53 KiB (1%) 345
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 16.932 μs (5%) 41.61 KiB (1%) 421
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 23.795 μs (5%) 63.94 KiB (1%) 603
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 13.636 μs (5%) 26.06 KiB (1%) 287
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 82.394 μs (5%) 167.25 KiB (1%) 1251
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 124.673 μs (5%) 252.74 KiB (1%) 1005
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 265.417 μs (5%) 474.05 KiB (1%) 3095
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 111.528 μs (5%) 244.16 KiB (1%) 1229
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 158.357 μs (5%) 356.30 KiB (1%) 2455
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 1.392 ms (5%) 4.84 MiB (1%) 1673
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.296 ms (5%) 2.25 MiB (1%) 8608
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 306.544 μs (5%) 968.47 KiB (1%) 2393
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 327.573 μs (5%) 619.97 KiB (1%) 6111
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 210.446 ms (5%) 20.066 ms 1.01 GiB (1%) 4811
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.229 ms (5%) 8.29 MiB (1%) 25256
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 792.965 μs (5%) 3.09 MiB (1%) 6069

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2445 MHz       2454 s          0 s        219 s       8275 s          0 s
       #2  2445 MHz       3597 s          0 s        227 s       7131 s          0 s
       #3  3258 MHz       2681 s          0 s        213 s       8044 s          0 s
       #4  3243 MHz       2397 s          0 s        210 s       8338 s          0 s
  Memory: 15.606494903564453 GB (13761.09375 MB free)
  Uptime: 1099.51 sec
  Load Avg:  1.04  1.04  0.84
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.87
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected
Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft

@adrhill
Copy link
Owner Author

adrhill commented May 13, 2024

Looks like I broke DuplicateVector's performance @gdalle.

@github-actions GitHub Actions
Copy link

Benchmark result

Judge result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmarks:
    • Target: 13 May 2024 - 17:16
    • Baseline: 13 May 2024 - 17:20
  • Package commits:
    • Target: b650ee
    • Baseline: e7342b
  • Julia commits:
    • Target: 0b4590
    • Baseline: 0b4590
  • Julia command flags:
    • Target: None
    • Baseline: None
  • Environment variables:
    • Target: None
    • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 4.67 (5%) ❌ 2.46 (1%) ❌
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 1.07 (5%) ❌ 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 0.65 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 0.94 (5%) ✅ 0.98 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 4.32 (5%) ❌ 2.44 (1%) ❌
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 1.02 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 0.96 (5%) 0.97 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 4.53 (5%) ❌ 2.43 (1%) ❌
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 0.99 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 0.68 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 5.63 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 1.02 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 0.62 (5%) ✅ 0.62 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 1.01 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 5.95 (5%) ❌ 3.53 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 1.03 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 0.64 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 0.99 (5%) 1.03 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 6.07 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 1.03 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 0.68 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.02 (5%) 1.01 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 2.40 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 1.16 (5%) ❌ 1.23 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 0.74 (5%) ✅ 0.59 (1%) ✅
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 1.21 (5%) ❌ 1.16 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 3.00 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 1.17 (5%) ❌ 1.22 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 0.68 (5%) ✅ 0.60 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 1.06 (5%) ❌ 1.05 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 0.75 (5%) ✅ 0.54 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 1.04 (5%) 1.03 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 0.99 (5%) 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 0.94 (5%) ✅ 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 8.20 (5%) ❌ 5.32 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 1.00 (5%) 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 0.74 (5%) ✅ 0.70 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 0.98 (5%) 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 3.67 (5%) ❌ 2.38 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 0.74 (5%) ✅ 0.68 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 0.97 (5%) 0.97 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 1.60 (5%) ❌ 0.68 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 1.01 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 0.76 (5%) ✅ 0.63 (1%) ✅

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Target

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3263 MHz       1652 s          0 s        191 s       6365 s          0 s
       #2  3243 MHz       1335 s          0 s        158 s       6709 s          0 s
       #3  2445 MHz       2467 s          0 s        182 s       5554 s          0 s
       #4  2718 MHz       3142 s          0 s        166 s       4900 s          0 s
  Memory: 15.606494903564453 GB (13374.359375 MB free)
  Uptime: 824.28 sec
  Load Avg:  1.12  1.1  0.78
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3244 MHz       1863 s          0 s        212 s       8450 s          0 s
       #2  2445 MHz       1570 s          0 s        185 s       8766 s          0 s
       #3  2445 MHz       2980 s          0 s        221 s       7320 s          0 s
       #4  3269 MHz       4444 s          0 s        195 s       5890 s          0 s
  Memory: 15.606494903564453 GB (13710.4296875 MB free)
  Uptime: 1056.62 sec
  Load Avg:  1.08  1.07  0.85
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 13 May 2024 - 17:16
  • Package commit: b650ee
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 49.380 ms (5%) 12.626 ms 145.98 MiB (1%) 426018
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 66.509 ms (5%) 4.324 ms 75.53 MiB (1%) 740057
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 43.896 ms (5%) 69.12 MiB (1%) 660060
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 10.564 ms (5%) 23.19 MiB (1%) 200057
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 878.218 μs (5%) 2.10 MiB (1%) 23418
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 3.506 ms (5%) 4.38 MiB (1%) 42665
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.197 ms (5%) 4.01 MiB (1%) 38060
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 589.710 μs (5%) 1.37 MiB (1%) 11561
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 46.737 μs (5%) 104.75 KiB (1%) 1254
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 214.990 μs (5%) 281.31 KiB (1%) 2693
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 110.686 μs (5%) 258.02 KiB (1%) 2408
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 37.621 μs (5%) 88.38 KiB (1%) 749
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 196.351 ms (5%) 46.136 ms 800.89 MiB (1%) 1225764
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 372.830 ms (5%) 30.193 ms 531.22 MiB (1%) 3100171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 304.553 ms (5%) 45.889 ms 360.10 MiB (1%) 2480179
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 36.439 ms (5%) 3.059 ms 80.25 MiB (1%) 460171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.485 ms (5%) 6.32 MiB (1%) 67431
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 17.427 ms (5%) 31.01 MiB (1%) 178708
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.810 ms (5%) 20.76 MiB (1%) 143004
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 1.814 ms (5%) 4.64 MiB (1%) 26644
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 134.150 μs (5%) 275.11 KiB (1%) 3233
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 1.068 ms (5%) 1.95 MiB (1%) 11295
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 718.369 μs (5%) 1.31 MiB (1%) 9071
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 129.131 μs (5%) 305.86 KiB (1%) 1791
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.722 s (5%) 302.980 ms 19.15 GiB (1%) 16384623
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 1.143 s (5%) 56.950 ms 1.20 GiB (1%) 7109869
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 3.288 s (5%) 525.200 ms 4.65 GiB (1%) 34480063
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 637.910 ms (5%) 60.611 ms 1.20 GiB (1%) 7159021
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 23.688 ms (5%) 1.962 ms 74.83 MiB (1%) 598714
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 38.250 ms (5%) 48.51 MiB (1%) 267454
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 108.240 ms (5%) 7.016 ms 181.66 MiB (1%) 1294798
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 20.266 ms (5%) 48.65 MiB (1%) 269806
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 15.599 μs (5%) 32.11 KiB (1%) 385
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 12.774 μs (5%) 22.23 KiB (1%) 227
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 24.316 μs (5%) 67.83 KiB (1%) 643
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 14.537 μs (5%) 25.72 KiB (1%) 281
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 74.649 μs (5%) 157.52 KiB (1%) 1275
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 965.311 μs (5%) 1.27 MiB (1%) 7617
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 271.697 μs (5%) 469.16 KiB (1%) 3109
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 81.753 μs (5%) 167.39 KiB (1%) 755
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 155.961 μs (5%) 339.17 KiB (1%) 2489
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 5.046 ms (5%) 11.99 MiB (1%) 25032
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.299 ms (5%) 2.22 MiB (1%) 8642
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 219.118 μs (5%) 652.56 KiB (1%) 1349
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 327.712 μs (5%) 598.12 KiB (1%) 6101
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 316.277 ms (5%) 159.426 ms 714.14 MiB (1%) 29854
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.175 ms (5%) 8.20 MiB (1%) 25147
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 603.566 μs (5%) 1.96 MiB (1%) 3145

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3263 MHz       1652 s          0 s        191 s       6365 s          0 s
       #2  3243 MHz       1335 s          0 s        158 s       6709 s          0 s
       #3  2445 MHz       2467 s          0 s        182 s       5554 s          0 s
       #4  2718 MHz       3142 s          0 s        166 s       4900 s          0 s
  Memory: 15.606494903564453 GB (13374.359375 MB free)
  Uptime: 824.28 sec
  Load Avg:  1.12  1.1  0.78
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 13 May 2024 - 17:20
  • Package commit: e7342b
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 47.860 ms (5%) 7.075 ms 146.89 MiB (1%) 426016
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 14.250 ms (5%) 30.66 MiB (1%) 260056
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 41.084 ms (5%) 70.03 MiB (1%) 660056
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 16.191 ms (5%) 35.39 MiB (1%) 320056
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 929.364 μs (5%) 2.15 MiB (1%) 23416
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 812.085 μs (5%) 1.80 MiB (1%) 15016
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.148 ms (5%) 4.07 MiB (1%) 38056
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 882.807 μs (5%) 2.07 MiB (1%) 18472
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 48.671 μs (5%) 108.00 KiB (1%) 1252
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 47.508 μs (5%) 115.88 KiB (1%) 964
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 111.277 μs (5%) 261.00 KiB (1%) 2404
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 55.414 μs (5%) 133.31 KiB (1%) 1180
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 194.867 ms (5%) 46.258 ms 798.75 MiB (1%) 1145762
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 66.239 ms (5%) 5.388 ms 152.12 MiB (1%) 540171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 298.050 ms (5%) 52.341 ms 347.59 MiB (1%) 2320171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 58.710 ms (5%) 4.568 ms 128.47 MiB (1%) 760171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.466 ms (5%) 6.20 MiB (1%) 62821
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 2.929 ms (5%) 8.78 MiB (1%) 31252
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.496 ms (5%) 20.04 MiB (1%) 133780
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 2.841 ms (5%) 7.42 MiB (1%) 43924
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 135.583 μs (5%) 267.05 KiB (1%) 2943
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 175.998 μs (5%) 570.80 KiB (1%) 2079
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 696.670 μs (5%) 1.26 MiB (1%) 8487
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 189.353 μs (5%) 483.61 KiB (1%) 2871
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.685 s (5%) 305.393 ms 18.88 GiB (1%) 11691915
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 475.734 ms (5%) 57.297 ms 1.13 GiB (1%) 4973251
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 2.840 s (5%) 431.019 ms 3.80 GiB (1%) 25113047
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 859.715 ms (5%) 73.467 ms 2.05 GiB (1%) 9428619
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 19.529 ms (5%) 1.407 ms 64.74 MiB (1%) 422406
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 12.756 ms (5%) 45.78 MiB (1%) 186908
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 92.898 ms (5%) 6.591 ms 148.85 MiB (1%) 943382
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 29.752 ms (5%) 1.649 ms 81.16 MiB (1%) 353804
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 14.688 μs (5%) 30.47 KiB (1%) 359
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 17.072 μs (5%) 40.89 KiB (1%) 414
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 23.354 μs (5%) 65.94 KiB (1%) 623
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 14.617 μs (5%) 27.08 KiB (1%) 305
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 79.438 μs (5%) 166.36 KiB (1%) 1275
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 117.709 μs (5%) 244.32 KiB (1%) 991
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 270.715 μs (5%) 493.89 KiB (1%) 3177
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 110.085 μs (5%) 238.64 KiB (1%) 1229
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 159.428 μs (5%) 356.92 KiB (1%) 2445
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 1.374 ms (5%) 5.04 MiB (1%) 1696
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.277 ms (5%) 2.22 MiB (1%) 8521
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 297.284 μs (5%) 963.89 KiB (1%) 2399
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 337.099 μs (5%) 616.72 KiB (1%) 6061
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 197.924 ms (5%) 4.221 ms 1.02 GiB (1%) 4806
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.120 ms (5%) 8.32 MiB (1%) 25329
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 791.416 μs (5%) 3.09 MiB (1%) 6075

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3244 MHz       1863 s          0 s        212 s       8450 s          0 s
       #2  2445 MHz       1570 s          0 s        185 s       8766 s          0 s
       #3  2445 MHz       2980 s          0 s        221 s       7320 s          0 s
       #4  3269 MHz       4444 s          0 s        195 s       5890 s          0 s
  Memory: 15.606494903564453 GB (13710.4296875 MB free)
  Uptime: 1056.62 sec
  Load Avg:  1.08  1.07  0.85
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.85
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected
Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft

@adrhill adrhill mentioned this pull request May 14, 2024
Closed
gdalle
gdalle approved these changes May 14, 2024
View reviewed changes
Copy link
Collaborator

@gdalle gdalle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Let's get this merged, figure out local sparsity and then split hairs on performance

src/operators.jl Show resolved Hide resolved
src/overload_hessian.jl Show resolved Hide resolved
src/tracers.jl Show resolved Hide resolved
src/tracers.jl Outdated Show resolved Hide resolved
src/tracers.jl Outdated Show resolved Hide resolved
@adrhill adrhill added the run benchmark Run benchmarks in CI label May 14, 2024
@github-actions GitHub Actions
Copy link

Benchmark result

Judge result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmarks:
    • Target: 14 May 2024 - 16:17
    • Baseline: 14 May 2024 - 16:21
  • Package commits:
    • Target: 87d50e
    • Baseline: 5a6771
  • Julia commits:
    • Target: 0b4590
    • Baseline: 0b4590
  • Julia command flags:
    • Target: None
    • Baseline: None
  • Environment variables:
    • Target: None
    • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 5.15 (5%) ❌ 2.46 (1%) ❌
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 1.03 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 0.65 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 0.98 (5%) 0.98 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 4.89 (5%) ❌ 2.44 (1%) ❌
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 1.00 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 1.01 (5%) 0.97 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 5.09 (5%) ❌ 2.43 (1%) ❌
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 0.98 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 1.10 (5%) ❌ 1.00 (1%)
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 6.48 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 1.04 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 0.63 (5%) ✅ 0.62 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 1.04 (5%) 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 6.46 (5%) ❌ 3.53 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 1.01 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 0.64 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 1.06 (5%) ❌ 1.03 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 6.60 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 1.02 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 0.69 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.05 (5%) ❌ 1.01 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 2.86 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 1.19 (5%) ❌ 1.23 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 0.73 (5%) ✅ 0.59 (1%) ✅
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 1.35 (5%) ❌ 1.16 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 3.20 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 1.15 (5%) ❌ 1.22 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 0.69 (5%) ✅ 0.60 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 1.00 (5%) 1.03 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 0.74 (5%) ✅ 0.53 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 1.03 (5%) 1.10 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 0.96 (5%) 0.93 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 1.02 (5%) 0.96 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 8.63 (5%) ❌ 5.50 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 1.12 (5%) ❌ 1.11 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 0.75 (5%) ✅ 0.70 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 1.03 (5%) 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 3.85 (5%) ❌ 2.40 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 0.75 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 1.06 (5%) ❌ 0.96 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 0.93 (5%) ✅ 0.74 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 0.76 (5%) ✅ 0.64 (1%) ✅

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Target

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3243 MHz       1599 s          0 s        156 s       7280 s          0 s
       #2  2445 MHz       2358 s          0 s        153 s       6523 s          0 s
       #3  2728 MHz       2404 s          0 s        206 s       6420 s          0 s
       #4  2621 MHz       2228 s          0 s        178 s       6615 s          0 s
  Memory: 15.606491088867188 GB (13306.328125 MB free)
  Uptime: 907.45 sec
  Load Avg:  1.08  1.08  0.77
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2594 MHz       1893 s          0 s        183 s       9282 s          0 s
       #2  2445 MHz       3317 s          0 s        201 s       7840 s          0 s
       #3  3246 MHz       3105 s          0 s        230 s       8020 s          0 s
       #4  3243 MHz       2542 s          0 s        193 s       8608 s          0 s
  Memory: 15.606491088867188 GB (13703.51953125 MB free)
  Uptime: 1140.25 sec
  Load Avg:  1.0  1.04  0.84
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 14 May 2024 - 16:17
  • Package commit: 87d50e
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 44.154 ms (5%) 6.007 ms 145.98 MiB (1%) 426018
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 74.716 ms (5%) 4.512 ms 75.53 MiB (1%) 740057
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 43.422 ms (5%) 69.12 MiB (1%) 660060
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 10.475 ms (5%) 23.19 MiB (1%) 200057
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 889.724 μs (5%) 2.10 MiB (1%) 23418
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 3.957 ms (5%) 4.38 MiB (1%) 42665
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.139 ms (5%) 4.01 MiB (1%) 38060
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 584.203 μs (5%) 1.37 MiB (1%) 11561
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 47.789 μs (5%) 104.75 KiB (1%) 1254
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 244.547 μs (5%) 281.31 KiB (1%) 2693
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 110.607 μs (5%) 258.02 KiB (1%) 2408
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 37.890 μs (5%) 88.38 KiB (1%) 749
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 210.517 ms (5%) 55.579 ms 800.89 MiB (1%) 1225764
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 411.441 ms (5%) 35.480 ms 531.22 MiB (1%) 3100171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 309.285 ms (5%) 49.802 ms 360.10 MiB (1%) 2480179
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 36.586 ms (5%) 3.217 ms 80.25 MiB (1%) 460171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.532 ms (5%) 6.32 MiB (1%) 67431
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 18.938 ms (5%) 31.01 MiB (1%) 178708
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.734 ms (5%) 20.76 MiB (1%) 143004
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 1.798 ms (5%) 4.64 MiB (1%) 26644
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 137.417 μs (5%) 275.11 KiB (1%) 3233
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 1.163 ms (5%) 1.95 MiB (1%) 11295
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 716.620 μs (5%) 1.31 MiB (1%) 9071
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 127.659 μs (5%) 305.86 KiB (1%) 1791
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.751 s (5%) 333.073 ms 19.15 GiB (1%) 16384623
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 1.209 s (5%) 63.131 ms 1.20 GiB (1%) 7109869
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 3.405 s (5%) 619.226 ms 4.65 GiB (1%) 34480063
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 639.276 ms (5%) 59.454 ms 1.20 GiB (1%) 7159021
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 24.066 ms (5%) 2.414 ms 74.83 MiB (1%) 598714
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 40.604 ms (5%) 48.51 MiB (1%) 267454
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 108.931 ms (5%) 9.483 ms 181.66 MiB (1%) 1294798
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 20.269 ms (5%) 48.65 MiB (1%) 269806
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 14.888 μs (5%) 30.66 KiB (1%) 363
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 12.874 μs (5%) 21.97 KiB (1%) 224
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 24.937 μs (5%) 71.42 KiB (1%) 679
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 13.936 μs (5%) 25.27 KiB (1%) 276
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 81.052 μs (5%) 160.86 KiB (1%) 1271
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 998.176 μs (5%) 1.29 MiB (1%) 7662
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 280.564 μs (5%) 486.40 KiB (1%) 3144
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 80.751 μs (5%) 168.42 KiB (1%) 748
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 164.908 μs (5%) 339.39 KiB (1%) 2477
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 5.494 ms (5%) 11.89 MiB (1%) 24968
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.278 ms (5%) 2.18 MiB (1%) 8616
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 217.597 μs (5%) 655.62 KiB (1%) 1346
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 350.636 μs (5%) 598.50 KiB (1%) 6107
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 163.344 ms (5%) 4.048 ms 731.48 MiB (1%) 29879
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.294 ms (5%) 8.30 MiB (1%) 25287
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 596.907 μs (5%) 1.98 MiB (1%) 3181

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3243 MHz       1599 s          0 s        156 s       7280 s          0 s
       #2  2445 MHz       2358 s          0 s        153 s       6523 s          0 s
       #3  2728 MHz       2404 s          0 s        206 s       6420 s          0 s
       #4  2621 MHz       2228 s          0 s        178 s       6615 s          0 s
  Memory: 15.606491088867188 GB (13306.328125 MB free)
  Uptime: 907.45 sec
  Load Avg:  1.08  1.08  0.77
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 14 May 2024 - 16:21
  • Package commit: 5a6771
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 44.909 ms (5%) 5.817 ms 146.89 MiB (1%) 426016
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 14.515 ms (5%) 30.66 MiB (1%) 260056
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 42.162 ms (5%) 70.03 MiB (1%) 660056
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 16.036 ms (5%) 35.39 MiB (1%) 320056
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 903.459 μs (5%) 2.15 MiB (1%) 23416
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 809.013 μs (5%) 1.80 MiB (1%) 15016
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.149 ms (5%) 4.07 MiB (1%) 38056
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 877.360 μs (5%) 2.07 MiB (1%) 18472
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 47.519 μs (5%) 108.00 KiB (1%) 1252
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 48.070 μs (5%) 115.88 KiB (1%) 964
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 113.171 μs (5%) 261.00 KiB (1%) 2404
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 56.826 μs (5%) 133.31 KiB (1%) 1180
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 190.837 ms (5%) 52.291 ms 798.75 MiB (1%) 1145762
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 63.496 ms (5%) 4.949 ms 152.12 MiB (1%) 540171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 296.445 ms (5%) 44.390 ms 347.59 MiB (1%) 2320171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 58.091 ms (5%) 4.877 ms 128.47 MiB (1%) 760171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.436 ms (5%) 6.20 MiB (1%) 62821
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 2.932 ms (5%) 8.78 MiB (1%) 31252
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.564 ms (5%) 20.04 MiB (1%) 133780
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 2.822 ms (5%) 7.42 MiB (1%) 43924
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 129.933 μs (5%) 267.05 KiB (1%) 2943
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 176.219 μs (5%) 570.80 KiB (1%) 2079
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 699.368 μs (5%) 1.26 MiB (1%) 8487
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 185.487 μs (5%) 483.61 KiB (1%) 2871
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.668 s (5%) 288.891 ms 18.88 GiB (1%) 11691915
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 421.859 ms (5%) 45.289 ms 1.13 GiB (1%) 4973251
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 2.852 s (5%) 440.224 ms 3.80 GiB (1%) 25113047
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 872.196 ms (5%) 77.284 ms 2.05 GiB (1%) 9428619
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 17.812 ms (5%) 64.74 MiB (1%) 422406
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 12.680 ms (5%) 45.78 MiB (1%) 186908
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 94.701 ms (5%) 5.287 ms 148.85 MiB (1%) 943382
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 29.417 ms (5%) 1.499 ms 81.16 MiB (1%) 353804
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 14.887 μs (5%) 29.81 KiB (1%) 349
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 17.393 μs (5%) 41.30 KiB (1%) 419
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 24.205 μs (5%) 65.12 KiB (1%) 615
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 14.577 μs (5%) 27.12 KiB (1%) 305
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 79.729 μs (5%) 168.34 KiB (1%) 1291
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 115.676 μs (5%) 240.62 KiB (1%) 1001
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 250.989 μs (5%) 437.56 KiB (1%) 2881
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 107.521 μs (5%) 239.14 KiB (1%) 1225
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 160.710 μs (5%) 358.00 KiB (1%) 2423
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 1.428 ms (5%) 4.96 MiB (1%) 1688
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.265 ms (5%) 2.18 MiB (1%) 8491
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 291.516 μs (5%) 988.52 KiB (1%) 2417
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 330.458 μs (5%) 620.22 KiB (1%) 6115
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 175.016 ms (5%) 2.985 ms 994.88 MiB (1%) 4741
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.145 ms (5%) 8.28 MiB (1%) 25257
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 783.385 μs (5%) 3.09 MiB (1%) 6071

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2594 MHz       1893 s          0 s        183 s       9282 s          0 s
       #2  2445 MHz       3317 s          0 s        201 s       7840 s          0 s
       #3  3246 MHz       3105 s          0 s        230 s       8020 s          0 s
       #4  3243 MHz       2542 s          0 s        193 s       8608 s          0 s
  Memory: 15.606491088867188 GB (13703.51953125 MB free)
  Uptime: 1140.25 sec
  Load Avg:  1.0  1.04  0.84
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.86
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected
Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft

@github-actions GitHub Actions
Copy link

Benchmark result

Judge result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmarks:
    • Target: 14 May 2024 - 16:23
    • Baseline: 14 May 2024 - 16:27
  • Package commits:
    • Target: 6ae8d4
    • Baseline: 5a6771
  • Julia commits:
    • Target: 0b4590
    • Baseline: 0b4590
  • Julia command flags:
    • Target: None
    • Baseline: None
  • Environment variables:
    • Target: None
    • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 4.47 (5%) ❌ 2.46 (1%) ❌
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 1.01 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 0.62 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 0.97 (5%) 0.98 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 4.22 (5%) ❌ 2.44 (1%) ❌
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 1.00 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 0.63 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 0.98 (5%) 0.97 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 4.35 (5%) ❌ 2.43 (1%) ❌
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 0.99 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 0.65 (5%) ✅ 0.66 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 1.11 (5%) ❌ 1.00 (1%)
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 5.90 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 1.08 (5%) ❌ 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 0.62 (5%) ✅ 0.62 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 1.05 (5%) ❌ 1.02 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 6.05 (5%) ❌ 3.53 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 1.04 (5%) 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 0.62 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 1.04 (5%) 1.03 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 6.09 (5%) ❌ 3.49 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 1.06 (5%) ❌ 1.04 (1%) ❌
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.63 (1%) ✅
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.15 (5%) ❌ 1.01 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 2.63 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 1.22 (5%) ❌ 1.23 (1%) ❌
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 0.75 (5%) ✅ 0.59 (1%) ✅
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 1.34 (5%) ❌ 1.16 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 3.24 (5%) ❌ 1.06 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 1.22 (5%) ❌ 1.22 (1%) ❌
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 0.67 (5%) ✅ 0.60 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 1.08 (5%) ❌ 1.03 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 0.79 (5%) ✅ 0.54 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 1.10 (5%) ❌ 1.08 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 0.97 (5%) 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 0.97 (5%) 0.96 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 7.44 (5%) ❌ 5.59 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 0.97 (5%) 0.94 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 0.71 (5%) ✅ 0.67 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 0.99 (5%) 0.95 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 4.02 (5%) ❌ 2.34 (1%) ❌
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.00 (5%) 0.99 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 0.77 (5%) ✅ 0.67 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 0.99 (5%) 0.96 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 1.80 (5%) ❌ 0.79 (1%) ✅
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 0.77 (5%) ✅ 0.64 (1%) ✅

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Target

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3243 MHz       1958 s          0 s        149 s       5609 s          0 s
       #2  3240 MHz       3616 s          0 s        211 s       3888 s          0 s
       #3  2445 MHz       2011 s          0 s        175 s       5526 s          0 s
       #4  3243 MHz       1236 s          0 s        177 s       6293 s          0 s
  Memory: 15.606494903564453 GB (13318.40625 MB free)
  Uptime: 775.23 sec
  Load Avg:  1.0  1.05  0.77
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2627 MHz       2299 s          0 s        164 s       7617 s          0 s
       #2  2634 MHz       4039 s          0 s        235 s       5806 s          0 s
       #3  2445 MHz       2922 s          0 s        218 s       6939 s          0 s
       #4  3243 MHz       1874 s          0 s        207 s       7991 s          0 s
  Memory: 15.606494903564453 GB (13699.203125 MB free)
  Uptime: 1012.24 sec
  Load Avg:  1.04  1.05  0.85
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 14 May 2024 - 16:23
  • Package commit: 6ae8d4
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 45.752 ms (5%) 5.557 ms 145.98 MiB (1%) 426018
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 65.804 ms (5%) 4.532 ms 75.53 MiB (1%) 740057
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 42.727 ms (5%) 69.12 MiB (1%) 660060
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 10.620 ms (5%) 23.19 MiB (1%) 200057
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 895.459 μs (5%) 2.10 MiB (1%) 23418
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 3.487 ms (5%) 4.38 MiB (1%) 42665
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.140 ms (5%) 4.01 MiB (1%) 38060
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 592.463 μs (5%) 1.37 MiB (1%) 11561
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 47.859 μs (5%) 104.75 KiB (1%) 1254
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 212.396 μs (5%) 281.31 KiB (1%) 2693
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 111.679 μs (5%) 258.02 KiB (1%) 2408
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 38.392 μs (5%) 88.38 KiB (1%) 749
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 234.268 ms (5%) 58.673 ms 800.89 MiB (1%) 1225764
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 388.180 ms (5%) 36.431 ms 531.22 MiB (1%) 3100171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 325.405 ms (5%) 55.926 ms 360.10 MiB (1%) 2480179
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 37.706 ms (5%) 3.695 ms 80.25 MiB (1%) 460171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.582 ms (5%) 6.32 MiB (1%) 67431
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 17.659 ms (5%) 31.01 MiB (1%) 178708
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.892 ms (5%) 20.76 MiB (1%) 143004
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 1.830 ms (5%) 4.64 MiB (1%) 26644
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 139.640 μs (5%) 275.11 KiB (1%) 3233
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 1.088 ms (5%) 1.95 MiB (1%) 11295
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 731.123 μs (5%) 1.31 MiB (1%) 9071
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 133.018 μs (5%) 305.86 KiB (1%) 1791
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 2.012 s (5%) 418.030 ms 19.15 GiB (1%) 16384623
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 1.253 s (5%) 78.699 ms 1.20 GiB (1%) 7109869
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 3.485 s (5%) 632.225 ms 4.65 GiB (1%) 34480063
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 678.892 ms (5%) 72.581 ms 1.20 GiB (1%) 7159021
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 26.377 ms (5%) 2.386 ms 74.83 MiB (1%) 598714
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 41.031 ms (5%) 48.51 MiB (1%) 267454
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 113.245 ms (5%) 8.526 ms 181.66 MiB (1%) 1294798
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 20.971 ms (5%) 48.65 MiB (1%) 269806
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 15.529 μs (5%) 31.44 KiB (1%) 375
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 13.235 μs (5%) 22.39 KiB (1%) 228
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 25.818 μs (5%) 70.88 KiB (1%) 669
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 14.296 μs (5%) 25.19 KiB (1%) 275
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 80.049 μs (5%) 161.34 KiB (1%) 1289
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 919.764 μs (5%) 1.32 MiB (1%) 7491
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 275.494 μs (5%) 466.91 KiB (1%) 3087
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 79.728 μs (5%) 163.28 KiB (1%) 759
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 159.998 μs (5%) 341.41 KiB (1%) 2495
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 5.526 ms (5%) 11.86 MiB (1%) 24989
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.244 ms (5%) 2.09 MiB (1%) 8245
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 235.789 μs (5%) 651.61 KiB (1%) 1354
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 331.097 μs (5%) 596.06 KiB (1%) 6069
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 330.179 ms (5%) 150.669 ms 790.49 MiB (1%) 29910
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.324 ms (5%) 8.26 MiB (1%) 25330
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 637.989 μs (5%) 1.97 MiB (1%) 3171

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  3243 MHz       1958 s          0 s        149 s       5609 s          0 s
       #2  3240 MHz       3616 s          0 s        211 s       3888 s          0 s
       #3  2445 MHz       2011 s          0 s        175 s       5526 s          0 s
       #4  3243 MHz       1236 s          0 s        177 s       6293 s          0 s
  Memory: 15.606494903564453 GB (13318.40625 MB free)
  Uptime: 775.23 sec
  Load Avg:  1.0  1.05  0.77
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/SparseConnectivityTracer.jl/SparseConnectivityTracer.jl

Job Properties

  • Time of benchmark: 14 May 2024 - 16:27
  • Package commit: 5a6771
  • Julia commit: 0b4590
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["Jacobian", "Global", "brusselator", "N=100", "BitSet"] 46.135 ms (5%) 5.541 ms 146.89 MiB (1%) 426016
["Jacobian", "Global", "brusselator", "N=100", "DuplicateVector{UInt64}"] 14.715 ms (5%) 30.66 MiB (1%) 260056
["Jacobian", "Global", "brusselator", "N=100", "Set{UInt64}"] 42.194 ms (5%) 70.03 MiB (1%) 660056
["Jacobian", "Global", "brusselator", "N=100", "SortedVector{UInt64}"] 16.996 ms (5%) 35.39 MiB (1%) 320056
["Jacobian", "Global", "brusselator", "N=24", "BitSet"] 924.592 μs (5%) 2.15 MiB (1%) 23416
["Jacobian", "Global", "brusselator", "N=24", "DuplicateVector{UInt64}"] 826.600 μs (5%) 1.80 MiB (1%) 15016
["Jacobian", "Global", "brusselator", "N=24", "Set{UInt64}"] 2.145 ms (5%) 4.07 MiB (1%) 38056
["Jacobian", "Global", "brusselator", "N=24", "SortedVector{UInt64}"] 935.844 μs (5%) 2.07 MiB (1%) 18472
["Jacobian", "Global", "brusselator", "N=6", "BitSet"] 48.961 μs (5%) 108.00 KiB (1%) 1252
["Jacobian", "Global", "brusselator", "N=6", "DuplicateVector{UInt64}"] 48.811 μs (5%) 115.88 KiB (1%) 964
["Jacobian", "Global", "brusselator", "N=6", "Set{UInt64}"] 112.279 μs (5%) 261.00 KiB (1%) 2404
["Jacobian", "Global", "brusselator", "N=6", "SortedVector{UInt64}"] 59.150 μs (5%) 133.31 KiB (1%) 1180
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "BitSet"] 210.869 ms (5%) 53.356 ms 798.75 MiB (1%) 1145762
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "DuplicateVector{UInt64}"] 65.762 ms (5%) 5.240 ms 152.12 MiB (1%) 540171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "Set{UInt64}"] 301.174 ms (5%) 48.629 ms 347.59 MiB (1%) 2320171
["Jacobian", "Global", "brusselator_ode_solve", "N=100", "SortedVector{UInt64}"] 60.721 ms (5%) 7.169 ms 128.47 MiB (1%) 760171
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "BitSet"] 2.455 ms (5%) 6.20 MiB (1%) 62821
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "DuplicateVector{UInt64}"] 2.918 ms (5%) 8.78 MiB (1%) 31252
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "Set{UInt64}"] 12.342 ms (5%) 20.04 MiB (1%) 133780
["Jacobian", "Global", "brusselator_ode_solve", "N=24", "SortedVector{UInt64}"] 2.942 ms (5%) 7.42 MiB (1%) 43924
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "BitSet"] 134.300 μs (5%) 267.05 KiB (1%) 2943
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "DuplicateVector{UInt64}"] 178.743 μs (5%) 570.80 KiB (1%) 2079
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "Set{UInt64}"] 692.370 μs (5%) 1.26 MiB (1%) 8487
["Jacobian", "Global", "brusselator_ode_solve", "N=6", "SortedVector{UInt64}"] 198.590 μs (5%) 483.61 KiB (1%) 2871
["Jacobian", "Global", "conv", "size=128x128x3", "BitSet"] 1.756 s (5%) 299.708 ms 18.88 GiB (1%) 11691915
["Jacobian", "Global", "conv", "size=128x128x3", "DuplicateVector{UInt64}"] 476.749 ms (5%) 71.127 ms 1.13 GiB (1%) 4973251
["Jacobian", "Global", "conv", "size=128x128x3", "Set{UInt64}"] 2.867 s (5%) 447.266 ms 3.80 GiB (1%) 25113047
["Jacobian", "Global", "conv", "size=128x128x3", "SortedVector{UInt64}"] 901.165 ms (5%) 96.897 ms 2.05 GiB (1%) 9428619
["Jacobian", "Global", "conv", "size=28x28x3", "BitSet"] 19.612 ms (5%) 1.449 ms 64.74 MiB (1%) 422406
["Jacobian", "Global", "conv", "size=28x28x3", "DuplicateVector{UInt64}"] 12.668 ms (5%) 45.78 MiB (1%) 186908
["Jacobian", "Global", "conv", "size=28x28x3", "Set{UInt64}"] 93.121 ms (5%) 6.723 ms 148.85 MiB (1%) 943382
["Jacobian", "Global", "conv", "size=28x28x3", "SortedVector{UInt64}"] 31.470 ms (5%) 1.859 ms 81.16 MiB (1%) 353804
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "BitSet"] 14.387 μs (5%) 30.47 KiB (1%) 359
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "DuplicateVector{UInt64}"] 16.761 μs (5%) 41.42 KiB (1%) 421
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "Set{UInt64}"] 23.554 μs (5%) 65.92 KiB (1%) 623
["Jacobian", "Global", "sparse_matmul", "sparsity=0.01", "SortedVector{UInt64}"] 14.697 μs (5%) 26.44 KiB (1%) 291
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "BitSet"] 82.173 μs (5%) 167.62 KiB (1%) 1273
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "DuplicateVector{UInt64}"] 123.630 μs (5%) 241.95 KiB (1%) 1012
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "Set{UInt64}"] 284.439 μs (5%) 497.70 KiB (1%) 3141
["Jacobian", "Global", "sparse_matmul", "sparsity=0.05", "SortedVector{UInt64}"] 113.071 μs (5%) 244.84 KiB (1%) 1233
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "BitSet"] 162.283 μs (5%) 358.14 KiB (1%) 2429
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "DuplicateVector{UInt64}"] 1.376 ms (5%) 5.08 MiB (1%) 1706
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "Set{UInt64}"] 1.239 ms (5%) 2.12 MiB (1%) 8492
["Jacobian", "Global", "sparse_matmul", "sparsity=0.1", "SortedVector{UInt64}"] 307.172 μs (5%) 977.58 KiB (1%) 2389
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "BitSet"] 335.384 μs (5%) 617.72 KiB (1%) 6077
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "DuplicateVector{UInt64}"] 183.163 ms (5%) 4.179 ms 1006.17 MiB (1%) 4758
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "Set{UInt64}"] 4.251 ms (5%) 8.27 MiB (1%) 25046
["Jacobian", "Global", "sparse_matmul", "sparsity=0.25", "SortedVector{UInt64}"] 827.672 μs (5%) 3.10 MiB (1%) 6073

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["Jacobian", "Global", "brusselator", "N=100"]
  • ["Jacobian", "Global", "brusselator", "N=24"]
  • ["Jacobian", "Global", "brusselator", "N=6"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=100"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=24"]
  • ["Jacobian", "Global", "brusselator_ode_solve", "N=6"]
  • ["Jacobian", "Global", "conv", "size=128x128x3"]
  • ["Jacobian", "Global", "conv", "size=28x28x3"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.01"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.05"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.1"]
  • ["Jacobian", "Global", "sparse_matmul", "sparsity=0.25"]

Julia versioninfo

Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.4 LTS
  uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1  2627 MHz       2299 s          0 s        164 s       7617 s          0 s
       #2  2634 MHz       4039 s          0 s        235 s       5806 s          0 s
       #3  2445 MHz       2922 s          0 s        218 s       6939 s          0 s
       #4  3243 MHz       1874 s          0 s        207 s       7991 s          0 s
  Memory: 15.606494903564453 GB (13699.203125 MB free)
  Uptime: 1012.24 sec
  Load Avg:  1.04  1.05  0.85
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.85
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected
Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft

@adrhill adrhill mentioned this pull request May 14, 2024
Closed
@adrhill adrhill merged commit 6750c46 into main May 14, 2024
3 checks passed
@adrhill adrhill deleted the ah/theory-refactor branch May 14, 2024 16:36
@adrhill adrhill mentioned this pull request May 14, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gdalle gdalle gdalle approved these changes

Assignees
No one assigned
Labels
run benchmark Run benchmarks in CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add local tracers
3 participants
@adrhill @codecov-commenter @gdalle