Dirichlet tensor performance #222

wouterwln · 2025-01-08T12:47:27Z

Benchmark report:

Benchmark Report for /Users/wnuijten/.julia/dev/ExponentialFamily

Job Properties

Time of benchmarks:
- Target: 8 Jan 2025 - 13:39
- Baseline: 8 Jan 2025 - 13:44
Package commits:
- Target: 0134a8
- Baseline: baf051
Julia commits:
- Target: 4976d0
- Baseline: 4976d0
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["tensordirichlet", "logpdf", "rank=3, d=10"]`	0.93 (10%)	0.07 (5%) ✅
`["tensordirichlet", "logpdf", "rank=3, d=20"]`	0.83 (10%) ✅	0.03 (5%) ✅
`["tensordirichlet", "logpdf", "rank=3, d=5"]`	1.50 (10%) ❌	0.30 (5%) ✅
`["tensordirichlet", "logpdf", "rank=4, d=10"]`	0.84 (10%) ✅	0.03 (5%) ✅
`["tensordirichlet", "logpdf", "rank=4, d=20"]`	0.81 (10%) ✅	0.02 (5%) ✅
`["tensordirichlet", "logpdf", "rank=4, d=5"]`	0.87 (10%) ✅	0.11 (5%) ✅
`["tensordirichlet", "logpdf", "rank=5, d=10"]`	0.83 (10%) ✅	0.03 (5%) ✅
`["tensordirichlet", "logpdf", "rank=5, d=20"]`	0.78 (10%) ✅	0.02 (5%) ✅
`["tensordirichlet", "logpdf", "rank=5, d=5"]`	0.79 (10%) ✅	0.06 (5%) ✅
`["tensordirichlet", "logpdf", "rank=6, d=10"]`	0.83 (10%) ✅	0.03 (5%) ✅
`["tensordirichlet", "logpdf", "rank=6, d=20"]`	0.76 (10%) ✅	0.02 (5%) ✅
`["tensordirichlet", "logpdf", "rank=6, d=5"]`	0.80 (10%) ✅	0.04 (5%) ✅
`["tensordirichlet", "mean", "rank=5, d=20"]`	1.48 (10%) ❌	1.00 (5%)
`["tensordirichlet", "prod", "Closed(rank=3, d=10)"]`	0.06 (10%) ✅	0.13 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=3, d=20)"]`	0.05 (10%) ✅	0.16 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=3, d=5)"]`	0.05 (10%) ✅	0.10 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=4, d=10)"]`	0.05 (10%) ✅	0.13 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=4, d=20)"]`	0.05 (10%) ✅	0.16 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=4, d=5)"]`	0.06 (10%) ✅	0.10 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=5, d=10)"]`	0.06 (10%) ✅	0.13 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=5, d=20)"]`	0.05 (10%) ✅	0.16 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=5, d=5)"]`	0.06 (10%) ✅	0.10 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=6, d=10)"]`	0.09 (10%) ✅	0.13 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=6, d=20)"]`	0.06 (10%) ✅	0.16 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=6, d=5)"]`	0.08 (10%) ✅	0.10 (5%) ✅
`["tensordirichlet", "rand", "rank=4, d=10"]`	0.89 (10%) ✅	1.00 (5%)
`["tensordirichlet", "rand", "rank=4, d=5"]`	0.81 (10%) ✅	1.00 (5%)
`["tensordirichlet", "rand", "rank=5, d=10"]`	0.90 (10%) ✅	1.00 (5%)
`["tensordirichlet", "rand", "rank=5, d=5"]`	0.82 (10%) ✅	1.00 (5%)
`["tensordirichlet", "rand", "rank=6, d=10"]`	0.89 (10%) ✅	1.00 (5%)
`["tensordirichlet", "rand", "rank=6, d=5"]`	0.82 (10%) ✅	1.00 (5%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["bernoulli", "convert"]
["bernoulli", "prod"]
["tensordirichlet", "cov"]
["tensordirichlet", "logpdf"]
["tensordirichlet", "mean"]
["tensordirichlet", "prod"]
["tensordirichlet", "rand"]
["tensordirichlet", "var"]

Julia versioninfo

Target

Julia Version 1.10.7
Commit 4976d05258e (2024-11-26 15:57 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64 arm
  CPU: Apple M1 Pro: 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz    2893546 s          0 s    1669711 s    6067808 s          0 s
       #2  2400 MHz    2880828 s          0 s    1596766 s    6155411 s          0 s
       #3  2400 MHz    5232448 s          0 s     461696 s    4968983 s          0 s
       #4  2400 MHz    4521738 s          0 s     379756 s    5767146 s          0 s
       #5  2400 MHz    3277119 s          0 s     316931 s    7077832 s          0 s
       #6  2400 MHz    1799801 s          0 s     200369 s    8675563 s          0 s
       #7  2400 MHz    1236763 s          0 s     170477 s    9270588 s          0 s
       #8  2400 MHz     875567 s          0 s     151528 s    9652296 s          0 s
  Memory: 32.0 GB (5275.578125 MB free)
  Uptime: 7.701852e6 sec
  Load Avg:  3.82421875  4.07470703125  4.94677734375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 6 virtual cores)

Baseline

Julia Version 1.10.7
Commit 4976d05258e (2024-11-26 15:57 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64 arm
  CPU: Apple M1 Pro: 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz    2893871 s          0 s    1669968 s    6070455 s          0 s
       #2  2400 MHz    2881149 s          0 s    1597017 s    6158069 s          0 s
       #3  2400 MHz    5235114 s          0 s     461923 s    4969333 s          0 s
       #4  2400 MHz    4524322 s          0 s     379966 s    5767595 s          0 s
       #5  2400 MHz    3279623 s          0 s     317134 s    7078368 s          0 s
       #6  2400 MHz    1800801 s          0 s     200452 s    8677724 s          0 s
       #7  2400 MHz    1237399 s          0 s     170552 s    9273120 s          0 s
       #8  2400 MHz     875970 s          0 s     151593 s    9655072 s          0 s
  Memory: 32.0 GB (4949.625 MB free)
  Uptime: 7.702176e6 sec
  Load Avg:  4.20849609375  4.35595703125  4.80615234375
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 6 virtual cores)

wouterwln · 2025-01-08T12:47:45Z

I will work on other methods as well @Raphael-Tresor

wouterwln · 2025-01-09T12:49:04Z

Benchmark Report for /Users/wnuijten/.julia/dev/ExponentialFamily

Job Properties

Time of benchmarks:
- Target: 9 Jan 2025 - 13:41
- Baseline: 9 Jan 2025 - 13:46
Package commits:
- Target: bcebba
- Baseline: 6a7b02
Julia commits:
- Target: 4976d0
- Baseline: 4976d0
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["tensordirichlet", "cov", "rank=4, d=20"]`	1.10 (10%) ❌	1.00 (5%)
`["tensordirichlet", "logpdf", "rank=3, d=10"]`	0.23 (10%) ✅	0.34 (5%) ✅
`["tensordirichlet", "logpdf", "rank=3, d=20"]`	0.21 (10%) ✅	0.39 (5%) ✅
`["tensordirichlet", "logpdf", "rank=3, d=5"]`	0.23 (10%) ✅	0.32 (5%) ✅
`["tensordirichlet", "logpdf", "rank=4, d=10"]`	0.22 (10%) ✅	0.33 (5%) ✅
`["tensordirichlet", "logpdf", "rank=4, d=20"]`	0.21 (10%) ✅	0.39 (5%) ✅
`["tensordirichlet", "logpdf", "rank=4, d=5"]`	0.21 (10%) ✅	0.30 (5%) ✅
`["tensordirichlet", "logpdf", "rank=5, d=10"]`	0.21 (10%) ✅	0.32 (5%) ✅
`["tensordirichlet", "logpdf", "rank=5, d=20"]`	0.22 (10%) ✅	0.39 (5%) ✅
`["tensordirichlet", "logpdf", "rank=5, d=5"]`	0.22 (10%) ✅	0.29 (5%) ✅
`["tensordirichlet", "logpdf", "rank=6, d=10"]`	0.22 (10%) ✅	0.32 (5%) ✅
`["tensordirichlet", "logpdf", "rank=6, d=20"]`	0.21 (10%) ✅	0.39 (5%) ✅
`["tensordirichlet", "logpdf", "rank=6, d=5"]`	0.22 (10%) ✅	0.28 (5%) ✅
`["tensordirichlet", "mean", "rank=3, d=10"]`	0.62 (10%) ✅	0.90 (5%) ✅
`["tensordirichlet", "mean", "rank=3, d=20"]`	0.51 (10%) ✅	0.95 (5%) ✅
`["tensordirichlet", "mean", "rank=3, d=5"]`	0.52 (10%) ✅	0.81 (5%) ✅
`["tensordirichlet", "mean", "rank=4, d=10"]`	0.65 (10%) ✅	0.91 (5%) ✅
`["tensordirichlet", "mean", "rank=4, d=20"]`	0.52 (10%) ✅	0.95 (5%)
`["tensordirichlet", "mean", "rank=4, d=5"]`	0.65 (10%) ✅	0.82 (5%) ✅
`["tensordirichlet", "mean", "rank=5, d=10"]`	0.68 (10%) ✅	0.91 (5%) ✅
`["tensordirichlet", "mean", "rank=5, d=20"]`	0.47 (10%) ✅	0.95 (5%)
`["tensordirichlet", "mean", "rank=5, d=5"]`	0.63 (10%) ✅	0.83 (5%) ✅
`["tensordirichlet", "mean", "rank=6, d=10"]`	0.74 (10%) ✅	0.91 (5%) ✅
`["tensordirichlet", "mean", "rank=6, d=20"]`	0.77 (10%) ✅	0.95 (5%)
`["tensordirichlet", "mean", "rank=6, d=5"]`	0.75 (10%) ✅	0.83 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=3, d=10)"]`	0.93 (10%)	0.19 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=3, d=20)"]`	2.20 (10%) ❌	0.19 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=3, d=5)"]`	0.69 (10%) ✅	0.20 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=4, d=10)"]`	1.42 (10%) ❌	0.18 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=4, d=20)"]`	2.38 (10%) ❌	0.19 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=4, d=5)"]`	0.66 (10%) ✅	0.19 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=5, d=10)"]`	1.56 (10%) ❌	0.18 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=5, d=20)"]`	1.81 (10%) ❌	0.19 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=5, d=5)"]`	0.72 (10%) ✅	0.18 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=6, d=10)"]`	1.43 (10%) ❌	0.18 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=6, d=20)"]`	0.99 (10%)	0.19 (5%) ✅
`["tensordirichlet", "prod", "Closed(rank=6, d=5)"]`	1.00 (10%)	0.18 (5%) ✅
`["tensordirichlet", "var", "rank=3, d=10"]`	0.06 (10%) ✅	0.25 (5%) ✅
`["tensordirichlet", "var", "rank=3, d=20"]`	0.05 (10%) ✅	0.28 (5%) ✅
`["tensordirichlet", "var", "rank=3, d=5"]`	0.08 (10%) ✅	0.23 (5%) ✅
`["tensordirichlet", "var", "rank=4, d=10"]`	0.06 (10%) ✅	0.24 (5%) ✅
`["tensordirichlet", "var", "rank=4, d=20"]`	0.05 (10%) ✅	0.28 (5%) ✅
`["tensordirichlet", "var", "rank=4, d=5"]`	0.07 (10%) ✅	0.22 (5%) ✅
`["tensordirichlet", "var", "rank=5, d=10"]`	0.07 (10%) ✅	0.24 (5%) ✅
`["tensordirichlet", "var", "rank=5, d=20"]`	0.06 (10%) ✅	0.28 (5%) ✅
`["tensordirichlet", "var", "rank=5, d=5"]`	0.09 (10%) ✅	0.21 (5%) ✅
`["tensordirichlet", "var", "rank=6, d=10"]`	0.09 (10%) ✅	0.24 (5%) ✅
`["tensordirichlet", "var", "rank=6, d=20"]`	0.08 (10%) ✅	0.28 (5%) ✅
`["tensordirichlet", "var", "rank=6, d=5"]`	0.11 (10%) ✅	0.21 (5%) ✅

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["bernoulli", "convert"]
["bernoulli", "prod"]
["tensordirichlet", "cov"]
["tensordirichlet", "logpdf"]
["tensordirichlet", "mean"]
["tensordirichlet", "prod"]
["tensordirichlet", "rand"]
["tensordirichlet", "var"]

Julia versioninfo

Target

Julia Version 1.10.7
Commit 4976d05258e (2024-11-26 15:57 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64 arm
  CPU: Apple M1 Pro: 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz    2972556 s          0 s    1717936 s    6230080 s          0 s
       #2  2400 MHz    2959642 s          0 s    1643024 s    6319926 s          0 s
       #3  2400 MHz    5443770 s          0 s     474188 s    5035591 s          0 s
       #4  2400 MHz    4708715 s          0 s     390976 s    5859439 s          0 s
       #5  2400 MHz    3405539 s          0 s     327119 s    7229746 s          0 s
       #6  2400 MHz    1891216 s          0 s     205955 s    8869241 s          0 s
       #7  2400 MHz    1300484 s          0 s     175448 s    9492596 s          0 s
       #8  2400 MHz     912638 s          0 s     155961 s    9901528 s          0 s
  Memory: 32.0 GB (2389.6875 MB free)
  Uptime: 7.78841e6 sec
  Load Avg:  5.35986328125  5.50732421875  5.2998046875
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 6 virtual cores)

Baseline

Julia Version 1.10.7
Commit 4976d05258e (2024-11-26 15:57 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  uname: Darwin 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64 arm
  CPU: Apple M1 Pro: 
              speed         user         nice          sys         idle          irq
       #1  2400 MHz    2973444 s          0 s    1718361 s    6231866 s          0 s
       #2  2400 MHz    2960531 s          0 s    1643434 s    6321727 s          0 s
       #3  2400 MHz    5445892 s          0 s     474466 s    5036303 s          0 s
       #4  2400 MHz    4710714 s          0 s     391238 s    5860291 s          0 s
       #5  2400 MHz    3407390 s          0 s     327390 s    7230738 s          0 s
       #6  2400 MHz    1893165 s          0 s     206067 s    8870296 s          0 s
       #7  2400 MHz    1301367 s          0 s     175575 s    9494702 s          0 s
       #8  2400 MHz     913207 s          0 s     156060 s    9903977 s          0 s
  Memory: 32.0 GB (4134.328125 MB free)
  Uptime: 7.788722e6 sec
  Load Avg:  4.431640625  5.06591796875  5.18408203125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 6 virtual cores)

wouterwln requested a review from Raphael-Tresor January 8, 2025 12:47

wouterwln added 8 commits January 9, 2025 13:25

Reimplement prod

6c9d756

Update judge benchmarks

40f08e2

Speedup logpdf

78026b8

Update benchmarks

e43b376

Save alpha0

678af98

reimplement var

89dd0ba

Speedup var

bd657cb

Speedup logpdf

2aba23d

wouterwln force-pushed the dirichlet-tensor-performance branch from 68b4a12 to 2aba23d Compare January 9, 2025 12:28

wouterwln added 2 commits January 9, 2025 13:36

Introduce error msg for negative alpha

2aa166d

Make benchmarks robust

bcebba4

wouterwln merged commit 6160272 into DirichletTensor Jan 9, 2025
1 of 2 checks passed

wouterwln deleted the dirichlet-tensor-performance branch January 9, 2025 12:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dirichlet tensor performance #222

Dirichlet tensor performance #222

wouterwln commented Jan 8, 2025

wouterwln commented Jan 8, 2025

wouterwln commented Jan 9, 2025

Dirichlet tensor performance #222

Dirichlet tensor performance #222

Conversation

wouterwln commented Jan 8, 2025

Benchmark Report for /Users/wnuijten/.julia/dev/ExponentialFamily

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

wouterwln commented Jan 8, 2025

wouterwln commented Jan 9, 2025

Benchmark Report for /Users/wnuijten/.julia/dev/ExponentialFamily

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline