Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #304 (simplify init clock) #323

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

turion
Copy link
Owner

@turion turion commented May 16, 2024

Fixes #304.

  • Run benchmarks

@turion
Copy link
Owner Author

turion commented May 16, 2024

Benchmarks

There is quite some variance, and no speedup visible yet. I should look at the Core generated by the Sum example to understand whether it has improved at all. See also #309.

with this change (8adcca8):

benchmarking WordCount/rhine
time                 655.5 ms   (598.4 ms .. 703.1 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 664.7 ms   (657.4 ms .. 672.0 ms)
std dev              8.804 ms   (5.161 ms .. 10.93 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking WordCount/automaton
time                 114.4 ms   (111.7 ms .. 116.8 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 115.1 ms   (114.1 ms .. 117.5 ms)
std dev              2.273 ms   (784.7 μs .. 3.578 ms)
variance introduced by outliers: 11% (moderately inflated)

benchmarking WordCount/Text/IORef
time                 87.06 ms   (85.73 ms .. 88.76 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 88.06 ms   (87.46 ms .. 88.56 ms)
std dev              930.9 μs   (691.0 μs .. 1.301 ms)

benchmarking WordCount/Text/no IORef
time                 175.2 ms   (166.2 ms .. 197.3 ms)
                     0.994 R²   (0.977 R² .. 1.000 R²)
mean                 186.1 ms   (178.6 ms .. 203.4 ms)
std dev              15.33 ms   (2.881 ms .. 21.51 ms)
variance introduced by outliers: 16% (moderately inflated)

benchmarking WordCount/Text/Lazy
time                 105.5 ms   (103.3 ms .. 108.9 ms)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 105.1 ms   (104.3 ms .. 106.4 ms)
std dev              1.529 ms   (984.1 μs .. 2.212 ms)

benchmarking Sum/rhine
time                 61.34 ms   (59.25 ms .. 64.30 ms)
                     0.994 R²   (0.985 R² .. 0.999 R²)
mean                 62.31 ms   (61.16 ms .. 64.29 ms)
std dev              2.719 ms   (1.851 ms .. 3.937 ms)

benchmarking Sum/rhine flow
time                 847.5 ms   (823.7 ms .. 859.8 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 838.5 ms   (832.8 ms .. 842.4 ms)
std dev              5.581 ms   (2.444 ms .. 7.625 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking Sum/automaton
time                 34.02 ms   (33.10 ms .. 35.58 ms)
                     0.994 R²   (0.984 R² .. 0.999 R²)
mean                 35.14 ms   (34.18 ms .. 37.34 ms)
std dev              3.112 ms   (1.272 ms .. 5.645 ms)
variance introduced by outliers: 36% (moderately inflated)

benchmarking Sum/direct
time                 581.8 μs   (580.0 μs .. 584.6 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 590.1 μs   (587.8 μs .. 592.6 μs)
std dev              7.778 μs   (6.721 μs .. 9.283 μs)

benchmarking Sum/direct monad
time                 589.0 μs   (585.6 μs .. 593.6 μs)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 587.2 μs   (584.2 μs .. 593.8 μs)
std dev              13.54 μs   (6.966 μs .. 25.76 μs)
variance introduced by outliers: 14% (moderately inflated)

master (d2ae786)

benchmarking WordCount/rhine
time                 481.0 ms   (377.8 ms .. 527.6 ms)
                     0.994 R²   (0.987 R² .. 1.000 R²)
mean                 519.5 ms   (499.4 ms .. 530.2 ms)
std dev              19.76 ms   (23.56 μs .. 24.70 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking WordCount/automaton
time                 107.6 ms   (103.6 ms .. 112.2 ms)
                     0.997 R²   (0.993 R² .. 1.000 R²)
mean                 107.4 ms   (105.6 ms .. 110.9 ms)
std dev              3.836 ms   (2.304 ms .. 5.703 ms)

benchmarking WordCount/Text/IORef
time                 81.61 ms   (75.91 ms .. 88.01 ms)
                     0.994 R²   (0.992 R² .. 0.999 R²)
mean                 83.53 ms   (80.31 ms .. 91.50 ms)
std dev              8.113 ms   (2.637 ms .. 13.02 ms)
variance introduced by outliers: 29% (moderately inflated)

benchmarking WordCount/Text/no IORef
time                 225.5 ms   (206.0 ms .. 265.3 ms)
                     0.987 R²   (0.967 R² .. 1.000 R²)
mean                 207.2 ms   (189.2 ms .. 218.4 ms)
std dev              19.63 ms   (10.63 ms .. 31.34 ms)
variance introduced by outliers: 30% (moderately inflated)

benchmarking WordCount/Text/Lazy
time                 114.8 ms   (107.3 ms .. 120.6 ms)
                     0.993 R²   (0.977 R² .. 0.999 R²)
mean                 107.4 ms   (101.6 ms .. 112.3 ms)
std dev              8.633 ms   (6.403 ms .. 10.90 ms)
variance introduced by outliers: 21% (moderately inflated)

benchmarking Sum/rhine
time                 69.81 ms   (49.84 ms .. 107.4 ms)
                     0.772 R²   (0.541 R² .. 0.983 R²)
mean                 74.78 ms   (67.68 ms .. 85.35 ms)
std dev              15.21 ms   (10.06 ms .. 20.82 ms)
variance introduced by outliers: 71% (severely inflated)

benchmarking Sum/rhine flow
time                 819.5 ms   (620.9 ms .. 992.2 ms)
                     0.990 R²   (0.989 R² .. 1.000 R²)
mean                 685.1 ms   (634.2 ms .. 754.4 ms)
std dev              66.09 ms   (5.111 ms .. 82.52 ms)
variance introduced by outliers: 22% (moderately inflated)

benchmarking Sum/automaton
time                 39.01 ms   (32.57 ms .. 44.55 ms)
                     0.927 R²   (0.859 R² .. 0.996 R²)
mean                 33.44 ms   (31.93 ms .. 37.37 ms)
std dev              4.729 ms   (2.209 ms .. 8.095 ms)
variance introduced by outliers: 55% (severely inflated)

benchmarking Sum/direct
time                 566.9 μs   (563.4 μs .. 572.7 μs)
                     0.998 R²   (0.996 R² .. 1.000 R²)
mean                 576.0 μs   (570.7 μs .. 584.8 μs)
std dev              23.73 μs   (13.50 μs .. 42.51 μs)
variance introduced by outliers: 34% (moderately inflated)

benchmarking Sum/direct monad
time                 569.3 μs   (562.1 μs .. 581.0 μs)
                     0.999 R²   (0.997 R² .. 1.000 R²)
mean                 568.8 μs   (565.8 μs .. 574.4 μs)
std dev              12.94 μs   (8.486 μs .. 21.61 μs)
variance introduced by outliers: 14% (moderately inflated)

@turion turion force-pushed the dev_fix_#304_simplify_initClock branch from 8adcca8 to 5410ef7 Compare May 17, 2024 14:02
@turion turion force-pushed the dev_fix_#304_simplify_initClock branch 2 times, most recently from 2de85f0 to 19076ef Compare July 18, 2024 15:20
@turion turion force-pushed the dev_fix_#304_simplify_initClock branch from 19076ef to 66cf9cd Compare August 8, 2024 18:50
@turion
Copy link
Owner Author

turion commented Aug 13, 2024

This PR alone cannot give the speedups I'm hoping for, because clock erasure of SNs still doesn't happen. Only in combination with #348 I'll be able to get the full speedup I'm hoping for.

@turion
Copy link
Owner Author

turion commented Aug 13, 2024

#349 is helpful because it adds a more realistic benchmark where initClock is not inlined yet because it contains an IO action.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Simplify initClock?
1 participant