Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Measurement and Scoring details to about page #467

Merged
merged 12 commits into from
Jan 8, 2025
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,25 @@ benchmark, actual users should benefit. In order to achieve this, it should:
- Be accessible to the public and useful to browser engineers. It should run in every modern
browser by visiting a normal web page. It should run relatively quickly, while providing
enough test coverage to be reflective of the real-world Web.

# Score and Test Measurement

Each test can contain several steps who contributed to the test duration. Note that the prepare step of a test is _unmeasured_ and thus does not contribute to the score. All following test
steps are **measured** and summed up in the test step time.

<img src="./resources/measurement-timeline.svg" />

Each step consists of the following phases:

<dl>
<dt>Sync:</dt>
<dd>Time spent in synchronous JS execution.</dd>
<dt>Async:</dt>
<dd>Time spent in asynchronous JS like Promise or setTimeout tasks scheduled by the sync-step.</dd>
</dl>

## Score

The final benchmark score is calculated based of the inverse of [geomean](https://en.wikipedia.org/wiki/Geometric_mean) of all tests. We average the score over multiple iterations with the [arithmetic mean](https://en.wikipedia.org/wiki/Arithmetic_mean). By using the geomean to combine the test durations we maintain the invariant that relative improvements are favoured equally amongst all tests, even though they can have vastly different durations.

<img src="./resources/score.svg" />
88 changes: 88 additions & 0 deletions resources/measurement-timeline.svg
camillobruni marked this conversation as resolved.
Show resolved Hide resolved
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
201 changes: 201 additions & 0 deletions resources/score.svg
camillobruni marked this conversation as resolved.
Show resolved Hide resolved
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading