Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up sync #2316

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

Speed up sync #2316

wants to merge 25 commits into from

Conversation

shawn-zil
Copy link
Contributor

@shawn-zil shawn-zil commented Feb 10, 2025

Speeds up Phase 1, by firing 10 concurrent requests at a time, processing 10 responses, then firing the next 10, etc. The speedup in this sense is fairly linear (at least up to 10), as expected. Since Phase 2 is CPU-bound, firing concurrent requests is unnecessary.

When processing the Phase 1 responses, it also splits the chain segment into sub-segments, based on the estimated response payload size. So, if there are large blocks, it will split the segment into several sub-segments for Phase 2 processing.

This removes support for V1 compatibility. Do not merge this until the hard-fork to >= 0.6.0

@shawn-zil shawn-zil linked an issue Feb 10, 2025 that may be closed by this pull request
Copy link
Contributor

github-actions bot commented Feb 10, 2025

🐰 Bencher Report

Branch2158-speedup-sync
Testbedself-hosted
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
milliseconds (ms)
(Result Δ%)
Upper Boundary
milliseconds (ms)
(Limit %)
full-blocks-erc20-transfers/full-blocks-erc20-transfers📈 view plot
🚷 view threshold
1,447.10
(+21.14%)Baseline: 1,194.52
1,595.83
(90.68%)
full-blocks-evm-transfers/full-blocks-evm-transfers📈 view plot
🚷 view threshold
437.71
(-2.99%)Baseline: 451.22
593.57
(73.74%)
full-blocks-zil-transfers/full-blocks-zil-transfers📈 view plot
🚷 view threshold
4,106.50
(+3.68%)Baseline: 3,960.87
5,320.34
(77.18%)
process-empty/process-empty📈 view plot
🚷 view threshold
10.17
(+9.19%)Baseline: 9.31
11.81
(86.11%)
🐰 View full continuous benchmarking report in Bencher

@shawn-zil shawn-zil marked this pull request as ready for review February 13, 2025 09:52
@shawn-zil shawn-zil requested review from bzawisto and 86667 February 13, 2025 09:52
@shawn-zil shawn-zil marked this pull request as draft February 13, 2025 11:51
@shawn-zil shawn-zil marked this pull request as ready for review February 14, 2025 08:48
@shawn-zil shawn-zil enabled auto-merge February 14, 2025 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Speed up new sync algorithm
1 participant