-
Notifications
You must be signed in to change notification settings - Fork 867
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve debug_traceBlock calls performance #8103
Open
ahamlat
wants to merge
16
commits into
hyperledger:main
Choose a base branch
from
ahamlat:debug-trace-with-pipeline
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
+4,059
−7,079
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Change debug_traceBlockByNumber implementation by using a pipeline - Change the defaults to match Geth and Nethermind defaults - Improve general performance - Add unit tests Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
…egration test because they were not relevant. Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
from Besu Controller. Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
Signed-off-by: Ameziane H. <[email protected]>
…debug-trace-with-pipeline
…ethodsFactory Signed-off-by: Ameziane H. <[email protected]>
…t not performant enough Signed-off-by: Ameziane H. <[email protected]>
ahamlat
changed the title
Debug trace with pipeline
Improve debug_traceBlock calls performance
Jan 10, 2025
matkt
force-pushed
the
debug-trace-with-pipeline
branch
from
January 10, 2025 14:41
c00f7db
to
447907b
Compare
Signed-off-by: ahamlat <[email protected]>
8 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR description
This PR is the same as this PR #8076, I closed by error.
It introduces several key improvements to the debug_traceBlock, debug_traceBlockByNumber, and debug_traceBlockByHash methods, significantly enhancing both performance and memory usage:
The core change is the adoption of a pipeline architecture, where:
One thread handles transaction execution.
Other threads processes the execution results and generates the JSON output. This parallelization ensures better resource utilization and reduces overall execution time.
Memory usage is optimized by the pipeline processing approach. Instead of retaining all trace frames for every transaction, the pipeline now keeps only the trace frames relevant to the current transaction, significantly reducing memory consumption.
The transformation of trace frames into Java JSON representations now occurs in parallel.
Default trace options have been adjusted to align with the behavior of Geth and Nethermind:
disableStorage is set to false (enabling storage tracing).
disableMemory is set to true (disabling memory tracing).
disableStack is set to false (enabling stack tracing).
The structure of the JSON output has been modified to match Geth's format:
Hexadecimal values are now displayed as short hex strings (e.g., 0x1, 0xabc), which is more compact and efficient.
The stack array is always displayed when the option is enabled, even when empty, to ensure consistency with other Ethereum clients. However, memory and storage are omitted if they are empty, which reduces unnecessary data.
Storage keys and values are now displayed without the 0x prefix, in line with Geth's behavior.
Change the implementation of toShortHexString, by creating a new method toCompactHex. The main improvement is to not relay on Bytes, but on byte[] to avoid dynamic dispatch mechanism related to the use of methods get(i) and size(). Those methods are not used anymore thanks to the switch to StringBuilder, even if it has its own cons. This method will replace in the future toFastHex implementation, once we have finished the work on Tuweni.
These changes have resulted in a 3X performance improvement, reducing the size of the output by 2 to 3 times, making previously unresponsive calls fast enough with comparable performance with other clients. The optimizations not only improve response times but also allow for smoother operation with lower memory usage, even when processing large traces.
Overall, the new approach ensures better performance, reduced resource usage, and more consistent output with other Ethereum clients, particularly Geth and Nethermind.
Fixed Issue(s)
Fixes #5322
Thanks for sending a pull request! Have you done the following?
doc-change-required
label to this PR if updates are required.Locally, you can run these tests to catch failures early:
./gradlew build
./gradlew acceptanceTest
./gradlew integrationTest
./gradlew ethereum:referenceTests:referenceTests