Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add StaticFileSegment::BlockMeta to StaticFileProducer and Pruner #13666

Open
wants to merge 1 commit into
base: joshie/bmeta
Choose a base branch
from
Open

feat: add StaticFileSegment::BlockMeta to StaticFileProducer and Pruner #13666

wants to merge 1 commit into from

Conversation

joshieDo
Copy link
Collaborator

@joshieDo joshieDo commented Jan 6, 2025 •
edited
Loading

on top of #13226

  • Enable lz4 for the segment
  • Adds handling of the segment in the StaticFileProducer and Pruner.
  • Enables copying (BodyIndices, Withdrawals and Ommers) from database to static files. This will happen during a new start of the node. It does not append during any pipeline stage or live sync though.
  • Adds provider handling with get_with_static_file_or_database for the relevant types.
  • Add to reth db get static-file block-meta ...

@joshieDo joshieDo added C-enhancement New feature or request A-pruning Related to pruning or full node A-static-files Related to static files labels Jan 6, 2025
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch 2 times, most recently from f36699c to cb754f1 Compare January 6, 2025 11:41
@joshieDo joshieDo changed the base branch from main to joshie/bmeta January 6, 2025 12:04
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch from cb754f1 to 3030b07 Compare January 6, 2025 14:36
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch from 3030b07 to 3e10222 Compare January 7, 2025 12:21
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch 2 times, most recently from 082c72f to 2e718be Compare January 8, 2025 11:38
@joshieDo joshieDo marked this pull request as ready for review January 8, 2025 15:29
mattsse
mattsse requested changes Jan 8, 2025
View reviewed changes
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests look good, although all the iter logic is a bit hard to follow and I'd appreciate some additional docs here

crates/prune/prune/src/segments/static_file/block_meta.rs Outdated Show resolved Hide resolved
crates/prune/prune/src/segments/static_file/block_meta.rs Outdated Show resolved Hide resolved
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch from 2e718be to dcf0e59 Compare January 9, 2025 11:38
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch from dcf0e59 to 4b3f7c3 Compare January 9, 2025 12:55
@joshieDo joshieDo force-pushed the joshie/bmeta branch 2 times, most recently from 2874270 to 1eb018d Compare January 9, 2025 13:52
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch 2 times, most recently from 4624976 to d7e41c8 Compare January 9, 2025 14:26
@joshieDo joshieDo force-pushed the joshie/bmeta branch 2 times, most recently from 7ed1564 to 674a603 Compare January 9, 2025 14:38
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch from d7e41c8 to 63adc23 Compare January 9, 2025 14:38
@joshieDo joshieDo force-pushed the joshie/bmeta-pruner-producer branch from 63adc23 to f66f6de Compare January 9, 2025 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattsse mattsse mattsse requested changes

@rakita rakita Awaiting requested review from rakita rakita is a code owner

@shekhirin shekhirin Awaiting requested review from shekhirin shekhirin is a code owner

@onbjerg onbjerg Awaiting requested review from onbjerg onbjerg is a code owner

Assignees
No one assigned
Labels
A-pruning Related to pruning or full node A-static-files Related to static files C-enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@joshieDo @mattsse