feat: implement EIP-7691 increase blob throughput #7309

ensi321 · 2024-12-17T01:23:19Z

Outstanding items that is not covered in this PR:

Make callers of requestSszTypeByMethod to provide fork info
Make InboundRateLimitQuota fork-aware so byPeer limit can be correctly set for BlobSidecarsByRange and BlobSidecarsByRoot.
Make ReqRespRateLimiter to re-set the rate limit on fork activation

codecov · 2024-12-18T22:48:25Z

Codecov Report

Attention: Patch coverage is 38.09524% with 52 lines in your changes missing coverage. Please review.

Project coverage is 48.60%. Comparing base (265579f) to head (6e0399f).
Report is 1 commits behind head on devnet-5.

Additional details and impacted files

@@             Coverage Diff              @@
##           devnet-5    #7309      +/-   ##
============================================
- Coverage     48.61%   48.60%   -0.02%     
============================================
  Files           603      603              
  Lines         40457    40513      +56     
  Branches       2065     2065              
============================================
+ Hits          19667    19690      +23     
- Misses        20752    20785      +33     
  Partials         38       38

ensi321 · 2024-12-18T23:08:58Z

Please ignore E2E Tests and spec tests as they will be addressed in the next PR.

packages/beacon-node/src/network/network.ts

packages/beacon-node/src/chain/validation/blobSidecar.ts

packages/config/src/constantsHelper.ts

packages/state-transition/src/block/processExecutionPayload.ts

packages/beacon-node/src/chain/validation/blobSidecar.ts

g11tech · 2024-12-20T08:34:27Z

packages/beacon-node/src/network/network.ts

+      this.sendReqRespRequest(
+        peerId,
+        ReqRespMethod.BlobSidecarsByRoot,
+        [isForkPostElectra(fork) ? Version.V2 : Version.V1],


whats different in v2?

Max size of the param in V1 is MAX_REQUEST_BLOB_SIDECARS(768) , V2 is MAX_REQUEST_BLOB_SIDECARS_ELECTRA(1152)

there is some discussion around this on discord, we might get rid of v2 again in a future spec release

packages/beacon-node/src/network/processor/gossipHandlers.ts

g11tech · 2024-12-20T08:38:54Z

packages/beacon-node/src/network/reqresp/ReqRespBeaconNode.ts

@@ -207,7 +207,8 @@ export class ReqRespBeaconNode extends ReqResp {
    versions: number[],
    request: Req
  ): AsyncIterable<ResponseIncoming> {
-    const requestType = requestSszTypeByMethod(this.config)[method];
+    const fork = ForkName[ForkSeq[this.currentRegisteredFork] as keyof typeof ForkName];
+    const requestType = requestSszTypeByMethod(this.config, fork)[method];


we need this for v1/v2?

Yes, request type for BlobSidecarsByRootV2 has changed. But ReqRespMethod does not communicate version info, so we can only pass in fork info to determine the correct type.

g11tech · 2024-12-20T08:41:15Z

packages/beacon-node/src/network/reqresp/handlers/index.ts

@@ -38,7 +39,8 @@ export function getReqRespHandlers({db, chain}: {db: IBeaconDb; chain: IBeaconCh
      return onBeaconBlocksByRoot(body, chain, db);
    },
    [ReqRespMethod.BlobSidecarsByRoot]: (req) => {
-      const body = BlobSidecarsByRootRequestType(chain.config).deserialize(req.data);
+      const fork = req.version === Version.V2 ? ForkName.electra : ForkName.deneb;
+      const body = BlobSidecarsByRootRequestType(fork, chain.config).deserialize(req.data);


wouldn't type be the same just that v2 will have context bytes/fork type?

V2 accepts higher number of blob requests so the ssz container is different.

packages/params/src/index.ts

nflaig

LGTM - we might have to remove req/resp v2 methods again in the future but as it's part of the spec for devnet-5 we should probably keep it for now as other clients might rely on it

nflaig · 2025-01-08T12:30:58Z

packages/params/src/presets/minimal.ts

-  MAX_BLOB_COMMITMENTS_PER_BLOCK: 16,
-  KZG_COMMITMENT_INCLUSION_PROOF_DEPTH: 9,
+  MAX_BLOB_COMMITMENTS_PER_BLOCK: 32,
+  KZG_COMMITMENT_INCLUSION_PROOF_DEPTH: 10,


It looks like there is a consensus issue with Lighthouse in our sim tests due to this, need to use a image with the updated minimal preset values. Should not be a blocker to merge this but should figure out a solution (ie. updating LH image) before we merge the devnet-5 branch to unstable

ensi321 · 2025-01-08T23:33:41Z

LGTM - we might have to remove req/resp v2 methods again in the future but as it's part of the spec for devnet-5 we should probably keep it for now as other clients might rely on it

Looks like removing v2 might be happening in devnet 5. Decision will be made in ACD this week. Will make another PR to remove them if included in devnet 5

ensi321 force-pushed the devnet-5 branch from 324562a to 031214e Compare December 17, 2024 06:22

ensi321 added 2 commits December 16, 2024 22:22

Init

13d6448

Add reqresp v2 definition

a8d9f73

ensi321 force-pushed the nc/eip-7691 branch from dc398e5 to a8d9f73 Compare December 17, 2024 06:22

ensi321 added 7 commits December 16, 2024 22:25

Update validateGossipBlock

e6984f4

Partial commit

6b48588

Reqresp. Add todos

1722971

polish

f031dc3

Fork-aware requestSszTypeByMethod

a8f34b8

Fixed minimal constants

d3361b4

Bump config test version

ebc2aae

ensi321 marked this pull request as ready for review December 18, 2024 23:07

ensi321 requested a review from a team as a code owner December 18, 2024 23:07

nflaig added 7 commits December 19, 2024 12:16

Merge branch 'devnet-5' into nc/eip-7691

7a51d37

Update blob sidecar subnet computation

13ad885

Pass proper commitment limit to block gossip error

2c68274

Update blob sidecar index check

c22f78b

Lint

aa8a218

Update kzg unit test

41be261

Subscribe to correct number of blob sidecar subnets

1c2b5ed

nflaig reviewed Dec 19, 2024

View reviewed changes

packages/beacon-node/src/network/network.ts Outdated Show resolved Hide resolved

packages/beacon-node/src/chain/validation/blobSidecar.ts Outdated Show resolved Hide resolved

ensi321 added 2 commits December 19, 2024 16:06

Refactor constants getter to constantsHelper

a267c34

address comment

e813294

nflaig reviewed Dec 20, 2024

View reviewed changes

packages/config/src/constantsHelper.ts Outdated Show resolved Hide resolved

packages/config/src/constantsHelper.ts Outdated Show resolved Hide resolved

packages/state-transition/src/block/processExecutionPayload.ts Outdated Show resolved Hide resolved

g11tech reviewed Dec 20, 2024

View reviewed changes

packages/beacon-node/src/chain/validation/blobSidecar.ts Outdated Show resolved Hide resolved

g11tech reviewed Dec 20, 2024

View reviewed changes

packages/beacon-node/src/network/processor/gossipHandlers.ts Outdated Show resolved Hide resolved

g11tech reviewed Dec 20, 2024

View reviewed changes

packages/params/src/index.ts Show resolved Hide resolved

nflaig added 2 commits December 20, 2024 10:07

Pass fork as first arg

744c0be

Update packages/state-transition/src/block/processExecutionPayload.ts

760fbb8

nflaig mentioned this pull request Dec 20, 2024

refactor: move helper to get max blobs per block to fork config #7322

Merged

nflaig added 3 commits January 6, 2025 17:00

refactor: move helper to get max blobs per block to fork config (#7322)

8a49542

Simplify type cast

d5b0038

Merge branch 'devnet-5' into nc/eip-7691

6e0399f

nflaig approved these changes Jan 8, 2025

View reviewed changes

nflaig reviewed Jan 8, 2025

View reviewed changes

ensi321 mentioned this pull request Jan 9, 2025

Fix sim test in devnet-5 branch #7339

Open

ensi321 merged commit 42d6982 into devnet-5 Jan 9, 2025
19 of 23 checks passed

ensi321 deleted the nc/eip-7691 branch January 9, 2025 06:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement EIP-7691 increase blob throughput #7309

feat: implement EIP-7691 increase blob throughput #7309

ensi321 commented Dec 17, 2024 •

edited by wemeetagain

Loading

codecov bot commented Dec 18, 2024 •

edited

Loading

ensi321 commented Dec 18, 2024

g11tech Dec 20, 2024

ensi321 Jan 6, 2025

nflaig Jan 8, 2025

g11tech Dec 20, 2024 •

edited

Loading

ensi321 Jan 6, 2025

g11tech Dec 20, 2024

ensi321 Jan 6, 2025

nflaig left a comment

nflaig Jan 8, 2025 •

edited

Loading

ensi321 commented Jan 8, 2025

feat: implement EIP-7691 increase blob throughput #7309

feat: implement EIP-7691 increase blob throughput #7309

Conversation

ensi321 commented Dec 17, 2024 • edited by wemeetagain Loading

codecov bot commented Dec 18, 2024 • edited Loading

Codecov Report

ensi321 commented Dec 18, 2024

g11tech Dec 20, 2024

Choose a reason for hiding this comment

ensi321 Jan 6, 2025

Choose a reason for hiding this comment

nflaig Jan 8, 2025

Choose a reason for hiding this comment

g11tech Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

ensi321 Jan 6, 2025

Choose a reason for hiding this comment

g11tech Dec 20, 2024

Choose a reason for hiding this comment

ensi321 Jan 6, 2025

Choose a reason for hiding this comment

nflaig left a comment

Choose a reason for hiding this comment

nflaig Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

ensi321 commented Jan 8, 2025

ensi321 commented Dec 17, 2024 •

edited by wemeetagain

Loading

codecov bot commented Dec 18, 2024 •

edited

Loading

g11tech Dec 20, 2024 •

edited

Loading

nflaig Jan 8, 2025 •

edited

Loading