save light client updates (#14618)

* Light Client - use the new consensus types (#14549) * in progress * completed logic * var name * additional logic changes * fix createDefaultLightClientUpdate * empty fields * unused context * Return the correct light client payload proof (#14565) * Return the correct payload proof * changelog <3 * Set fields of wrapped proto object in light client setters (#14573) * Set fields of wrapped proto object in light client setters * changelog <3 * fix TODOs for events (#14570) * fix TODOs for events * address review comments * Update beacon-chain/rpc/eth/events/events.go Co-authored-by: Radosław Kapka <[email protected]> * Update beacon-chain/rpc/eth/events/events.go Co-authored-by: Radosław Kapka <[email protected]> * nits --------- Co-authored-by: Radosław Kapka <[email protected]> * align changelog with develop branch * bzl * LC Updates by Range to read from DB (#14531) * change updatebyrange * lcupdateresponse from consensus * range altair test * range forks tests * finish tests * changelog * remove unused functions * Update beacon-chain/rpc/eth/light-client/handlers.go Co-authored-by: Radosław Kapka <[email protected]> * Update beacon-chain/rpc/eth/light-client/handlers.go Co-authored-by: Radosław Kapka <[email protected]> * use slice instead of array * refactor code * refactor tests * refactor tests * refactor tests * add configCleanup in tests * refactor missing updates testcase * Light Client - use the new consensus types (#14549) * in progress * completed logic * var name * additional logic changes * fix createDefaultLightClientUpdate * empty fields * unused context * change updatesByRange to use new structs * Light Client - use the new consensus types (#14549) * in progress * completed logic * var name * additional logic changes * fix createDefaultLightClientUpdate * empty fields * unused context * fix rpc/helpers_test * Return the correct light client payload proof (#14565) * Return the correct payload proof * changelog <3 * merge * Set fields of wrapped proto object in light client setters (#14573) * Set fields of wrapped proto object in light client setters * changelog <3 * fixing tests... * core tests fixed * kv tests fixed * fix TODOs for events (#14570) * fix TODOs for events * address review comments * Update beacon-chain/rpc/eth/events/events.go Co-authored-by: Radosław Kapka <[email protected]> * Update beacon-chain/rpc/eth/events/events.go Co-authored-by: Radosław Kapka <[email protected]> * nits --------- Co-authored-by: Radosław Kapka <[email protected]> * tests fixed * remove unused function * fix slice capacity * address issues * address issues * fix circular import error * remove unused func * fix changelog --------- Co-authored-by: Radosław Kapka <[email protected]> Co-authored-by: Radosław Kapka <[email protected]> Co-authored-by: Rupam Dey <[email protected]> * Light Client: Electra finality branch (#14597) * extract from lc-p2p branch * generate code * trixy's review * test fixes * fix issue in `CreateDefaultLightClientUpdate` function (#14585) * use state in `CreateDefaultLightClientUpdate` * lint * add `stateSlot` to `update.go` structs * Revert "add `stateSlot` to `update.go` structs" This reverts commit 84468ae. * set sync committee based on attestedHeader in updateElectra * dependencies * add check to `SetNextSyncCommitteeBranchElectra` * add detailed error messages to `update.go` * dependencies * fix `createDefaultLightClientUpdate` * deps * fix errors * deps * revert error messages * deps * save update * save update * move create lc bootstrap to core * bootstrap db * save bootstrap * testing * testing progress * testing * testing * checkpoint working * set genesis time manually * savebootstrap tests * conflicts resolved * fix lint issues * fix lint issues * address reviews * revert changes to consensus-types/lc/updates.go * add lightClientHeaderToJSON support for Electra --------- Co-authored-by: Radosław Kapka <[email protected]> Co-authored-by: Rupam Dey <[email protected]> Co-authored-by: Radosław Kapka <[email protected]> Co-authored-by: Inspector-Butters <[email protected]>
prysmaticlabs · Nov 28, 2024 · ae779a7 · ae779a7
1 parent 9574915
commit ae779a7
Show file tree

Hide file tree

Showing 19 changed files with 1,601 additions and 919 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -34,7 +34,7 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
 - Updated the default `scrape-interval` in `Client-stats` to 2 minutes to accommodate Beaconcha.in API rate limits.
 - Switch to compounding when consolidating with source==target.
 - Revert block db save when saving state fails.
-- Return false from HasBlock if the block is being synced. 
+- Return false from HasBlock if the block is being synced.
 - Cleanup forkchoice on failed insertions.
 - Use read only validator for core processing to avoid unnecessary copying.
 - Use ROBlock across block processing pipeline.
@@ -88,9 +88,9 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
 
 ### Security
 
-## [v5.1.2](https://github.com/prysmaticlabs/prysm/compare/v5.1.1...v5.1.2) - 2024-10-16 
+## [v5.1.2](https://github.com/prysmaticlabs/prysm/compare/v5.1.1...v5.1.2) - 2024-10-16
 
-This is a hotfix release with one change. 
+This is a hotfix release with one change.
 
 Prysm v5.1.1 contains an updated implementation of the beacon api streaming events endpoint. This
 new implementation contains a bug that can cause a panic in certain conditions. The issue is
@@ -102,20 +102,20 @@ prysm REST mode validator (a feature which requires the validator to be configur
 api intead of prysm's stock grpc endpoints) or accessory software that connects to the events api,
 like https://github.com/ethpandaops/ethereum-metrics-exporter
 
-### Fixed 
+### Fixed
 
 - Recover from panics when writing the event stream [#14545](https://github.com/prysmaticlabs/prysm/pull/14545)
 
 ## [v5.1.1](https://github.com/prysmaticlabs/prysm/compare/v5.1.0...v5.1.1) - 2024-10-15
 
-This release has a number of features and improvements. Most notably, the feature flag 
-`--enable-experimental-state` has been flipped to "opt out" via `--disable-experimental-state`. 
+This release has a number of features and improvements. Most notably, the feature flag
+`--enable-experimental-state` has been flipped to "opt out" via `--disable-experimental-state`.
 The experimental state management design has shown significant improvements in memory usage at
 runtime. Updates to libp2p's gossipsub have some bandwidith stability improvements with support for
-IDONTWANT control messages. 
+IDONTWANT control messages.
 
 The gRPC gateway has been deprecated from Prysm in this release. If you need JSON data, consider the
-standardized beacon-APIs. 
+standardized beacon-APIs.
 
 Updating to this release is recommended at your convenience.
 
@@ -157,7 +157,7 @@ Updating to this release is recommended at your convenience.
 - `grpc-gateway-corsdomain` is renamed to http-cors-domain. The old name can still be used as an alias.
 - `api-timeout` is changed from int flag to duration flag, default value updated.
 - Light client support: abstracted out the light client headers with different versions.
-- `ApplyToEveryValidator` has been changed to prevent misuse bugs, it takes a closure that takes a `ReadOnlyValidator` and returns a raw pointer to a `Validator`. 
+- `ApplyToEveryValidator` has been changed to prevent misuse bugs, it takes a closure that takes a `ReadOnlyValidator` and returns a raw pointer to a `Validator`.
 - Removed gorilla mux library and replaced it with net/http updates in go 1.22.
 - Clean up `ProposeBlock` for validator client to reduce cognitive scoring and enable further changes.
 - Updated k8s-io/client-go to v0.30.4 and k8s-io/apimachinery to v0.30.4
@@ -168,7 +168,7 @@ Updating to this release is recommended at your convenience.
 - Updated Sepolia bootnodes.
 - Make committee aware packing the default by deprecating `--enable-committee-aware-packing`.
 - Moved `ConvertKzgCommitmentToVersionedHash` to the `primitives` package.
-- Updated correlation penalty for EIP-7251. 
+- Updated correlation penalty for EIP-7251.
 
 ### Deprecated
 - `--disable-grpc-gateway` flag is deprecated due to grpc gateway removal.
@@ -641,34 +641,34 @@ AVX support (eg Celeron) after the Deneb fork. This is not an issue for mainnet.
 
 - Linter: Wastedassign linter enabled to improve code quality.
 - API Enhancements:
-    - Added payload return in Wei for /eth/v3/validator/blocks.
-    - Added Holesky Deneb Epoch for better epoch management.
+  - Added payload return in Wei for /eth/v3/validator/blocks.
+  - Added Holesky Deneb Epoch for better epoch management.
 - Testing Enhancements:
-    - Clear cache in tests of core helpers to ensure test reliability.
-    - Added Debug State Transition Method for improved debugging.
-    - Backfilling test: Enabled backfill in E2E tests for more comprehensive coverage.
+  - Clear cache in tests of core helpers to ensure test reliability.
+  - Added Debug State Transition Method for improved debugging.
+  - Backfilling test: Enabled backfill in E2E tests for more comprehensive coverage.
 - API Updates: Re-enabled jwt on keymanager API for enhanced security.
 - Logging Improvements: Enhanced block by root log for better traceability.
 - Validator Client Improvements:
-    - Added Spans to Core Validator Methods for enhanced monitoring.
-    - Improved readability in validator client code for better maintenance (various commits).
+  - Added Spans to Core Validator Methods for enhanced monitoring.
+  - Improved readability in validator client code for better maintenance (various commits).
 
 ### Changed
 
 - Optimizations and Refinements:
-    - Lowered resource usage in certain processes for efficiency.
-    - Moved blob rpc validation closer to peer read for optimized processing.
-    - Cleaned up validate beacon block code for clarity and efficiency.
-    - Updated Sepolia Deneb fork epoch for alignment with network changes.
-    - Changed blob latency metrics to milliseconds for more precise measurement.
-    - Altered getLegacyDatabaseLocation message for better clarity.
-    - Improved wait for activation method for enhanced performance.
-    - Capitalized Aggregated Unaggregated Attestations Log for consistency.
-    - Modified HistoricalRoots usage for accuracy.
-    - Adjusted checking of attribute emptiness for efficiency.
+  - Lowered resource usage in certain processes for efficiency.
+  - Moved blob rpc validation closer to peer read for optimized processing.
+  - Cleaned up validate beacon block code for clarity and efficiency.
+  - Updated Sepolia Deneb fork epoch for alignment with network changes.
+  - Changed blob latency metrics to milliseconds for more precise measurement.
+  - Altered getLegacyDatabaseLocation message for better clarity.
+  - Improved wait for activation method for enhanced performance.
+  - Capitalized Aggregated Unaggregated Attestations Log for consistency.
+  - Modified HistoricalRoots usage for accuracy.
+  - Adjusted checking of attribute emptiness for efficiency.
 - Database Management:
-    - Moved --db-backup-output-dir as a deprecated flag for database management simplification.
-    - Added the Ability to Defragment the Beacon State for improved database performance.
+  - Moved --db-backup-output-dir as a deprecated flag for database management simplification.
+  - Added the Ability to Defragment the Beacon State for improved database performance.
 - Dependency Update: Bumped quic-go version from 0.39.3 to 0.39.4 for up-to-date dependencies.
 
 ### Removed
@@ -679,12 +679,12 @@ AVX support (eg Celeron) after the Deneb fork. This is not an issue for mainnet.
 ### Fixed
 
 - Bug Fixes:
-    - Fixed off by one error for improved accuracy.
-    - Resolved small typo in error messages for clarity.
-    - Addressed minor issue in blsToExecChange validator for better validation.
-    - Corrected blobsidecar json tag for commitment inclusion proof.
-    - Fixed ssz post-requests content type check.
-    - Resolved issue with port logging in bootnode.
+  - Fixed off by one error for improved accuracy.
+  - Resolved small typo in error messages for clarity.
+  - Addressed minor issue in blsToExecChange validator for better validation.
+  - Corrected blobsidecar json tag for commitment inclusion proof.
+  - Fixed ssz post-requests content type check.
+  - Resolved issue with port logging in bootnode.
 - Test Fixes: Re-enabled Slasher E2E Test for more comprehensive testing.
 
 ### Security
@@ -1111,9 +1111,9 @@ No security issues in thsi release.
   now features runtime detection, automatically enabling optimized code paths if your CPU supports it.
 - **Multiarch Containers Preview Available**: multiarch (:wave: arm64 support :wave:) containers will be offered for
   preview at the following locations:
-    - Beacon Chain: [gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0](gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0)
-    - Validator: [gcr.io/prylabs-dev/prysm/validator:v4.1.0](gcr.io/prylabs-dev/prysm/validator:v4.1.0)
-    - Please note that in the next cycle, we will exclusively use these containers at the canonical URLs.
+  - Beacon Chain: [gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0](gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0)
+  - Validator: [gcr.io/prylabs-dev/prysm/validator:v4.1.0](gcr.io/prylabs-dev/prysm/validator:v4.1.0)
+  - Please note that in the next cycle, we will exclusively use these containers at the canonical URLs.
 
 ### Added
 

diff --git a/api/server/structs/conversions_lightclient.go b/api/server/structs/conversions_lightclient.go
@@ -171,9 +171,59 @@ func lightClientHeaderToJSON(header interfaces.LightClientHeader) (json.RawMessa
 			Execution:       execution,
 			ExecutionBranch: branchToJSON(executionBranch[:]),
 		}
+	case version.Electra:
+		exInterface, err := header.Execution()
+		if err != nil {
+			return nil, err
+		}
+		ex, ok := exInterface.Proto().(*enginev1.ExecutionPayloadHeaderElectra)
+		if !ok {
+			return nil, fmt.Errorf("execution data is not %T", &enginev1.ExecutionPayloadHeaderElectra{})
+		}
+		execution, err := ExecutionPayloadHeaderElectraFromConsensus(ex)
+		if err != nil {
+			return nil, err
+		}
+		executionBranch, err := header.ExecutionBranch()
+		if err != nil {
+			return nil, err
+		}
+		result = &LightClientHeaderDeneb{
+			Beacon:          BeaconBlockHeaderFromConsensus(header.Beacon()),
+			Execution:       execution,
+			ExecutionBranch: branchToJSON(executionBranch[:]),
+		}
 	default:
 		return nil, fmt.Errorf("unsupported header version %s", version.String(v))
 	}
 
 	return json.Marshal(result)
 }
+
+func LightClientBootstrapFromConsensus(bootstrap interfaces.LightClientBootstrap) (*LightClientBootstrap, error) {
+	header, err := lightClientHeaderToJSON(bootstrap.Header())
+	if err != nil {
+		return nil, errors.Wrap(err, "could not marshal light client header")
+	}
+
+	var scBranch [][32]byte
+	if bootstrap.Version() >= version.Electra {
+		b, err := bootstrap.CurrentSyncCommitteeBranchElectra()
+		if err != nil {
+			return nil, err
+		}
+		scBranch = b[:]
+	} else {
+		b, err := bootstrap.CurrentSyncCommitteeBranch()
+		if err != nil {
+			return nil, err
+		}
+		scBranch = b[:]
+	}
+
+	return &LightClientBootstrap{
+		Header:                     header,
+		CurrentSyncCommittee:       SyncCommitteeFromConsensus(bootstrap.CurrentSyncCommittee()),
+		CurrentSyncCommitteeBranch: branchToJSON(scBranch),
+	}, nil
+}
diff --git a/beacon-chain/blockchain/process_block.go b/beacon-chain/blockchain/process_block.go
@@ -68,6 +68,7 @@ func (s *Service) postBlockProcess(cfg *postBlockProcessConfig) error {
 		defer s.handleSecondFCUCall(cfg, fcuArgs)
 	}
 	defer s.sendLightClientFeeds(cfg)
+	defer s.saveLightClientUpdate(cfg)
 	defer s.sendStateFeedOnBlock(cfg)
 	defer reportProcessingTime(startTime)
 	defer reportAttestationInclusion(cfg.roblock.Block())

diff --git a/beacon-chain/blockchain/process_block_helpers.go b/beacon-chain/blockchain/process_block_helpers.go
@@ -5,12 +5,11 @@ import (
 	"fmt"
 	"time"
 
-	lightclient "github.com/prysmaticlabs/prysm/v5/beacon-chain/core/light-client"
-
 	"github.com/ethereum/go-ethereum/common"
 	"github.com/pkg/errors"
 	"github.com/prysmaticlabs/prysm/v5/beacon-chain/core/feed"
 	statefeed "github.com/prysmaticlabs/prysm/v5/beacon-chain/core/feed/state"
+	lightclient "github.com/prysmaticlabs/prysm/v5/beacon-chain/core/light-client"
 	"github.com/prysmaticlabs/prysm/v5/beacon-chain/core/transition"
 	doublylinkedtree "github.com/prysmaticlabs/prysm/v5/beacon-chain/forkchoice/doubly-linked-tree"
 	forkchoicetypes "github.com/prysmaticlabs/prysm/v5/beacon-chain/forkchoice/types"
@@ -129,6 +128,107 @@ func (s *Service) sendLightClientFeeds(cfg *postBlockProcessConfig) {
 	}
 }
 
+// saveLightClientUpdate saves the light client update for this block
+// if it's better than the already saved one, when feature flag is enabled.
+func (s *Service) saveLightClientUpdate(cfg *postBlockProcessConfig) {
+	//if !features.Get().EnableLightClient {
+	//	return
+	//}
+
+	attestedRoot := cfg.roblock.Block().ParentRoot()
+	attestedBlock, err := s.getBlock(cfg.ctx, attestedRoot)
+	if err != nil {
+		log.WithError(err).Error("Saving light client update failed: Could not get attested block")
+		return
+	}
+	if attestedBlock == nil || attestedBlock.IsNil() {
+		log.Error("Saving light client update failed: Attested block is nil")
+		return
+	}
+	attestedState, err := s.cfg.StateGen.StateByRoot(cfg.ctx, attestedRoot)
+	if err != nil {
+		log.WithError(err).Error("Saving light client update failed: Could not get attested state")
+		return
+	}
+	if attestedState == nil || attestedState.IsNil() {
+		log.Error("Saving light client update failed: Attested state is nil")
+		return
+	}
+
+	finalizedRoot := attestedState.FinalizedCheckpoint().Root
+	finalizedBlock, err := s.getBlock(cfg.ctx, [32]byte(finalizedRoot))
+	if err != nil {
+		log.WithError(err).Error("Saving light client update failed: Could not get finalized block")
+		return
+	}
+
+	update, err := lightclient.NewLightClientUpdateFromBeaconState(
+		cfg.ctx,
+		s.CurrentSlot(),
+		cfg.postState,
+		cfg.roblock,
+		attestedState,
+		attestedBlock,
+		finalizedBlock,
+	)
+	if err != nil {
+		log.WithError(err).Error("Saving light client update failed: Could not create light client update")
+		return
+	}
+
+	period := slots.SyncCommitteePeriod(slots.ToEpoch(attestedState.Slot()))
+
+	oldUpdate, err := s.cfg.BeaconDB.LightClientUpdate(cfg.ctx, period)
+	if err != nil {
+		log.WithError(err).Error("Saving light client update failed: Could not get current light client update")
+		return
+	}
+
+	if oldUpdate == nil {
+		if err := s.cfg.BeaconDB.SaveLightClientUpdate(cfg.ctx, period, update); err != nil {
+			log.WithError(err).Error("Saving light client update failed: Could not save light client update")
+		} else {
+			log.WithField("period", period).Debug("Saving light client update: Saved new update")
+		}
+		return
+	}
+
+	isNewUpdateBetter, err := lightclient.IsBetterUpdate(update, oldUpdate)
+	if err != nil {
+		log.WithError(err).Error("Saving light client update failed: Could not compare light client updates")
+		return
+	}
+
+	if isNewUpdateBetter {
+		if err := s.cfg.BeaconDB.SaveLightClientUpdate(cfg.ctx, period, update); err != nil {
+			log.WithError(err).Error("Saving light client update failed: Could not save light client update")
+		} else {
+			log.WithField("period", period).Debug("Saving light client update: Saved new update")
+		}
+	} else {
+		log.WithField("period", period).Debug("Saving light client update: New update is not better than the current one. Skipping save.")
+	}
+}
+
+// saveLightClientBootstrap saves a light client bootstrap for this block
+// when feature flag is enabled.
+func (s *Service) saveLightClientBootstrap(cfg *postBlockProcessConfig) {
+	//if !features.Get().EnableLightClient {
+	//	return
+	//}
+
+	blockRoot := cfg.roblock.Root()
+	bootstrap, err := lightclient.CreateLightClientBootstrap(cfg.ctx, s.CurrentSlot(), cfg.postState, cfg.roblock)
+	if err != nil {
+		log.WithError(err).Error("Saving light client bootstrap failed: Could not create light client bootstrap")
+		return
+	}
+	err = s.cfg.BeaconDB.SaveLightClientBootstrap(cfg.ctx, blockRoot[:], bootstrap)
+	if err != nil {
+		log.WithError(err).Error("Saving light client bootstrap failed: Could not save light client bootstrap in DB")
+	}
+}
+
 func (s *Service) tryPublishLightClientFinalityUpdate(
 	ctx context.Context,
 	signed interfaces.ReadOnlySignedBeaconBlock,