Implement pruning point proof #1832

someone235 · 2021-10-17T13:18:23Z

…number of connections in the integration test to 32

codecov · 2021-10-17T13:32:41Z

Codecov Report

Merging #1832 (9ba63c2) into v0.11.0-dev (afaac28) will decrease coverage by 0.05%.
The diff coverage is 68.52%.

@@               Coverage Diff               @@
##           v0.11.0-dev    #1832      +/-   ##
===============================================
- Coverage        62.17%   62.11%   -0.06%     
===============================================
  Files              621      619       -2     
  Lines            28477    28748     +271     
===============================================
+ Hits             17705    17857     +152     
- Misses            8307     8370      +63     
- Partials          2465     2521      +56

Impacted Files	Coverage Δ
...ocesses/dagtraversalmanager/dagtraversalmanager.go	`77.58% <ø> (+2.58%)`	⬆️
domain/consensus/ruleerrors/rule_error.go	`87.87% <ø> (ø)`
domain/consensus/test_consensus_render_to_dot.go	`0.00% <0.00%> (ø)`
.../network/netadapter/server/grpcserver/p2pserver.go	`69.69% <ø> (ø)`
...rotocol/flows/blockrelay/ibd_with_headers_proof.go	`42.36% <37.50%> (-2.89%)`	⬇️
.../consensus/processes/dagtraversalmanager/window.go	`69.23% <60.00%> (-1.27%)`	⬇️
...ocesses/pruningproofmanager/pruningproofmanager.go	`62.02% <62.02%> (ø)`
.../pruning_violation_proof_of_work_and_difficulty.go	`68.25% <63.63%> (-5.66%)`	⬇️
domain/consensus/test_consensus_getters.go	`72.22% <71.42%> (-2.07%)`	⬇️
domain/consensus/consensus.go	`68.91% <74.35%> (-0.49%)`	⬇️
... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update afaac28...9ba63c2. Read the comment docs.

stasatdaglabs · 2021-10-25T05:15:38Z

testing/integration/64_incoming_connections_test.go

@@ -10,8 +10,8 @@ import (
 )

 func Test64IncomingConnections(t *testing.T) {
-	// Much more than 64 hosts creates a risk of running out of available file descriptors for leveldb
-	const numBullies = 64
+	// Much more than 32 hosts creates a risk of running out of available file descriptors for leveldb


Please update the number in this comment.

stasatdaglabs · 2021-10-25T05:17:06Z

domain/dagconfig/params.go

@@ -174,6 +174,8 @@ type Params struct {

 	// BaseSubsidy is the starting subsidy amount for mined blocks.
 	BaseSubsidy uint64
+
+	PruningProofM uint64


Please add a comment explaining what this is.

stasatdaglabs · 2021-10-25T05:17:28Z

infrastructure/network/connmanager/connmanager.go

@@ -91,6 +91,7 @@ func (c *ConnectionManager) Stop() {
 	}

 	c.loopTicker.Stop()
+	c.run()


Why did you add this?

This forces the next iteration in the connection loop, so you won't have to wait 30 seconds before the connection loop stops.

stasatdaglabs · 2021-10-25T05:23:55Z

domain/consensus/ruleerrors/rule_error.go

+	ErrPruningProofSelectedTipIsNotThePruningPoint    = newRuleError("ErrPruningProofSelectedTipIsNotThePruningPoint")
+	ErrPruningProofInsufficientBlueWork               = newRuleError("ErrPruningProofInsufficientBlueWork")
+	ErrPruningProofMissingBlockAtDepthMFromNextLevel  = newRuleError("ErrPruningProofMissingBlockAtDepthMFromNextLevel")
+	ErrPruningProofMissesBlocksBelowPruningPoint      = newRuleError("ErrPruningProofMissesBlocksBelowPruningPoint")


Weren't these elsewhere before? Why did you copy them here?

What do you mean? I didn't copy them

Ah, sorry, brain fart. Nevermind.

stasatdaglabs · 2021-10-25T05:31:01Z

domain/consensus/processes/dagtraversalmanager/window.go

+			return nil, err
+		}
+
+		if !isNotFoundError {


Is there any other time except for the virtual genesis when the current block would not be found?

The point is that we always prefer to use the data from daaWindowStore if available, because when we import blocks with trusted data we change their ghostdata data and drop from the merge set every pruned header. So if we rely on the ghostdag data of blocks with trusted data when calculating difficulty we'll get the wrong difficulty.

stasatdaglabs · 2021-10-25T06:15:21Z

domain/consensus/processes/pruningproofmanager/pruningproofmanager.go

+			}
+
+			visited.Add(current)
+			isAncestorOfPruningPoint, err := ppm.dagTopologyManagers[blockLevel].IsAncestorOf(stagingArea, current, selectedTip)


The selected tip is not necessarily the pruning point. Please rename this variable.

stasatdaglabs · 2021-10-26T06:06:02Z

infrastructure/network/connmanager/connmanager.go

@@ -91,7 +91,7 @@ func (c *ConnectionManager) Stop() {
 	}

 	c.loopTicker.Stop()
-	// Force the next iteration so the connection loop will stop immediately and not after `connectionsLoopInterval`.
+	// Force the next iteration so the connection loop will stop immediately and not after `connectionsLoopInterval`ErrPruningProofMissesBlocksBelowPruningPoint.


Copy+paste error.

This reverts commit bca1080.

someone235 added 30 commits September 13, 2021 15:58

Calculate GHOSTDAG, reachability etc for each level

21827c4

Don't preallocate cache for dag stores except level 0 and reduce the …

204f790

…number of connections in the integration test to 32

Reduce the number of connections in the integration test to 16

70ba9c8

Increase page file

0d8980b

BuildPruningPointProof

a183a94

BuildPruningPointProof

6f4a9c5

Add PruningProofManager

a0c6cf0

Implement ApplyPruningPointProof

367dcbb

Add prefix and fix blockAtDepth and fill headersByLevel

d78e2d7

Some bug fixes

ba12089

Include all relevant blocks for each level in the proof

fc706b4

Fix syncAndValidatePruningPointProof to return the right block hash

ec499fb

Fix block window

9028c01

Fix isAncestorOfPruningPoint

0fa5635

Ban for rule errors on pruning proof

9c624cb

Find common ancestor for blockAtDepthMAtNextLevel

1f639a9

Use pruning proof in TestValidateAndInsertImportedPruningPoint

59382c4

stage status and finality point for proof blocks

1c5e120

Uncomment golint

18a4c52

Change test timeouts

0017617

Calculate merge set for ApplyPruningPointProof

8ac5a48

Increase test timeout

911484d

Add better caching for daa window store

8bf65b2

Return to default timeout

8ae689d

Add ErrPruningProofMissesBlocksBelowPruningPoint

7b08552

Add errDAAWindowBlockNotFound

da19b88

Force connection loop next iteration on connection manager stop

e2a7c73

Revert to Test64IncomingConnections

5edc4e2

Remove BlockAtDepth from DAGTraversalManager

951792a

numBullies->16

ccf60ae

Set page file size to 8gb

1b1fb05

Increase p2p max message size

c3da6a3

stasatdaglabs requested changes Oct 25, 2021

View reviewed changes

someone235 added 6 commits October 25, 2021 18:52

Test64IncomingConnections->Test16IncomingConnections

4a3b72f

Add comment for PruningProofM

c5f5f9d

Add comment in func (c *ConnectionManager) Stop()

c72f563

Rename isAncestorOfPruningPoint->isAncestorOfSelectedTip

8256318

Revert page file to 16gb

c9f7c93

Improve ExpectedHeaderPruningPoint perf

bca1080

stasatdaglabs requested changes Oct 26, 2021

View reviewed changes

someone235 added 2 commits October 26, 2021 09:08

Fix comment

2a761e1

Revert "Improve ExpectedHeaderPruningPoint perf"

885ca57

This reverts commit bca1080.

stasatdaglabs previously approved these changes Oct 26, 2021

View reviewed changes

Don't test windows

9ba63c2

someone235 dismissed stasatdaglabs’s stale review via 9ba63c2 October 26, 2021 06:24

stasatdaglabs approved these changes Oct 26, 2021

View reviewed changes

someone235 merged commit 5dbb1da into kaspanet:v0.11.0-dev Oct 26, 2021

someone235 deleted the build-dag-per-level branch October 26, 2021 06:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement pruning point proof #1832

Implement pruning point proof #1832

someone235 commented Oct 17, 2021 •

edited

Loading

codecov bot commented Oct 17, 2021 •

edited

Loading

stasatdaglabs Oct 25, 2021

stasatdaglabs Oct 25, 2021

stasatdaglabs Oct 25, 2021

someone235 Oct 25, 2021

stasatdaglabs Oct 25, 2021

someone235 Oct 25, 2021

stasatdaglabs Oct 26, 2021

stasatdaglabs Oct 25, 2021

someone235 Oct 25, 2021

stasatdaglabs Oct 25, 2021

stasatdaglabs Oct 26, 2021

Implement pruning point proof #1832

Implement pruning point proof #1832

Conversation

someone235 commented Oct 17, 2021 • edited Loading

codecov bot commented Oct 17, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

someone235 commented Oct 17, 2021 •

edited

Loading

codecov bot commented Oct 17, 2021 •

edited

Loading