Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: apply cronos store #207

Merged

Conversation

dudong2
Copy link

@dudong2 dudong2 commented Oct 30, 2024

Description

Closes: #XXXX


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

yihuang and others added 12 commits October 30, 2024 14:26
generic interface

generic btree

generic cachekv

generic transient store

support ObjStore

changelog

Update CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

object store key

Apply review suggestions

fix merge conflict

fix snapshot

revert dependers
Solution:
- remove the api

changelog
* Problem: MultiStore interface is bloated

Solution:
- Split out specialied methods from it, keeping the MultiStore generic

* Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>
* Problem: nested cache store not efficient

Solution:
- introduce copy-on-write btree based cache store

temp

* changelog

* rename

* Update store/cachekv/store.go

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>
Solution:
- init cachestore on cachestore lazily.
- cleanup some unused stuff.

Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>
* Support RunAtomic API

* add unit test
Solution:
- fix and add test to support the usage pattern in ethermint

add Discard method to CacheWrap

better testing
* feat: save some memory allocations on un-used cache store

* Update CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

* Update store/internal/btree/btree.go

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>
…258)

Solution:
- add API NewFromParent to cache multistore.

Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

fix test

fix lint
Copy link

@dudong2 your pull request is missing a changelog!

Comment on lines +137 to +139
for _, store := range cms.stores {
store.Discard()
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
store = cms.initStore(key, cms.parentStore(key))
}
if key == nil || store == nil {
panic(fmt.Sprintf("kv store with key %v has not been registered in stores", key))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
panic(fmt.Sprintf("kv store with key %v has not been registered in stores", key))
store, ok := cms.getCacheWrapper(key).(types.KVStore)
if !ok {
panic(fmt.Sprintf("store with key %v is not KVStore", key))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
Comment on lines +193 to +195
for k, v := range cms.stores {
stores[k] = v.(types.BranchStore).Clone().(types.CacheWrap)
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
Comment on lines +212 to +217
for k, v := range cms.stores {
if _, ok := other.stores[k]; !ok {
// clear the cache store if it's not in the other
v.Discard()
}
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
Comment on lines +220 to +227
for k, v := range other.stores {
store, ok := cms.stores[k]
if !ok {
store = cms.initStore(k, cms.parentStore(k))
}

store.(types.BranchStore).Restore(v.(types.BranchStore))
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
func (ts *BTreeStore[V]) Iterator(start, end []byte) types.GIterator[V] {
it, err := ts.BTree.Iterator(start, end)
if err != nil {
panic(err)

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
func (ts *BTreeStore[V]) ReverseIterator(start, end []byte) types.GIterator[V] {
it, err := ts.BTree.ReverseIterator(start, end)
if err != nil {
panic(err)

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
store := types.KVStore(s)
store, ok := s.(types.KVStore)
if !ok {
panic(fmt.Sprintf("store with key %v is not KVStore", key))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
// AssertValidValueLength checks if the value length is within length limit
func AssertValidValueLength(l int) {
if l > MaxValueLength {
panic(errors.New("value is too large"))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
@dudong2 dudong2 self-assigned this Oct 30, 2024
…#588)

* Problem: versiondb/memiavl don't compatible with upstream sdk

Solution:
- avoid api breaking change

* fix build
@dudong2 dudong2 force-pushed the feat/apply-cronos-store branch from 67e8bef to 539dfed Compare November 6, 2024 04:25
@dudong2 dudong2 requested review from dongsam and cloudgray November 6, 2024 04:47
Copy link
Collaborator

@cloudgray cloudgray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dudong2 dudong2 merged commit a25eb3a into basechain/develop-v0.50.8-store/v1.1.0 Nov 6, 2024
41 of 44 checks passed
dudong2 added a commit that referenced this pull request Dec 17, 2024
* Support object store (#206)

generic interface

generic btree

generic cachekv

generic transient store

support ObjStore

changelog

Update CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

object store key

Apply review suggestions

fix merge conflict

fix snapshot

revert dependers

* Problem: CacheWrapWithTrace api is not used (#207)

Solution:
- remove the api

changelog

* prefix store support object store (#236)

* Problem: MultiStore interface is bloated (#240)

* Problem: MultiStore interface is bloated

Solution:
- Split out specialied methods from it, keeping the MultiStore generic

* Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>

* Problem: nested cache store not efficient (#241)

* Problem: nested cache store not efficient

Solution:
- introduce copy-on-write btree based cache store

temp

* changelog

* rename

* Update store/cachekv/store.go

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>

* Problem: many memory allocs in cache store wrapping (#242)

Solution:
- init cachestore on cachestore lazily.
- cleanup some unused stuff.

Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

* Problem: no API to use the new CoW branch store (#243)

* Support RunAtomic API

* add unit test

* Problem: store key type assertion is incorrect (#244)

fix and add test

* Problem: Restore don't work snapshot revert usage (#245)

Solution:
- fix and add test to support the usage pattern in ethermint

add Discard method to CacheWrap

better testing

* feat: save some memory allocations on un-used cache store (#248)

* feat: save some memory allocations on un-used cache store

* Update CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

* Update store/internal/btree/btree.go

Signed-off-by: yihuang <[email protected]>

---------

Signed-off-by: yihuang <[email protected]>

* Problem: no api to create cachemulti store from external cache store (#258)

Solution:
- add API NewFromParent to cache multistore.

Update store/CHANGELOG.md

Signed-off-by: yihuang <[email protected]>

fix test

fix lint

* Problem: snapshot for object store is not skipped (cosmos#585)

* Problem: versiondb/memiavl don't compatible with upstream sdk (cosmos#588)

* Problem: versiondb/memiavl don't compatible with upstream sdk

Solution:
- avoid api breaking change

* fix build

* chore: Update test.yml go version 1.21

* chore: fix lint

---------

Signed-off-by: yihuang <[email protected]>
Co-authored-by: yihuang <[email protected]>
Co-authored-by: mmsqe <[email protected]>
Co-authored-by: Kyuhyeon Choi <[email protected]>
dudong2 added a commit that referenced this pull request Dec 19, 2024
* Support object store (#206)

generic interface

generic btree

generic cachekv

generic transient store

support ObjStore

changelog

Update CHANGELOG.md



object store key

Apply review suggestions

fix merge conflict

fix snapshot

revert dependers

* Problem: CacheWrapWithTrace api is not used (#207)

Solution:
- remove the api

changelog

* prefix store support object store (#236)

* Problem: MultiStore interface is bloated (#240)

* Problem: MultiStore interface is bloated

Solution:
- Split out specialied methods from it, keeping the MultiStore generic

* Update store/CHANGELOG.md



---------



* Problem: nested cache store not efficient (#241)

* Problem: nested cache store not efficient

Solution:
- introduce copy-on-write btree based cache store

temp

* changelog

* rename

* Update store/cachekv/store.go



---------



* Problem: many memory allocs in cache store wrapping (#242)

Solution:
- init cachestore on cachestore lazily.
- cleanup some unused stuff.

Update store/CHANGELOG.md



* Problem: no API to use the new CoW branch store (#243)

* Support RunAtomic API

* add unit test

* Problem: store key type assertion is incorrect (#244)

fix and add test

* Problem: Restore don't work snapshot revert usage (#245)

Solution:
- fix and add test to support the usage pattern in ethermint

add Discard method to CacheWrap

better testing

* feat: save some memory allocations on un-used cache store (#248)

* feat: save some memory allocations on un-used cache store

* Update CHANGELOG.md



* Update store/internal/btree/btree.go



---------



* Problem: no api to create cachemulti store from external cache store (#258)

Solution:
- add API NewFromParent to cache multistore.

Update store/CHANGELOG.md



fix test

fix lint

* Problem: snapshot for object store is not skipped (cosmos#585)

* Problem: versiondb/memiavl don't compatible with upstream sdk (cosmos#588)

* Problem: versiondb/memiavl don't compatible with upstream sdk

Solution:
- avoid api breaking change

* fix build

* chore: Update test.yml go version 1.21

* chore: fix lint

---------

Signed-off-by: yihuang <[email protected]>
Co-authored-by: yihuang <[email protected]>
Co-authored-by: mmsqe <[email protected]>
Co-authored-by: Kyuhyeon Choi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants