Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Centralize entity count tracking in EntityIdService #17316

Open
5 of 9 tasks
tinker-michaelj opened this issue Jan 10, 2025 · 0 comments · May be fixed by #17383
Open
5 of 9 tasks

Centralize entity count tracking in EntityIdService #17316

tinker-michaelj opened this issue Jan 10, 2025 · 0 comments · May be fixed by #17383
Assignees
Milestone

Comments

@tinker-michaelj
Copy link
Collaborator

tinker-michaelj commented Jan 10, 2025

Problem

Because the current implementation of KVStates is a dedicated VirtualMap, it is straightforward to implement Store.size() methods like this one as here:

    @Override
    public long sizeOfState() {
        return tokenState.size();
    }

As all state is consolidated into one VirtualMap (aka MegaMap), the effort of preserving the OnDiskKVState.size() method will no longer be a good cost/benefit ratio. This means Stores need a new implementation of their e.g. sizeOfState() methods.

Solution

  • Add a new singleton ENTITY_COUNTS state in EntityIdService that has a field (e.g. uint64 num_accounts or uint64 num_bytecodes) for each type of entity.
  • Add a 0.59-only step in SystemSetup.doPostUpgradeSetup() that initializes all these counts using direct calls to KVState.size() for every K/V state used by every service.
  • Update the WritableEntityIdStore API to "vend" entity id numbers by type, so that each time a service wants to create a new entity of a certain type, the EntityIdService can increment the correct ENTITY_COUNTS field.
  • Replace all Store.sizeOfState() implementations by getting a ReadableEntityIdStore and returning its e.g. numAccounts() or numBytecodes() getter as appropriate.
  • Switch StoreMetrics.updateCount() callbacks to be done by a StateChangeListener with stateTypes() == Set.of(SINGLETON) instead of in OnDiskWritableKVState.
  • Mark KVState.size() as deprecated and ensure there are no remaining non-test usages of this API.

Testing

  • Unit tests
  • Load a state and validate the counts are correctly updating in Entity_Counts map
  • All existing tests should pass
@tinker-michaelj tinker-michaelj added this to the v0.59 milestone Jan 10, 2025
@github-project-automation github-project-automation bot moved this to 📋 Backlog in Services Team Jan 10, 2025
@Neeharika-Sompalli Neeharika-Sompalli self-assigned this Jan 10, 2025
@Neeharika-Sompalli Neeharika-Sompalli moved this from 📋 Backlog to 👷🏼‍♀️ In Progress in Services Team Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 👷🏼‍♀️ In Progress
Development

Successfully merging a pull request may close this issue.

2 participants