Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize data-node PostgreSQL usage #10296

Open
daniel1302 opened this issue Dec 27, 2023 · 0 comments
Open

Optimize data-node PostgreSQL usage #10296

daniel1302 opened this issue Dec 27, 2023 · 0 comments
Labels
datanode feature new feature to be developed improvement
Milestone

Comments

@daniel1302
Copy link
Contributor

daniel1302 commented Dec 27, 2023

Feature Overview

We have the data node that uses PostgreSQL as a backend. It uses 2.2TB after 230 days of running the network.

It is hard to run the archival data node so far. But if the data increase is linear it will be 3TB every year. There is also not much traffic on the network at the moment.

However, most of the data comes from a single table - orders. The biggest tables are:

  • Orders 2.1TB
  • Market data: 320GB
  • Ledger: 90GB
  • Balances: 36GB

The rest tables are smaller.

Maybe We can think about optimizing the orders table? We can use some data pointers instead of raw data? We should think about it.

@daniel1302 daniel1302 added the feature new feature to be developed label Dec 27, 2023
@gordsport gordsport added this to the ⏭️ TBC milestone Jan 3, 2024
@gordsport gordsport moved this to Todo in Core Kanban Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datanode feature new feature to be developed improvement
Projects
Status: Todo
Development

No branches or pull requests

2 participants