Addressing cardano-db-sync resource waste
#102
Replies: 3 comments 9 replies
-
|
I'm strongly opinionated about this. So feel free to take all my points with a bit of salt. The DID PRISM method. Lives on the Cardano transaction's metadata. You can either use another DID method or create one. Either way for the Identus PoV you should care about supporting for other DID Methods #79 They are Blockchain that were designed only for DID methods but they failed to get adoption. IMO they were not decentralized enough for because of that. It's not that the ideas and intention to make a SSI was not there. In my PoV, I agree that running the PRISM Node is very expensive, a bit more that running a Cardano Node (because it's running one...). But they are people all over the world running their own Cardano Nodes. This is what it is makes the Cardano blockchain generally available and decentralized. Those people still make money from running the Node. The So for a moment, ignore the software PRISM Node. If this person creates a staking pool. Would I convince you to join that person staking pool If I offer the promise of have a endpoint that you can query by a very convenient index? You just need to trust that in endpoint returns you all the data and not only partially. Basically, the same trust that you put on any lightweight wallet for any blockchain. I mean, the idea is not new. I didn't invented. It's not even hard to do it! That was reading path. The writing path is even easier. Because it already done!! At some point, I was thinking about... (╯°□°)╯︵ ┻━┻ Don't take me wrong. I'm very grateful for the opportunity given by IOHK (IOG) to work on open source. Specifically close to this vision that I had about what I wanted SSI to be. Long before heard about the term SSI for the first time. Too much? Maybe. I am still trying to sell this idea through the VRD's specification and implementation. On IOHK company's working hours and a bit from the Catalyst project 1300189. I want to believe this will bring adoption to the PRISM DID, Identus and the Cardano Blockchain. |
Beta Was this translation helpful? Give feedback.
-
|
So I put some working hours, long nights and weekends and I come up with this: https://github.com/FabioPinheiro/prism-vdr I'm still testing this indexer and need to fix some things. |
Beta Was this translation helpful? Give feedback.
-
|
Could you pls have a Readme.md so we can try out the nice work you have done ? |
Beta Was this translation helpful? Give feedback.
-
|
i do not see any indexer code, only the output. Will that be made available as well ? Would that be a separate process that can be connected to the cardano-node and a postgres db ? |
Beta Was this translation helpful? Give feedback.
-
|
Yes I will make it available |
Beta Was this translation helpful? Give feedback.
-
|
The draft implementation it's here: All the pipeline logic of this indexer is on the file |
Beta Was this translation helpful? Give feedback.
-
|
@FabioPinheiro Are we talking about Cardano DB-Sync or PrismNode? Cardano DB-Sync is simply an indexer that indexes everything on the blockchain, whereas PrismNode is specifically tailored for Prism-related functionality. If we need a customized DB-Sync, we can create one that indexes only Prism-specific labels, but it would still need to process the entire blockchain. |
Beta Was this translation helpful? Give feedback.
-
|
Yes but my point is that not everyone needs to run the whole infrastructure. The same way as people with light wallets you don't run the Cardano Node. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, a DB-Sync needs to process the entire chain, it will interface with a cardano node to read the ledger, but then, instead of saving everything into a database, how can we make it save just PRISM DIDs? that way the infra requirements to run a cloud agent will be a lot leaner. Basically, it needs to process the entire chain but it doesn't need to copy the entire chain into the database for the purpose of identifying when a DID gets published, that's where I see an opening to figure out something to make that happen. |
Beta Was this translation helpful? Give feedback.
-
|
Yes. We just need a separate db-sync-did process, that does not process all data, except the DID. Then we can create periodic snapshot of the output of this process, which contains only the DIDs on chain. This is a good suggestion. Thus a dedicated indexer of for metadata label |
Beta Was this translation helpful? Give feedback.
-
|
I have discussed this point already with @robertocarvajal. Why don't we propose a new CIP where we create a smart contract where utxos can be created related to the DIDs. A single address can in principle hold infinite utxos (with their associated datums holding the DIDs). We could even have a host of addresses holding the DIDs. Then a merkle tree based proof can be constructed. (https://github.com/aiken-lang/merkle-patricia-forestry). As long as this merkle trie is updated for every new utxo commit, we have a more succinct way of dealing with DIDs on chain. |
Beta Was this translation helpful? Give feedback.
-
|
That sounds like a good proposal for a new DID |
Beta Was this translation helpful? Give feedback.
-
By far
cardano-db-syncis the most resource intensive part of the infrastructure required for production, the big problem is that it is agnostic to whoever is querying and its job is to make the entire blockchain available over a database which is extremely wasteful in our case because we are only using it to fetchDIDsthat are published on chain, plus it has its own internal full node that is not shared with the wallet interface by default but that's another topic.Starting on a recent epoch seems to be the only way to limit the amount of data that
cardano-db-syncwill store from the blockchain.What can we do about it? I have a few ideas but I'd like to open the discussion to the rest of the community:
I'm interested to brainstorm ideas on how to solve this.
Beta Was this translation helpful? Give feedback.
All reactions