feat: atrium-repo

[DrChat]

This builds on str4d's work here: #168.

Big notes:

• Complete merkle search tree implementation (but still lacking fuzz tests)
• Complete Repository implementation with support for basic CRUD operations
• Added a new pair of traits for block storage: AsyncBlockStoreRead and AsyncBlockStoreWrite.
  • In-memory storage backend (primarily intended for testing)
  • CAR storage (supporting both reading and writing)
  • Users can implement other storage sources, like Azure-backed block storage.

Test coverage (cargo llvm-cov -p atrium-repo --html --open):

---

• Merkle Search Tree
  • Parse Node
    • Read Node from bytes
    • Verify depth and sort order
    • Limit the number of TreeEntrys per Node to a statistically unlikely maximum length.
    • Consider limiting the overall depth of the repo, or other parameters, to prevent more sophisticated key mining attacks.
  • Locate key within node
  • Locate keys within node with a given prefix
  • Add entry
  • Edit entry
  • Delete entry
• Repository
  • Load from a CAR file
  • Load from a firehose record
  • Read/write records inside of a repository
• Storage
  • CAR files
    • Reading CAR files
    • Verify completeness of the repository structure.
    • Robustness to both duplication and de-duplication of blocks.
    • Ignore any unnecessary or unlinked blocks.
  • Commits from firehose records

[DrChat]

Is it even possible to serialize a graph with a cycle?
The two nodes would point at each other, and by definition it should be impossible to compute their CIDs if there is a cycle.

[DrChat]

So I actually don't think this is possible, which means the extra validation logic here is probably optional and very unlikely to be hit in practice.
To err on the side of safety though, I think it would be best to leave this in-place.

[DrChat]

I think the PR is in a pretty good state for a MVP.
My time is limited so I won't be able to quickly iterate on the functionality for creating new commits for a repository. Still planning on it, but the work here can be pushed up in the meantime :)

[erlend-sh]

Pinging @str4d since this builds on their prior work.

[sugyan]

I have written several comments on areas unrelated to the internal logic. There are also a couple of points from clippy that I hope you can correct as well.

[DrChat]

Note: It looks like there's a pretty exhaustive MST test suite here that could probably be leveraged to validate this implementation.
https://github.com/DavidBuchanan314/mst-test-suite

[DrChat]

@sugyan Alright, I think this code is ready to be reviewed and merged :)

[sugyan]

I commented on a minor point regarding Cargo.toml, README, etc.

There seem to be a few warnings caused by cargo clippy and I'd like to see that fixed if possible.

I'd love to have @str4d review the implementation details if possible, but he's busy...?