Rhizome tasks breakdown and roadmapping

[bdehaynin]

Lists to be filled and discussed for the creation of the roadmap

Resources:

• https://talk.fission.codes/t/boris-quinn-riff-settling-on-rhizomedb/3707

Preliminary Questions

• What are our objectives for the next 6 months?
  • Support querying over end-to-end encrypted stores of content addressable facts
  • Build a standard library of CRDTs and utility functions, implemented using Rhizome
  • Provide an integration with ODD SDK for building collaborative apps
  • Expose a higher-level query DSL for more ergonomically working with JSON-shaped data
  • Automatically incrementalize queries
• What dependencies does rs-rhizome have?
  • Rust language

Research

What topics still need to be investigated?

• How tightly would we like to initially integrate Rhizome and ODD SDK together?
• Are there frontend frameworks, like React or Svelte, that we'd like to target deeper integrations with, in the short term?
• What higher-level DSL should we target?

Tasks to breakdown and prioritize

• Standard Library
  • Support importing rules into a program #38
  • Support registering aggregation functions for a program #39
  • CRDTs
    • Implement G-Counters in the standard library #40
    • Implement PN-Counters in the standard library #41
    • Implement G-Sets in the standard library #42
    • Implement 2P-Sets in the standard library #43
    • Implement OR-Sets in the standard library #44
    • Implement LWW-Sets in the standard library #45
    • Implement LWW-Registers in the standard library #46
    • Implement Sequences in the standard library #47
  • Aggregation
    • Implement Min in the standard library #48
    • Implement Max in the standard library #49
    • Implement Count in the standard library #50
    • Implement Sum in the standard library #51
    • Implement Mean in the standard library #52
  • Cryptographic
    • Implement Signing in the standard library #53
    • Implement Signature Verification in the standard library #54
    • Implement Hashing in the standard library #55
• Encrypted Data
  • Specify the data format for encrypted tuple stores #56
  • Implement encrypted tuple stores #57
• Indexing
  • Compute required indexes for a rule #58
  • Implement Hexastore-style relations for the EDB #59
  • Implement trie-based relations for IDB tuples #60
• Ergonomics
  • Support running ad-hoc goals against the database #61
  • Support querying the materialized view of the database #62
  • Track timestamped EDB snapshots for time travel #63
  • Implement HAMT-based unencrypted tuple stores #64
  • Improve diagnostics context for errors regarding invalid programs #65
• Incremental View Maintenance
  • Implement dataflow runtime #66
  • Implement differential trace data structures #67
  • Compile from RAM to dataflow #68
• WebAssembly bindings
  • Setup wasm-bindgen-test #69
  • Propagate and convert errors from Rust code to WebAssembly host #70
  • Expand WebAssembly bindings to support user defined predicates #71
  • Expand WebAssembly bindings to support aggregation #72
  • Expose WebAssembly bindings through a worker #73
  • Explore possible higher-level DSLs to expose through WebAssembly bindings #74
• Maintainability and Testing
  • Extract rhizome-wasm to a separate Cargo workspace #75
  • Add benchmark comparing naive and semi-naive evaluation #76
  • Add benchmark comparing evaluation with and without indexes #77
  • Define property testing generators for Rhizome programs #78
  • Add property test verifying that the order of rules in a program does not impact the results #79
  • Add property test verifying that all valid orderings of a rule body return the same results #80
  • Add metamorphic test verifying that duplicating a relational predicate with the same binding does not alter the results #81
  • Add metamorphic test verifying that removing rules can only decrease the number of strata #82

Milestones

1. Ship TodoMVC as an example app #83
2. Ship collaborative text editor as an example app #84
3. Expand collaborative text editor to include Upwelling style privacy controls #85
4. (maybe?) Ship ATProto feed generator
   • Demonstrates backend deployments for stream processing

Documentation to write

• Getting started tutorial
• HACKING.md for explaining the organization and architectural decisions in the codebase
• Example programs
  • Native Rust
  • WebAssembly