[Feature Request]: Multi-Replica documentation

[tkg61]

Describe the problem

i was assuming if i used this chart it would setup the environment to where if more than 1 replica was in use, they would be synced to each other. it seems that they are not, i would like to have this documented so that others do not go down a rabbit hole like i did.

Describe the proposed solution

Document that multi-replicas is not the same as a high availability cluster and it is up to the developer to sync the databases manually

Alternatives considered

No response

Importance

I cannot run the chart without it

Additional Information

No response

[tazarov]

hey @tkg61, the chart in current state runs single-node Chroma. The downside of this is lack of distributed storage support.

Single-node Chroma relies on HNSW and sqlite3 for its embeddings and metadata storage, none of those are process-safe, e.g. multiple replicas reading/writing to the same set of files.

Hosted and distributed Chroma has recently been released. What I can do is to include a distributed Chroma in this chart which will solve your problem, although distributed Chroma does not have HA in the traditional sense of the word.

Another approach in a scenario where active-passive HA is an ok approach then I can help you with a Envoy plugin that can manage the replication of the traffic to one or more replicas.

There's also a storage-based replication that can be setup with something like longhorn so that your storage has a secondary replica on another node where in the event of failure of primary Chroma server you can then fallback to by attaching the replicated storage.