RL Swarm is a fully open source framework for creating RL training swarms over the internet. Running a swarm node allows you to launch a new swarm; alternatively, you can connect to an existing swarm by peering with the public address of one of its constituent nodes.
Each swarm performs RL reasoning as a group, with a gossiping system (using Hivemind) for collaborative improvement between models.
RL Swarm is fully open and permissionless, meaning you can run it on a basic consumer laptop at home or on a powerful GPU in the cloud.
Note that this code is experimental - particularly on arm64 architectures.
Ensure you that you are using a supported machine/device/environment:
- arm64 CPU with minimum 16gb ram
- CUDA devices (officially supported):
- RTX 3090
- RTX 4090
- A100
- H100
- Python >=3.10 (for Mac, you will likely need to upgrade)
python3 -m venv .venv
source .venv/bin/activate
./run_rl_swarm.shIf you encounter issues with the coordinator peer, try this backup peer node:
DEFAULT_PEER_MULTI_ADDRS="/dns/rl-swarm.gensyn.ai/tcp/38331/p2p/QmQ2gEXoPJg6iMBSUFWGzAabS2VhnzuS782Y637hGjfsRJ" # gensyn coordinator node
(Experimental) fix to increase memory on macbook:
export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0
If you have issues running the above on your development machine or prefer
not to install dependencies locally, we offer a public Docker
image that is ready to run out-of-the-box.
Ensure your Docker engine is configured to allow
adequate space and memory (under System-->Resources) or you might see it being Killed. If you don't have a GPU, remove the --gpus all flag.
docker run --gpus all --pull=always -it --rm europe-docker.pkg.dev/gensyn-public-b7d9/public/rl-swarm:v0.0.1 ./run_hivemind_docker.sh