Mixture of Agents

[hazkun]

is there a chance in the future to build MoA as built in feature in koboldcpp ? or it has to be on llamacpp first? running 3 instance of 8B model seems better than running a 34B model.

[LostRuins]

You should be able to make a program to do it already, just run multiple instances of koboldcpp with different models on individual ports, and grab the responses from each of them, then pick the best and give that to the user.