We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
请教一下,看到wiki里的说明:如果负载均衡为random,则服务选择: 如果同一个模型包含多个服务实例,则会通过随机策略选择一个服务实例。,这里面的多个实例,是否检测model_redeirect设置?比如下面的config,客户端请求gpt-4o-mini的时候,会服从random的负载均衡设置吗?
random
服务选择: 如果同一个模型包含多个服务实例,则会通过随机策略选择一个服务实例。
gpt-4o-mini
... ... { "models": [ "cursor/gpt-4o", "cursor/gpt-4o-mini", "cursor/o1-mini" ], "enabled": true, "credentials": { "api_key": "AAA" }, "model_redirect": { "gpt-4o": "cursor/gpt-4o", "gpt-4o-mini": "cursor/gpt-4o-mini", "gpt-o1-mini": "cursor/o1-mini" }, "server_url":"http://1.1.1.1:18085/v1" }, { "models": [ "gpt-4o-mini" ], "enabled": true, "credentials": { "api_key": "BBB" }, "server_url":"http://2.2.2.2:18081/v1" } ... ...
The text was updated successfully, but these errors were encountered:
No branches or pull requests
请教一下,看到wiki里的说明:如果负载均衡为
random
,则服务选择: 如果同一个模型包含多个服务实例,则会通过随机策略选择一个服务实例。
,这里面的多个实例,是否检测model_redeirect设置?比如下面的config,客户端请求gpt-4o-mini
的时候,会服从random的负载均衡设置吗?The text was updated successfully, but these errors were encountered: