Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CosyVoice2使用trt加速，显存得增加4.5G左右，降低显存占用需要做什么特殊配置吗？ #835

Open
0xCAFEBABE0 opened this issue Jan 6, 2025 · 1 comment
Open

CosyVoice2使用trt加速，显存得增加4.5G左右，降低显存占用需要做什么特殊配置吗？ #835

0xCAFEBABE0 opened this issue Jan 6, 2025 · 1 comment

Comments

@0xCAFEBABE0
Copy link
Contributor

[01/06/2025-15:35:21] [TRT] [I] Loaded engine size: 158 MiB
[01/06/2025-15:35:22] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +4532, now: CPU 0, GPU 4668 (MiB)

如果切换至CosyVoice1进行trt加速，显存大概增加300M，CosyVoice2上的trt需要做什么特殊配置吗？
@aluminumbox
Copy link
Collaborator

可以指定fp16=True，我们没有做什么特殊配置，可以查一下trt官网有没有什么建议

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@0xCAFEBABE0 @aluminumbox