Replies: 1 comment 1 reply
-
|
Probably won't be adding this on my own, unless llama.cpp also adds it, then koboldcpp will inherit their support. |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
makes sense. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Im sure you already seen it already but theres a another new model format. AWQ. Claims to be "blazing-fast" with much lower vram requirements. Looks like an almost 45% reduction in reqs.
Beta Was this translation helpful? Give feedback.
All reactions