What is 'Use QuantMatMul'? #361
StripedPuppy
started this conversation in
General
Replies: 1 comment
-
|
I'll add it to the wiki soon. It's basically a new approach for prompt processing from upstream. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I can't seem to find documentation anywhere on the net. I'm just not sure if I should mess with it or not.
Preset: CuBLAS
GPU: Nvidia RTX-3060
CPU: Intel i7-12700
Model: Mostly 7b models at 8_0 quant
Beta Was this translation helpful? Give feedback.
All reactions