Nobody quantized this model yet?
#1
by
ElvisM
- opened
Seems like one of the few interesting finetunes of Mistral 0.2. Surprised that it hasn't been quantized yet.
You can download the quantized version here : https://huggingface.co/LoneStriker/Mistral-7B-Instruct-v0.2-Neural-Story-GGUF
I just put an AWQ quant up here: https://huggingface.co/solidrust/Mistral-7B-Instruct-v0.2-Neural-Story-AWQ