Can you produce a 4.5bpw quantized model of this model?
#1
by
xldistance
- opened
Can you produce a 4.5bpw quantized model of this model?
@xldistance done.
https://huggingface.co/matatonic/Qwen2.5-72B-Instruct-abliterated-v2-4.5bpw-exl2
Recommend you use it with this draft model also:
https://huggingface.co/cgus/Qwen2.5-0.5B-Instruct-exl2/tree/6.5bpw-h8
Thank you so much.
matatonic
changed discussion status to
closed