Quantization
#12
by
Zelyanoth
- opened
Hello, did anyone have some issue while trying to load the model in 4 or 8 bits ?
@Zelyanoth , Please have a look at this similar issue where user successfully quantized Gemma 3 27B to Q8 using llama.cpp. I hope this helps!