HA ARM quant
#1
by
EloyOn
- opened
You don't think this tiny model is worthy of being given the HA ARM treatment like the other ones?
I didn't try it but 1b must struggle to be coherent even at q8_0.
It's 1B, so even at FP16 it's not that coherent, so yeah, it wouldn't make much of a difference.
With 2day's hardware, even a mid tier CPU could probably run 3B model with ease.