Possibility of an AWQ version?
#1
by
fredericodeveloper
- opened
Hey Ton, really appreciate you taking the time to quantize and share the model! Any chance you could also release an AWQ version when you get a moment? Love what you're doing, thanks again!
Uploading :)
Hey Ton, really appreciate you taking the time to quantize and share the model! Any chance you could also release an AWQ version when you get a moment? Love what you're doing, thanks again!
Is AWQ better than GPTQ-4bit?
In short, yes.
In long, very possibly.
There might not be enough, and even contradictory, evidences to conclude that AWQ is better than GPTQ. There are reddit posts and comments, and even my experiences, that confirms that AWQ is better than GPTQ. But there was also a blog from 2 years ago said otherwise.