Quantised Version
#19
by
steee
- opened
Hey!
Really appreciate all you've done with this - are there plans to release an official quantised/smaller version of the model? We were hoping to run it on an A100 40GB
Yes, I would also be interested in AWQ and FP8! Whatever is feasible.
Blessings