1.3b
#7
by
jvwinc
- opened
Is there a 1.3b gguf?
You dont need it... it takes like 5gb to run it.
Wait.. what? 5gb? Is that Q_4?
1.3b at fp 16 takes my 3060 5gb vram to run
A set of GGUFs for the 1.3B would be very welcome!
You dont need it... it takes like 5gb to run it.
Its not about the vram per se, but there could be value in ggufs, since quantized versions generally run even faster, and since the high res fix seems to work, it could make sense to render a low res picture with the bigger model and then make it high res with the 1.3b model and with quantized it would speed that process up quite a bit.