I tried to port this model into gguf to get INT8 quants with llama.cpp, but it seems to not be possible out of the box. Any plans to quantize pi0? Anyone with experience in AWQ wants to help me figure out this?
gguf
· Sign up or log in to comment
