Note says to use IT versions instead of GGUF versions?

#2
by chrisd37 - opened

The note below the first group of the QAT models uses a carat ^ to indicate relevance to models above while saying it's preferred to use the IT versions rather than the GGUF versions bc the GGUF versions are for llama.cpp and Ollama. But all of the models above the carat ^ are GGUF. I no comprendo. Was that note supposed to read something like "use the IT versions rather than the PT versions bc PT versions are for llama.cpp and ollama?" If not, can somebody explain bc I don't get it, sorry. Not trying to be a pain. I am just confused, probably because I'm very new to all of this.

chrisd37 changed discussion title from Note says to use GGUFs instead of IT versions? to Note says to use IT versions instead of GGUF versions?

EDIT oh nm, i get it: use the GGUF models with llama.cpp and Ollama, and then given the choice between IT and PT versions of the GGUFs, IT is preferred. lol @ me!

chrisd37 changed discussion status to closed

Sign up or log in to comment