Failed to load model
Failed to load model
error loading model: error loading model architecture: unknown model architecture: 'glm4'
@bartowski
I assume with lm studio?
Should be a beta runtime out tonight or tomorrow that works
@bartowski Will probably need to requantize, see: https://github.com/ggml-org/llama.cpp/pull/12957
I'm doing a quant on the PR version right now, going to see if it works.
Please report back!
Want to leave these up for discussion, hate leaving them up cause people will download and have issues 😩😩
last comments https://github.com/ggml-org/llama.cpp/issues/12946
@bartowski my models are up here https://huggingface.co/ilintar/THUDM_GLM-Z1-9B-0414_iGGUF (just IQ4_NL, Q5_K_M and Q8 because my upload sucks) and they seem to work well with the llama.cpp branch from the pull request as well. There were changes both in the core and in the converter, so I think you need both (the new quants and the new llama.cpp engine) to run it.
ah gotcha, shame it requires both, but thanks for confirming!
I want to get on these ASAP but I'm also away from my PC so it's not easy for me to correct if something goes wrong again, so for now I think I'll wait until it's finalized and merged
Yeah, makes sense esp. since there is still some problem with the /props endpoint and I don't know if that's just a backend issue or if something's still wrong with the conversion, so no use reuploading only to have to pull it yet again :>