mmproj file
If this model is multimodal why is there no mmproj file?
There is. You can even choose between Q8.0 and F16 mmproj quants:
- https://huggingface.co/mradermacher/Qwen2.5-VL-32B-Instruct-abliterated-GGUF/blob/main/Qwen2.5-VL-32B-Instruct-abliterated.mmproj-Q8_0.gguf
- https://huggingface.co/mradermacher/Qwen2.5-VL-32B-Instruct-abliterated-GGUF/blob/main/Qwen2.5-VL-32B-Instruct-abliterated.mmproj-f16.gguf
@mradermacher we realy need to improve mmproj visability as part of the README.md update project. We have too many users not finding them.
It's a done deal, yes (well, maybe not the readme update, but linking to the overview page), I just have to implement it, but right now, I don't have the necessary leisure.
Thank you for the information. That is very helpful. I had already found these mmproj files, but I didn't know that they can also be used with a different model and a different quantization method. Tips in the README are really helpful in this case.
Thanks for your answer.
Yeah, the readme is clearly designed to make choice easier if you already know what you are doing and is not helpful when new features are introduced (such as mmproj files). If we had a nice tutorial, we surely would link to that...