bartowski

/

mistral-community_pixtral-12b-GGUF

like 5

I wanted to see if there is a difference in performance between the fp16 and bf16 versions of the mmproj file, but the bf16 version simply gives an error and crashes. I am using the latest stable version of LM Studio (build 0.3.15) and the latest beta CUDA runtime (v1.29, llama.cpp b5219).

Is the text model only meant to work with the fp16 version of the mmproj file, or is it some bug that requires an update to fix? If it is the former, would it be better to remove the bf16 version, and maybe replace it with a fp32 version (assuming there is a benefit in doing so)?