mradermacher

/

MiMo-VL-7B-RL-GGUF

like 2

Works fine for me. Can you reproduce your problems with llama.cpp? If not,. it's likely an issue with LM Studio.

> Who are you?
Hello! I am MiMo, an AI assistant developed by Xiaomi Corporation. My purpose is to understand and respond to users' questions through advanced natural language processing technology, providing information queries, answering questions, and engaging in interesting conversations. I exist to help you solve various questions and provide an informative and entertaining conversational experience. If you have any questions or need help, feel free to let me know, and I will do my best to support you.

It seems that the model only provides that output when prompted with an image. Removing the MiMo-VL-7B-RL.mmproj-f16.gguf vision adapter from the folder will make the model work normally.
I downloaded the Q8 of the adapter and tried it, it worked.
The issue is with the f16.gguf.

Your report was about the Q8_0, not about the f16, but for the record, the f16 works just as well.

Apologies for my illiteracy here, but what is the solution? For leveraging input image vision on the RL model I get repeated G responses as well in both Q8_0 and f16 using LMStudio. Happy to test other quants

Apologies for my illiteracy here, but what is the solution? For leveraging input image vision on the RL model I get repeated G responses as well in both Q8_0 and f16 using LMStudio. Happy to test other quants

You use MiMo-VL-7B-RL.mmproj-Q8_0.gguf and any LLM quant you like using latest llama.cpp and don't use LMStudio.

Apologies for my illiteracy here, but what is the solution? For leveraging input image vision on the RL model I get repeated G responses as well in both Q8_0 and f16 using LMStudio. Happy to test other quants

Apologies for the belated reply. I didn't do much. I just replaced the MiMo-VL-7B-RL.mmproj-f16.gguf vision adapter with the MiMo-VL-7B-RL.mmproj-Q_8.gguf vision adapter and the vision capability worked.

Your report was about the Q8_0, not about the f16, but for the record, the f16 works just as well.

Mradermacher, I am not criticizing your quants nor complaining. As a token of gratitude for the tremendous work you do for us for free and as part of my responsibility, I reported an issue I faced that could possibly be related specifically to LM Studio.
Again, the model quantization level I am using is Q8_0, and the vision adapter was MiMo-VL-7B-RL.mmproj-f16.gguf. Replacing with MiMo-VL-7B-RL.mmproj-Q_8.ggu solved the issue, for me at least.

I hope this helps.

@Abdelhak the confusion is about what your problem is. First you talked about the Q8_0 not working for text input, then the f16, then the mmproj file. I have no clue what your issue is at this point :) Using llama.cpp instead of lm studio seems to be a good recommendation, though.

@Abdelhak the confusion is about what your problem is. First you talked about the Q8_0 not working for text input, then the f16, then the mmproj file. I have no clue what your issue is at this point :) Using llama.cpp instead of lm studio seems to be a good recommendation, though.

Apologies for the belated reply. I just saw your message.
OK, let me clarify. The first thing is I downloaded these two files:
1-MiMo-VL-7B-RL.Q8_0.gguf (the model)
2- MiMo-VL-7B-RL.mmproj-f16.gguf (the vision adapter)
I was interested in the vision capabilities of this model, so naturally, I prompted it with an image. That's when I got the Gs output.
To trouble shoot it, I deleted the MiMo-VL-7B-RL.mmproj-f16.gguf from the model folder, and prompted it with text only. I got a good answer.
Finally, I downloaded the MiMo-VL-7B-RL.mmproj-Q8_0.gguf file instead and included it into the model folder, and that seemed to solve the Gs issue when prompted with an image.

I hope this helps.

Again, thank you for your awesome work. I appreciate all the help you provide.

I see - thanks for the expkanation, and good that you could solve your issue, all without our help, too :)