Other Mistral Small based models?
Hey, if it's not much trouble could you please convert Gryphe/Pantheon-RP-1.8-24b-Small-3.1 and/or OddTheGreat/Core_24B_V.1 the same way as you converted the base Mistral Small model to create this DavidAU/Mistral-Small-3.1-24B-Instruct-2503-MAX-NEO-Imatrix-GGUF model? They are based on the same Mistral Small model, only with RP aspect added to it. Standard quants are already pretty good, but since I found your Max quant performing better on coding task than the standard quant of the same model which makes it feel smarter, I thought maybe we could get a real gem with those RP finetunes converted the same way.
Excellent.
Could you tell me what quant(s) you used?
Helps with testing/research.
RE: two models.
I will see what I can do ; there are two parts here:
1 - Max quants
2 - Neo Imatrix
Both affect overall quality of the quants ; with NEO having specific "coding" qualities to it.
For RP models, I may go with Horror Imatrix which works better for certain models VS NEO.
Imatrix in general can "fix" certain kinds of issues in a model.
Bottom line I test both NEO/HORROR, then pick the winner and upload.
Excellent.
Could you tell me what quant(s) you used?
Helps with testing/research.RE: two models.
I will see what I can do ; there are two parts here:1 - Max quants
2 - Neo ImatrixBoth affect overall quality of the quants ; with NEO having specific "coding" qualities to it.
For RP models, I may go with Horror Imatrix which works better for certain models VS NEO.
Imatrix in general can "fix" certain kinds of issues in a model.Bottom line I test both NEO/HORROR, then pick the winner and upload.
On this special quant I used Q3_K_S. On regular quants I was able to use Q4_K_S, but on your Max quant I went with the lower quant, because I was worried it wouldn't load or that it would be too slow to be useable, since it is slightly bigger than regular quants. But yeah that Q3_K_S turned out to be fairly useful on certain settings with very low temperature on that coding task I mentioned to you here https://huggingface.co/DavidAU/Reka-Flash-3-21B-Reasoning-Uncensored-MAX-NEO-Imatrix-GGUF/discussions/1#67e1fb0b62795b029687c10d.