alamios

/

Mistral-Small-3.1-DRAFT-0.5B

like 20

you're welcome!

Is there something similar for gemma 3 27b? I actually tried gemma3 1B model as draft in lm studio, but it works even slower than without it. Maybe it's because of my slow hardware.

I'll try, but I don't think a light finetune would work better than their model. Anyway, stay tuned!

I don't think it works, but I wish this draft also worked with the first Mistral Small release. I dislike the 3.1 version for creative writing - it's just too boring. Also haven't had any luck using Gemma 1b as a draft model.

@ElvisM are you talking the 2501 or the 2409? I haven't tested the 2409 but I believe it works fine with the 2501.

@ElvisM are you talking the 2501 or the 2409? I haven't tested the 2409 but I believe it works fine with the 2501.

2409. Tested and confirmed that it doesn't work. LM Studio doesn't even allow to select it.

@ElvisM you can probably do a transplant.... Actually.. I have it all set up here, I just created a QwenPhi 0.5b, I will try here to make a transplant for the 2409 vocab for this draft model here.

I love this @alamios idea... it is working great here.

@ElvisM actually I am not so sure if I can actually do it... Because the 2409 is distributed under the mrl licence and the Qwen 2.5 is apache, I am trying to understand the compatibility of the mrl with the apache-2 license. I think if I include both licences and distribute under the mrl it is ok , but I will give it another read tomorrow, before releasing it.