General discussion.
I'm using the built-in Alpaca preset in ST and this model keeps forgetting the closing quotes/asterisk. So far I find TheSpice 0.1.3 to still be the best L3-based finetune in terms of formatting. Tested Q6_K for both.
Temperature: 1.0
Min P: 0.05
Temperature last
From my experience this V2 of Infinity really likes plaintext actions and quotation dialogues, anything outside of that is gonna be a struggle.
I'm waiting for the newest TheSpice which should be coming soon, since cgato has been testing for a while.
Hello, this seems to be a really good model, from the texts I read it looks the most "natural" I've seen that can fit in a 6gb card and not being extremely slow...
But I'm having a instruction problems, it is taking the role of the user, speaking for myself, not following the character description, not following the character message format (if you want it to give you a list in every output for example), not advancing with the role play, refusing to say something at all cost.. for example the character says that something is worrying them and you ask what is it and it just avoid answering even if you insisting on it. If you just forget about it and continue the role play on your own describing actions for example it keeps saying the same thing.
Something like She tries to sound positive while her eyes betray the pain she feels deep down. and keeps sending the same kind of output followed by "I'm fine I just had a bad dream, don't worry about me" in every output and still refusing to tell what is it...
It really looks a good model in terms of output quality but for me it is just acting reaaally bad compared to other older models for example that is supposed to be way inferior than this.
Could you give me some tips on how to fix it?
I'm using InfinityRP-v2-8B-Q4_K_M-imat.gguf loaded on KoboldCPP v1.81.1 with 8192 context and 21 layers with SillyTavern UI
Text completions, Context template, Instruct and SYstem prompt I'm using your presets lewdicu-3.0.2-mistral-0.2
Specific response formatting can be tricky, I'll recommend adding a good number of example messages in your character (4-5) displaying the expected format if you need something specific so the model can more consistently match it. In the User Settings tab jn SillyTavern, scroll down and look for Example Messages and set it to "Always Include Examples".
Make sure you add more instructions explicitly in the System Prompt to make sure it's more included to try to advance the plot, and you can adjust the characters's Personality description to be more dynamic if needed.
There's no sure one shot solution, you'll need to work with the model to alleviate limitations considering its size.
You might want to increase Temperature a bit.
Maybe increase Repetition Penality, but for characters that contain a persistent stat table this could make that part inconstant so depends.
@Lewdiculous I see, thank you very much for the attention and tips. This is odd because other models do not have this problem of repetition and even follow the response format as how it is written on the character description, be it 7b or 8b Q4.. this is the first model that I test that have this issues.