Instruct Template?
Hey! What's the instruct template for this model please, it's impossible to find... for me it runs absolutely terrible, talks for the user, stutters in its text, merging up words on its first response... hope a correct instruct can fix it. Thanks!
This model works with the Jijna template (embedded in the GGUFs).
In Lmstudio this will load automatically, as it also should in using Llama-Server.exe (Llamacpp Guthub).
You may be able to use Llama 3 instruct as a fall back.
In SillyTavern - use "text completion" - this should then use the Jinja Template.
To construct a template manually:
Contact the org model maker and open a ticket in the "community tab":
https://huggingface.co/RekaAI/reka-flash-3
Also I've tried it with Mistral template and it works too. This model is worth the extra hassle of setting it up. Trades blows with QWQ in terms of intelligence and the writing style is shockingly natural.
Excellent. ;
What about KoboldCpp??
I am using Koboldcpp backend + SillyTavern frontend and have selected 'text completion' and API type as 'koboldcpp' and in both the instruct and context template, I have checked "derive from metadata if possible". Also using your suggested settings for temp etc. Is this all I need to do for optimal results or am i missing something?
That should do it ; worst case is manually selecting Mistral Template.