Impressive
#1
by
Huegli
- opened
Hello mradermacher,
thank you for delivering the gguf versions once again.
The q6 is a very impressive model (with and without i1), feels very stable overall. It has barely any flaws, while q5 M surprisingly has some more, like breaking out of a chat with system messages. Wenn you reach the 8196 Mark, it tends to forget some details or Instructions from the very early part of the conversation, including the system prompt. But even this feels significantly less inconvenient, than the usual forgetfulness of most other models in the range between 7B and 24B, so it's still excellent and the best I have tried out so far.
Greetings,
Huegli