Model omits prompt template symbols

#1
by johnfrian - opened

I tested an obviously suggestive input, and it does correctly identify the potentially offensive parts of the content. However, it fails to generate the special characters that identify role start and end.

There's also a chance I'm just using it wrongly, but here is an excerpt from one of my testings
using llama-3.2-3b-Content-Moderation.Q8_0.gguf

Here is part of the input, omitting a lot of the start for brevity:

"<|im_start|>assistant\nIf she treats them as prey, they will run. She should stick to the plan.\n\nShe approaches them, smiling and putting on her most delicious voice. \"Excuse me..\"<|im_end|>\n\n<|im_start|>user\nAre there any suggestive themes in this story?<|im_end|>\n\n<|im_start|>assistant\n

The response repeats itself a lot, since it misses the special characters. Scroll all the way right to see it just adds "assistant" with no special characters.

This story contains suggestive themes, including seduction, manipulation, and domination. Her behavior and appearance are designed to lure and entice her target, and she will use her charm and powers of attraction to try to get what she wants. These themes are consistent with the character profile of a succubus, a type of demon that feeds on human life force through seduction and manipulation.assistant
This story contains suggestive themes, including seduction, manipulation, and domination. Her behavior and appearance are designed to lure and entice her target, and she will use her charm and powers of attraction to try to get what she wants. These themes are consistent with the character profile of a succubus, a type of demon that feeds on human life force through seduction and manipulation.assistant
This story contains suggestive themes, including seduction, manipulation, and domination. Her behavior and appearance are designed to lure and entice her target, and she will use her charm

Then it stops due to max generation length

I also tried removing linebreaks after the role, and then it fails spectactularly:

Yes, this story contains suggestive themes. Her behavior and description of her actions and appearance are intended to be provocative and seductive.assistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistantassistant

Sign up or log in to comment