<sep> not a special token?
#4
by
bartowski
- opened
You're using <sep> as a delimiter for roles as well as to indicate stopping of generation, but it's not specified as a special token (or a token at all in the vocab/tokenizer). This seems like it would lead to degraded performance
Is this intentional? Was it really trained to recognize tokens [366][29136][29] as being the EOS and turn separator?
Just tested via vLLM and prompting it through the OpenAI API, I get this as a response:
<reasoning>
The user previously asked for a semester-long walking fitness goal, and I provided a detailed plan with weekly targets, terrain variety, and tech integration. Now they're asking for a 1-sentence goal. Let me see why the shift.
Maybe they found the original goal too lengthy and want something more concise for presentation or social media. They might be sharing it with a group, needing brevity. The key is to retain the essence of the goal without the details.
The original goal focused on endurance, consistency, and mental well-being over 15 weeks. The 1-sentence version needs to encapsulate that. Let me think of synonyms: "achieve" instead of "build," "mileage" instead of "weekly mileage target." Also, including the duration and the benefits.
Possible options: "Walk 75 miles over 15 weeks, increasing daily distance by 1 mile each week to boost endurance and mental health." That's 1 sentence, includes the main elements. Check if it's clear and meets their needs. They might also appreciate a version that's even shorter, but still includes the key components. Let me make sure it's actionable and specific. Yep, that should work.
</reasoning>
**1-Sentence Goal:** *"By the end of the semester, I will walk 75 miles total (5 days/week, increasing mileage by 1 mile daily each week) to strengthen endurance, boost mental focus, and build a sustainable walking habit."*
Let me know if you’d like to simplify or emphasize something else!
<sep> human:
Kind of confused if generation is supposed to stop at <sep> or not?
It would also be nice to have clarification on whether <reasoning></reasoning> content should be removed from the previous messages for longer conversations.
yes we recommend to remove the reasoning content from previous conversations @qingy2024