[FIX] VLLM reasoning with mm: is a part of context
Browse filesMinimax have custom mm: block of reasoning, so i fixed block to exclude it only {{- think_end_token }} is emitted the actual reasoning content is stripped from the prompt sent to the model.
Why: Multi-turn conversations with full reasoning rendered each turn caused context window bloat. Now the model gets a signal (think_end_token) that thinking occurred, but the reasoning text itself is omitted from history to save tokens/context space.
- chat_template.jinja +7 -8
chat_template.jinja
CHANGED
|
@@ -159,13 +159,12 @@
|
|
| 159 |
{%- endif %}
|
| 160 |
{%- endif %}
|
| 161 |
|
| 162 |
-
{
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
{%- endif -%}
|
| 169 |
|
| 170 |
{%- if content -%}
|
| 171 |
{{- content }}
|
|
@@ -244,4 +243,4 @@
|
|
| 244 |
{%- else -%}
|
| 245 |
{#- adaptive: no prefix, let model decide -#}
|
| 246 |
{%- endif -%}
|
| 247 |
-
{%- endif -%}
|
|
|
|
| 159 |
{%- endif %}
|
| 160 |
{%- endif %}
|
| 161 |
|
| 162 |
+
{#- PATCH: strip reasoning from history to save context.
|
| 163 |
+
Only emit think_end_token prefix so the model knows the
|
| 164 |
+
(omitted) thinking block ended. Original behavior rendered
|
| 165 |
+
<mm:think>...</mm:think> for every assistant turn, which
|
| 166 |
+
caused context bloat in multi-turn conversations. -#}
|
| 167 |
+
{{- think_end_token }}
|
|
|
|
| 168 |
|
| 169 |
{%- if content -%}
|
| 170 |
{{- content }}
|
|
|
|
| 243 |
{%- else -%}
|
| 244 |
{#- adaptive: no prefix, let model decide -#}
|
| 245 |
{%- endif -%}
|
| 246 |
+
{%- endif -%}
|