dyyyyyyyy

/

Qwen2.5-1.5B-GenRM-QueryOnly

like 1

Chat_template validation incompatible with vLLM-style inference (add_generation_prompt=True with single user message).

Hi, thanks for the great work on this model!

I’m reporting a potential issue with the current chat_template logic in the tokenizer config. Specifically:

The template contains the following hard validation:

{% if messages|length != 2 or messages[0].role != 'user' or messages[1].role != 'assistant' %}
  {{ raise_exception('This template only accepts exactly one user message followed by one assistant message') }}
{% endif %}

This prevents generation-time usage where only a single user message is passed and add_generation_prompt=True is used. This is a common use case in vLLM, OpenAI-compatible API, or any apply_chat_template(messages, add_generation_prompt=True) scenario.
For example, usages like below will fail:

messages = [{"role": "user", "content": "What is 2 + 2?"}]
tokenizer.apply_chat_template(messages, add_generation_prompt=True)

raise_exception('This template only accepts exactly one user message followed by one assistant message')

Sorry I can't open a PR for this since the tokenizer config is manage by git lfs. But I would suggest the following change:

{% if messages|length == 1 and messages[0].role == "user" %}
  {% set user_content = messages[0].content %}
  {% set assistant_content = None %}
{% elif messages|length == 2 and messages[0].role == "user" and messages[1].role == "assistant" %}
  {% set user_content = messages[0].content %}
  {% set assistant_content = messages[1].content %}
{% else %}
  {{ raise_exception("This template expects one user message (optionally followed by assistant reply).") }}
{% endif %}

Would you consider relaxing the validation so that add_generation_prompt=True works with just one user message? This would make the model more compatible with downstream serving tools like vLLM, FastChat, and OpenAI API wrappers.

Thanks again!

PS: I came here because this model is used as a unit test for verl, since this model is changed, every test case using this model failed, including my own PR ci. So fixing the model will be really helpful for me and many other contributors.