Known Issue: Chat template not included in GGUF
#1
pinned
by
Doctor-Shotgun
- opened
The HF version of the model uses chat_template.jinja to define the chat template, which is supported in the latest transformers, however it appears that GGUF conversion at the time of writing does not support chat_template.jinja; it only supports reading the chat template from chat_template.jsonor tokenizer_config.json.
This results in the Llama 3 chat template not being included in the GGUF conversion, which will affect users who intend to host the model with chat completions enabled.
Doctor-Shotgun
pinned discussion