Missing file
#2
by
cahya
- opened
I get this error message when I use this model with vllm:ValueError: Failed to load mistral 'params.json' config for model unsloth/Devstral-Small-2505-bnb-4bit. Please check if the model is a mistral-format model and if the config file exists.
It seems params.json (https://huggingface.co/mistralai/Devstral-Small-2505/blob/main/params.json) is missing.
The tokenizer*json are also missing. I use following command:
vllm serve unsloth/Devstral-Small-2505-bnb-4bit --dtype half --tokenizer_mode mistral --config_format mistral --enable_auto_tool_choice --tool-call-parser mistral
Have you found a workaround?