Qwen
/

Text Generation
GGUF
conversational

Does vLLM support Qwen3 GGUF models?

#3
by ravens510127 - opened

My error:
pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
Value error, GGUF model with architecture qwen3 is not supported yet. [type=value_error, input_value=ArgsKwargs((), {'model': ..., 'model_impl': 'auto'}), input_type=ArgsKwargs]

vllm 0.9.1 this is already the latest version!

Any one knows how to fix? Just wait?

Sign up or log in to comment