Qwen

/

Qwen3-4B-Instruct-2507-FP8

like 35

Follow

Qwen 51.5k

I don't know why but inference on vllm with Qwen-4B-Instruct-2507 is OK.
But this quantization gives only: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
vllm version 10.0.1.1