Qwen
/

Qwen2-7B-Instruct-GPTQ-Int8

Text Generation

text-generation-inference

8-bit precision

Model card Files Files and versions

Resources

View closed (0)

I tried vllm and without vllm, "RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 during inference with transformer" still exist!

#2 opened 10 months ago by

use_exllama？

#1 opened 11 months ago by