Qwen
/

Qwen2.5-32B-Instruct-GPTQ-Int4

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions

Resources

View closed (2)

Improve language tag

#4 opened 12 days ago by

Generating through HuggingFace Transformers leads to RuntimeError: probability tensor contains either `inf`, `nan` or element < 0. Generating through vLLM encounters no issues.

#2 opened 3 months ago by