QuixiAI
/

DeepSeek-R1-AWQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions

Resources

View closed (24)

What is the calibration set used when using AutoAWQ to quantify BF16 format models to AWQ?

#37 opened 4 months ago by

AWQ is good, any following on deepseek-r1-0528

#36 opened 5 months ago by

H800 has errors

#35 opened 6 months ago by

update vllm to 0.8.x and meet some trouble

#34 opened 6 months ago by

HuggingLianWang

AMD Instinct MI210 + vllm fail to run this model, any solutions please? Is there any other deepseek-r1-671b models that can run succesfully on AMD Instinct MI210 + vllm? Thanks!

#33 opened 7 months ago by

More stable startup command, not easy oom.

#31 opened 7 months ago by

The awq quantization model may encounter garbled characters when performing inference on long texts.

#24 opened 8 months ago by

Add instructions to run R1-AWQ on SGLang

#22 opened 8 months ago by

requests get stuck when sending long prompts (already solved, but still don't know why?)

#18 opened 8 months ago by

Is there any accuracy results comparing to original DeepSeek-R1？

#15 opened 8 months ago by

Any one can run this model with SGlang framework？

#13 opened 8 months ago by

Regarding the issue of inconsistent calculation of tokens

#12 opened 9 months ago by

Max-Batch-Size, max-num-sequence, and fp_cache fp8_e4m3

#11 opened 9 months ago by

The inference performance of the DeepSeek-R1-AWQ model is weak compared to the DeepSeek-R1 model

#3 opened 9 months ago by