just use for

vllm serve lee5j/Athene-V2-Chat-gptq4 --gpu-memory-utilization 1

Downloads last month
4
Safetensors
Model size
11.9B params
Tensor type
I32
BF16
F16
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for lee5j/Athene-V2-Chat-gptq4

Base model

Qwen/Qwen2.5-72B
Quantized
(25)
this model