Llamacpp Quantizations of DeepSeek-V3-0324-MoE-Pruner-E192 by tflsxyy

Original model: tflsxyy/DeepSeek-V3-0324-MoE-Pruner-E192-bf16.

All quants made with modification of llama.cpp based on bartowski1182-llama.cpp.

All quants made using imatrix option based on tflsxyy/DeepSeek-V3-0324-MoE-Pruner-imatrix.

IQ1_M / Q4_K / Q8_0 : 110.19 GiB(1.87 BPW)

IQ1_S / Q4_K / Q8_0 : 99.45 GiB (1.68 BPW)

Downloads last month
98
GGUF
Model size
508B params
Architecture
deepseek2
Hardware compatibility
Log In to view the estimation

1-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bobchenyx/DeepSeek-V3-0324-HFK-E192-GGUF

Quantized
(17)
this model

Collection including bobchenyx/DeepSeek-V3-0324-HFK-E192-GGUF