Llamacpp Quantizations of DeepSeek-V3-0324 (MLA version)

Original model: Adopting BF16 & Imatrix from unsloth/DeepSeek-V3-0324-GGUF-UD.

All quants made with modification of llama.cpp based on bartowski1182-llama.cpp.

IQ1_M / Q4_K / Q8_0 : 144.24 GiB (1.85 BPW)

IQ1_S / Q4_K / Q8_0 : 129.94 GiB (1.66 BPW)

GGUF

Model size

671B params

Architecture

deepseek2

Hardware compatibility

1-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bobchenyx/DeepSeek-V3-0324-MLA-GGUF

Base model

Quantized

(17)

this model