eaddario
/

DeepSeek-R1-Distill-Llama-8B-GGUF

Text Generation

Model card Files Files and versions

DeepSeek-R1-Distill-Llama-8B-GGUF

108 GB

1 contributor

History: 63 commits

eaddario's picture

Layer-wise quantization Q4_K_S

b710841 verified 6 months ago

imatrix
Regenerate importance matrices 6 months ago
logits
Generate base model logits 8 months ago
scores
Generate Perplexity, KLD, ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores 6 months ago
.gitattributes

1.65 kB

Update .gitattributes 8 months ago
.gitignore

6.78 kB

Add .gitignore 8 months ago
DeepSeek-R1-Distill-Llama-8B-F16.gguf

16.1 GB
xet

Convert to GGUF @ F16 8 months ago
DeepSeek-R1-Distill-Llama-8B-IQ3_M.gguf

3.48 GB
xet

Selective quantization IQ3_M 6 months ago
DeepSeek-R1-Distill-Llama-8B-IQ3_S.gguf

3.24 GB
xet

Selective quantization IQ3_S 6 months ago
DeepSeek-R1-Distill-Llama-8B-IQ4_NL.gguf

4.3 GB
xet

Selective quantization IQ4_NL 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q3_K_L.gguf

3.45 GB
xet

Selective quantization Q3_K_L 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q3_K_M.gguf

3.37 GB
xet

Selective quantization Q3_K_M 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q3_K_S.gguf

3.28 GB
xet

Selective quantization Q3_K_S 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf

4.44 GB
xet

Selective quantization Q4_K_M 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q4_K_S.gguf

4.28 GB
xet

Layer-wise quantization Q4_K_S 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q5_K_M.gguf

5.38 GB
xet

Layer-wise quantization Q5_K_M 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q5_K_S.gguf

5.24 GB
xet

Layer-wise quantization Q5_K_S 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q6_K.gguf

6.57 GB
xet

Layer-wise quantization Q6_K 6 months ago
DeepSeek-R1-Distill-Llama-8B-Q8_0.gguf

7.73 GB
xet

Layer-wise quantization Q8_0 6 months ago
README.md

19.2 kB

Update README.md 6 months ago