Smoothie-Qwen3-0.6B-F32-GGUF

Smoothie Qwen is a lightweight adjustment tool that smooths token probabilities in Qwen and similar models, enhancing balanced multilingual generation capabilities. For more details, please refer to https://github.com/dnotitia/smoothie-qwen.

Model Files

File Name	Size	Type	Description
Smoothie-Qwen3-0.6B.Q2_K.gguf	296 MB	Model	Q2_K quantized model (smallest)
Smoothie-Qwen3-0.6B.Q3_K_S.gguf	323 MB	Model	Q3_K_S quantized model
Smoothie-Qwen3-0.6B.Q3_K_M.gguf	347 MB	Model	Q3_K_M quantized model
Smoothie-Qwen3-0.6B.Q3_K_L.gguf	368 MB	Model	Q3_K_L quantized model
Smoothie-Qwen3-0.6B.Q4_K_S.gguf	383 MB	Model	Q4_K_S quantized model
Smoothie-Qwen3-0.6B.Q4_K_M.gguf	397 MB	Model	Q4_K_M quantized model
Smoothie-Qwen3-0.6B.Q5_K_S.gguf	437 MB	Model	Q5_K_S quantized model
Smoothie-Qwen3-0.6B.Q5_K_M.gguf	444 MB	Model	Q5_K_M quantized model
Smoothie-Qwen3-0.6B.Q6_K.gguf	495 MB	Model	Q6_K quantized model
Smoothie-Qwen3-0.6B.Q8_0.gguf	639 MB	Model	Q8_0 quantized model
Smoothie-Qwen3-0.6B.BF16.gguf	1.2 GB	Model	BF16 precision model
Smoothie-Qwen3-0.6B.F16.gguf	1.2 GB	Model	F16 precision model
Smoothie-Qwen3-0.6B.F32.gguf	2.39 GB	Model	F32 full precision model (largest)
.gitattributes	2.52 kB	Config	Git LFS configuration
config.json	31 Bytes	Config	Model configuration
README.md	31 Bytes	Documentation	Repository documentation

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):