Qwen3-Reranker-0.6B-seq-cls-GGUF

Qwen3-Reranker-0.6B-seq-cls is a sequence classification adaptation of the Qwen3-Reranker-0.6B model, designed for advanced text reranking and classification tasks across over 100 languages, including code and multilingual retrieval scenarios. Built on the Qwen3 series, this 0.6B parameter model offers a 32k context window and supports instruction-aware customization for specific tasks, typically improving performance by 1% to 5% when using tailored English instructions. It inherits the robust reasoning, long-text understanding, and versatility of its parent models, excelling in text retrieval, code retrieval, clustering, classification, and bitext mining, and ranks among the top models in benchmarks such as the MTEB multilingual leaderboard.

Model Files

File name	Size	Quant Type
Qwen3-4B-abliterated.F32.gguf	16.1 GB	F32
Qwen3-4B-abliterated.BF16.gguf	8.05 GB	BF16
Qwen3-4B-abliterated.F16.gguf	8.05 GB	F16
Qwen3-4B-abliterated.Q8_0.gguf	4.28 GB	Q8_0
Qwen3-4B-abliterated.Q6_K.gguf	3.31 GB	Q6_K
Qwen3-4B-abliterated.Q5_K_M.gguf	2.89 GB	Q5_K_M
Qwen3-4B-abliterated.Q5_K_S.gguf	2.82 GB	Q5_K_S
Qwen3-4B-abliterated.Q4_K_M.gguf	2.5 GB	Q4_K_M
Qwen3-4B-abliterated.Q4_K_S.gguf	2.38 GB	Q4_K_S
Qwen3-4B-abliterated.Q3_K_L.gguf	2.24 GB	Q3_K_L
Qwen3-4B-abliterated.Q3_K_M.gguf	2.08 GB	Q3_K_M
Qwen3-4B-abliterated.Q3_K_S.gguf	1.89 GB	Q3_K_S
Qwen3-4B-abliterated.Q2_K.gguf	1.67 GB	Q2_K

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):