GRAM-Qwen3-1.7B-RewardModel-GGUF

GRAM-Qwen3-1.7B-RewardModel is a generative reward model developed by NiuTrans that follows a two-step training approach: it first pre-trains on a large amount of unlabeled data and then fine-tunes with supervised labeled data. This methodology, which incorporates label smoothing and a regularized ranking loss, enables effective reward generalization for large language models (LLMs). The model is built on the Qwen3-1.7B base, a compact language model with 1.7 billion parameters, 28 layers, and attention heads designed to handle long-context inputs (up to 32,768 tokens) and support both detailed reasoning and fast responses. GRAM-Qwen3-1.7B-RewardModel is intended for flexible application across diverse tasks, providing an open-source, plug-and-play reward model for aligning LLM outputs without requiring extensive task-specific retraining. It excels in evaluating and ranking the quality of AI-generated responses, operating effectively as a judge model in AI alignment scenarios.

Model Files

Model File name	Size	QuantType
GRAM-Qwen3-1.7B-RewardModel.BF16.gguf	3.45 GB	BF16
GRAM-Qwen3-1.7B-RewardModel.F16.gguf	3.45 GB	F16
GRAM-Qwen3-1.7B-RewardModel.F32.gguf	6.89 GB	F32
GRAM-Qwen3-1.7B-RewardModel.Q2_K.gguf	778 MB	Q2_K
GRAM-Qwen3-1.7B-RewardModel.Q3_K_L.gguf	1 GB	Q3_K_L
GRAM-Qwen3-1.7B-RewardModel.Q3_K_M.gguf	940 MB	Q3_K_M
GRAM-Qwen3-1.7B-RewardModel.Q3_K_S.gguf	867 MB	Q3_K_S
GRAM-Qwen3-1.7B-RewardModel.Q4_K_M.gguf	1.11 GB	Q4_K_M
GRAM-Qwen3-1.7B-RewardModel.Q4_K_S.gguf	1.06 GB	Q4_K_S
GRAM-Qwen3-1.7B-RewardModel.Q5_K_M.gguf	1.26 GB	Q5_K_M
GRAM-Qwen3-1.7B-RewardModel.Q5_K_S.gguf	1.23 GB	Q5_K_S
GRAM-Qwen3-1.7B-RewardModel.Q6_K.gguf	1.42 GB	Q6_K
GRAM-Qwen3-1.7B-RewardModel.Q8_0.gguf	1.83 GB	Q8_0

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):