sadaisystems/sadai-mrec-qwen2.5-3B-v0.0.1-GGUF

The model was developed a part of my master thesis.

The model is also available as:

• adapter
• GGUF
• W8A16
• W4A16

Recommended inference parameters:

top_p=0.8
temperature=0.7
repetition_penalty=1.1

Training setup

The model was fine-tuned on RTX 4090; run took 80 minutes.

random_state = 32

[model]
base_model_name = "unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit"
max_seq_length = 4096
dtype = "bfloat16"
load_in_4bit = true
chat_template = "chatml"
instruction_part="<|im_start|>user\n"
response_part="<|im_start|>assistant\n"

r = 32
alpha = 64
target_modules=[
    "q_proj",
    "k_proj",
    "v_proj",
    "o_proj",
    "gate_proj",
    "up_proj",
    "down_proj",
]
dropout = 0
bias="none"
use_gradient_checkpointing = "unsloth" 
use_rslora = false

eval_size = 0.05 

train_on_responses_only = true

per_device_train_batch_size = 8
per_device_eval_batch_size  = 4

gradient_accumulation_steps = 4

num_train_epochs = 2
warmup_ratio = 0.1

optimizer = "adamw_8bit"
learning_rate = 2e-4
weight_decay = 0.01
max_grad_norm = 0.3   # from QLoRa paper
beta = 0.1
lr_scheduler_type = "linear"   # linear, cosine

Prompt Template

The model supports the same prompt template as the base model (chatml).

Query Rewrite Prompt

# Purpose and Context

Given a user-generated Search Query describing music they wish to explore, you must create a set of short, diverse, search-optimized rewrites that can be issued ALONGSIDE the original query to maximize recall while preserving precision.

# Instructions

1. Generate distinct rewrites of the Search Query, for each of the five Rewrite Categories.
2. Respond in JSON, adhering strictly to the Reference Output Format.

# Music Genre Descriptor Keyword Taxonomy

- **Subjective Characteristics**:
  - Emotional & perceptual qualities (uplifting, melancholic, dreamy), thematic resonance
  - Describe the listener's inner feeling
- **Purpose-Based Characteristics**:
  - Intended context / scenario (workout, study, dinner party)
  - Describes listening setting, context, suitable activities
- **Technical Characteristics**:
  - Musical & production attributes (instrumentation, timbre, tempo, lo-fi)
  - Describes how the sound is made

# Rewrite Categories Specifications

- **General Rewrite**:
  - Core/Baseline concise, clean, descriptor-based rewrite of the original query that combines all crucial descriptors available.
  - Must follow exact descriptor wording of the original
- **Subjective Rewrite**:
  - Concise, clean, descriptor-based rewrite, focused solely on **Subjective Characteristics** descriptors from the original query
  - Must adhere to original wording of the descriptors while incorporating new diverse perspectives to maximize recall and coverage
- **Purpose Rewrite**:
  - Concise, clean, descriptor-based rewrite, focused solely on **Purpose-Based Characteristics** descriptors from the original query
  - Must adhere to original wording of the descriptors while incorporating new diverse perspectives to maximize recall and coverage
- **Technical Rewrite**:
  - Concise, clean, descriptor-based rewrite, focused solely on **Technical Characteristics** descriptors from the original query
  - Must adhere to original wording of the descriptors while incorporating new diverse perspectives to maximize recall and coverage
- **Curiosity-driven Rewrite**:
  - Concise, clean, descriptor-based exploratory rewrite that creatively expands, reinterprets, or provides a curiosity-driven alternative perspective on the original query
  - Must be grounded in original query, but be exploratory in nature, introducing novel semantic information

# Rewrite Generation Procedure

1. **Extract Music Genre Descriptor Keywords** from the original query, adhering to Music Genre Descriptor Keyword Taxonomy.
2. Formulate a Collection of Rewrites:
    - If the Search Query does not clearly hint at a particular descriptor keyword category, OMIT rewrites focused on that category (Subjective, Purpose-based, Technical).
    - ALWAYS include General and Curiosity-driven rewrites
    - Avoid redundancy; ensure each rewrite provides unique value
    - Maintain alignment with the user's original intent
    - Clarify ambiguities and remove noise
    - Rephrase negations into actionable positives (e.g., "not loud" → "quiet, gentle").
    - Replace vague adjectives with precise descriptors when contextually inferable.
    - Retain poetic or artistic language that strongly conveys a distinct aesthetic or emotional intent.
    - Keep rewrites short, clear, expressive, and optimized for search, no more then 10-15 words.
    - Preserve unique high-entropy descriptors that effectively signal specific musical genres or styles.

# Reference Output Format

```json
{{
    "general": "Concise, clean, descriptor-based rewrite of the original query that combines all crucial descriptors available.",
    "subjective": "Concise, clean, descriptor-based rewrite, focused solely on **Subjective Characteristics** descriptors from the original query (if applicable).",
    "purpose": "Concise, clean, descriptor-based rewrite, focused solely on **Purpose-Based Characteristics** descriptors from the original query (if applicable).",
    "technical": "Concise, clean, descriptor-based rewrite, focused solely on **Technical Characteristics** descriptors from the original query (if applicable).",
    "curiosity": "Concise, clean, descriptor-based exploratory rewrite that creatively expands, reinterprets, or provides a curiosity-driven alternative perspective on the original query."
}}
\```
# Search Query

Uploaded model

• Developed by: sadaisystems
• License: MIT
• Finetuned from model : unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.