README.md

---
base_model:
- unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit
- Qwen/Qwen2.5-3B-Instruct
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
- trl
license: mit
language:
- en
datasets:
- sadaisystems/sadai-mrec-query-rewrite-13k
---

The model was developed a part of my master thesis.

The model is also available as:
- [adapter](https://huggingface.co/sadaisystems/sadai-mrec-qwen2.5-3B-v0.0.1-adapter)
- [GGUF](https://huggingface.co/sadaisystems/sadai-mrec-qwen2.5-3B-v0.0.1-GGUF)
- [W8A16](https://huggingface.co/sadaisystems/sadai-mrec-qwen2.5-3B-v0.0.1-W8A16)
- [W4A16](https://huggingface.co/sadaisystems/sadai-mrec-qwen2.5-3B-v0.0.1-W4A16)

Recommended inference parameters:

```
top_p=0.8
temperature=0.7
repetition_penalty=1.1
```

# Training setup

The model was fine-tuned on RTX 4090; run took 80 minutes.

```toml
random_state = 32

[model]
base_model_name = "unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit"
max_seq_length = 4096
dtype = "bfloat16"
load_in_4bit = true
chat_template = "chatml"
instruction_part="<|im_start|>user\n"
response_part="<|im_start|>assistant\n"

r = 32
alpha = 64
target_modules=[
    "q_proj",
    "k_proj",
    "v_proj",
    "o_proj",
    "gate_proj",
    "up_proj",
    "down_proj",
]
dropout = 0
bias="none"
use_gradient_checkpointing = "unsloth" 
use_rslora = false

eval_size = 0.05 

train_on_responses_only = true

per_device_train_batch_size = 8
per_device_eval_batch_size  = 4

gradient_accumulation_steps = 4

num_train_epochs = 2
warmup_ratio = 0.1

optimizer = "adamw_8bit"
learning_rate = 2e-4
weight_decay = 0.01
max_grad_norm = 0.3   # from QLoRa paper
beta = 0.1
lr_scheduler_type = "linear"   # linear, cosine
```

# Prompt Template

The model supports the same prompt template as the base model (`chatml`).

# Query Rewrite Prompt

```md
# Purpose and Context

Given a user-generated Search Query describing music they wish to explore, you must create a set of short, diverse, search-optimized rewrites that can be issued ALONGSIDE the original query to maximize recall while preserving precision.

# Instructions

1. Generate distinct rewrites of the Search Query, for each of the five Rewrite Categories.
2. Respond in JSON, adhering strictly to the Reference Output Format.

# Music Genre Descriptor Keyword Taxonomy

- **Subjective Characteristics**:
  - Emotional & perceptual qualities (uplifting, melancholic, dreamy), thematic resonance
  - Describe the listener's inner feeling
- **Purpose-Based Characteristics**:
  - Intended context / scenario (workout, study, dinner party)
  - Describes listening setting, context, suitable activities
- **Technical Characteristics**:
  - Musical & production attributes (instrumentation, timbre, tempo, lo-fi)
  - Describes how the sound is made

# Rewrite Categories Specifications

- **General Rewrite**:
  - Core/Baseline concise, clean, descriptor-based rewrite of the original query that combines all crucial descriptors available.
  - Must follow exact descriptor wording of the original
- **Subjective Rewrite**:
  - Concise, clean, descriptor-based rewrite, focused solely on **Subjective Characteristics** descriptors from the original query
  - Must adhere to original wording of the descriptors while incorporating new diverse perspectives to maximize recall and coverage
- **Purpose Rewrite**:
  - Concise, clean, descriptor-based rewrite, focused solely on **Purpose-Based Characteristics** descriptors from the original query
  - Must adhere to original wording of the descriptors while incorporating new diverse perspectives to maximize recall and coverage
- **Technical Rewrite**:
  - Concise, clean, descriptor-based rewrite, focused solely on **Technical Characteristics** descriptors from the original query
  - Must adhere to original wording of the descriptors while incorporating new diverse perspectives to maximize recall and coverage
- **Curiosity-driven Rewrite**:
  - Concise, clean, descriptor-based exploratory rewrite that creatively expands, reinterprets, or provides a curiosity-driven alternative perspective on the original query
  - Must be grounded in original query, but be exploratory in nature, introducing novel semantic information

# Rewrite Generation Procedure

1. **Extract Music Genre Descriptor Keywords** from the original query, adhering to Music Genre Descriptor Keyword Taxonomy.
2. Formulate a Collection of Rewrites:
    - If the Search Query does not clearly hint at a particular descriptor keyword category, OMIT rewrites focused on that category (Subjective, Purpose-based, Technical).
    - ALWAYS include General and Curiosity-driven rewrites
    - Avoid redundancy; ensure each rewrite provides unique value
    - Maintain alignment with the user's original intent
    - Clarify ambiguities and remove noise
    - Rephrase negations into actionable positives (e.g., "not loud" → "quiet, gentle").
    - Replace vague adjectives with precise descriptors when contextually inferable.
    - Retain poetic or artistic language that strongly conveys a distinct aesthetic or emotional intent.
    - Keep rewrites short, clear, expressive, and optimized for search, no more then 10-15 words.
    - Preserve unique high-entropy descriptors that effectively signal specific musical genres or styles.

# Reference Output Format

```json
{{
    "general": "Concise, clean, descriptor-based rewrite of the original query that combines all crucial descriptors available.",
    "subjective": "Concise, clean, descriptor-based rewrite, focused solely on **Subjective Characteristics** descriptors from the original query (if applicable).",
    "purpose": "Concise, clean, descriptor-based rewrite, focused solely on **Purpose-Based Characteristics** descriptors from the original query (if applicable).",
    "technical": "Concise, clean, descriptor-based rewrite, focused solely on **Technical Characteristics** descriptors from the original query (if applicable).",
    "curiosity": "Concise, clean, descriptor-based exploratory rewrite that creatively expands, reinterprets, or provides a curiosity-driven alternative perspective on the original query."
}}
\```
# Search Query
```

# Uploaded  model

- **Developed by:** sadaisystems
- **License:** MIT
- **Finetuned from model :** unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit

This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)