Qwen3-4B-Dolly-1k

A fine-tuned version of Qwen3-4B optimized for instruction following and conversational AI tasks. This model was trained on a subset of the Databricks Dolly-15k dataset using parameter-efficient fine-tuning techniques.

Model Details

Base Model

Model: Qwen3-4B (4 billion parameters)
Architecture: Transformer-based causal language model
Training Dataset: Databricks Dolly-15k (1,000 samples)
Training Method: LoRA fine-tuning

Capabilities

This model excels at:

📝 Instruction following
🤔 Question answering
💡 Creative text generation
🔍 Information extraction
📊 Text summarization
🧠 Brainstorming and ideation

Usage

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("sweatSmile/Qwen3-4B-Dolly-1k")
model = AutoModelForCausalLM.from_pretrained("sweatSmile/Qwen3-4B-Dolly-1k")

# Format input using chat template
messages = [
    {"role": "user", "content": "Explain the concept of machine learning in simple terms."}
]

# Tokenize and generate
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_token=True)
with torch.no_grad():
    outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Chat Template Format

The model uses the standard Qwen chat template:

<|im_start|>user
Your instruction here<|im_end|>
<|im_start|>assistant
Model response here<|im_end|>

Training Details

The model was fine-tuned on a curated 1,000-sample subset from the Databricks Dolly-15k dataset, which includes diverse instruction-following tasks such as:

Brainstorming and creative tasks
Question answering (open and closed)
Text classification and generation
Information extraction and summarization

Intended Use

Research and development in conversational AI
Educational applications
Prototyping instruction-following systems
Personal assistant applications

Limitations

Optimized for instruction-following tasks
Performance may vary on highly specialized domains
Should be used responsibly with appropriate safety measures

Citation

@misc{qwen3-4b-dolly-1k,
  title={Qwen3-4B-Dolly-1k: Instruction-Following Model},
  author={sweatSmile},
  year={2025},
  note={Fine-tuned on Databricks Dolly-15k subset}
}

A compact and efficient instruction-following model based on Qwen3-4B.