metadata
tags:
- llama
- alpaca
- qlora
- unsloth
- instruction-tuning
- fine-tuned
base_model: unsloth/llama-3-3b-bnb-4bit
library_name: peft
license: apache-2.0
datasets:
- tatsu-lab/alpaca
language:
- en
pipeline_tag: text-generation
LLaMA-3 3B Fine-tuned with QLoRA (Unsloth) on Alpaca
This model is a fine-tuned version of unsloth/llama-3-3b-bnb-4bit using QLoRA and Unsloth for efficient instruction-tuning.
📖 Training Details
- Dataset:
tatsu-lab/alpaca - QLoRA: 4-bit quantization (NF4) using
bitsandbytes - LoRA Rank: 4 (adjust based on your config)
- LoRA Alpha: 8
- Batch Size: 2 per device
- Gradient Accumulation: 4
- Learning Rate: 2e-4
- Epochs: 1
- Trainer:
trl.SFTTrainer
💡 Notes
- Optimized for memory-efficient fine-tuning with Unsloth
- LoRA adapters are injected into Q, O, V attention projections
- No evaluation was run during training — please evaluate separately
📝 License
Apache 2.0