LLaMA-3 3B Fine-tuned with QLoRA (Unsloth) on Alpaca
This model is a fine-tuned version of unsloth/llama-3-3b-bnb-4bit using QLoRA and Unsloth for efficient instruction-tuning.
📖 Training Details
- Dataset:
tatsu-lab/alpaca - QLoRA: 4-bit quantization (NF4) using
bitsandbytes - LoRA Rank: 4 (adjust based on your config)
- LoRA Alpha: 8
- Batch Size: 2 per device
- Gradient Accumulation: 4
- Learning Rate: 2e-4
- Epochs: 1
- Trainer:
trl.SFTTrainer
💡 Notes
- Optimized for memory-efficient fine-tuning with Unsloth
- LoRA adapters are injected into Q, O, V attention projections
- No evaluation was run during training — please evaluate separately
📝 License
Apache 2.0
- Downloads last month
- 27
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support