LLaMA-3 3B Fine-tuned with QLoRA (Unsloth) on Alpaca
This model is a fine-tuned version of unsloth/llama-3-3b-bnb-4bit using QLoRA and Unsloth for efficient instruction-tuning.
📖 Training Details
- Dataset:
ParamDev/ICD10CM_HCC - QLoRA: 4-bit quantization (NF4) using
bitsandbytes - LoRA Rank: 16 (adjust based on your config)
- LoRA Alpha: 16
- Batch Size: 2 per device
- Gradient Accumulation: 4
- Learning Rate: 2e-4
- Epochs: 1
- Trainer:
trl.SFTTrainer
💡 Notes
- Optimized for memory-efficient fine-tuning with Unsloth
Eval:
--- BLEU Score ---
- BLEU: 0.1357
- Precisions: [0.1770549143057013, 0.14363674525991743, 0.12315443285984186, 0.10826509902722374]
--- ROUGE Score ---
- ROUGE-1: 0.1891 (unigram overlap)
- ROUGE-2: 0.0881 (bigram overlap)
- ROUGE-L: 0.1320 (longest common subsequence)
--- BERT-Score ---
- Average Precision: 0.9655
- Average Recall: 0.9632
- Average F1 Score: 0.9643
📝 License
Apache 2.0
- Downloads last month
- 12
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support