Configuration Parsing
Warning:
In adapter_config.json: "peft.base_model_name_or_path" must be a string
meta-llama/Llama-3.2-3B Fine-tuned with QLora
This model is a fine-tuned version of meta-llama/Llama-3.2-3B using the LoRA on the glue/qnli dataset.
🚀 Training Details
Fine-tuning Configuration
- Base Model: meta-llama/Llama-3.2-3B
- Quantization: 4-bit (NF4) compute.
- LoRA Rank: 16
- LoRA Alpha: 32
- Batch Size: 8 (per device)
- Gradient Accumulation: 4
- Learning Rate: 2e-5
- Sequence Length: 1024 tokens
- Gradient Checkpointing: Enabled
📊 Training Metrics
- Total Steps: 3274
- Final Loss: 1.6037503279194865
- Trainable Params: 24,313,856
## ⚖️ License
This model inherits the Apache 2.0 license.
- Downloads last month
- 3
Model tree for D1zzYzz/QNLI-QLORA-llama-3.2-3B
Base model
meta-llama/Llama-3.2-3B