File size: 961 Bytes
fedf793 252b3f7 300e5ae 2004a57 fedf793 d1e7414 fedf793 d1e7414 fedf793 a78837b fedf793 b3ca7ee fedf793 d1e7414 b3ca7ee d1e7414 fedf793 d1e7414 fedf793 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
---
base_model: meta-llama/Llama-3.2-1B-Instruct
library_name: peft
pipeline_tag: text-generation
datasets:
- AvaLovelace/ASCII-Art
language:
- en
license: mit
---
# Model Card for LLaMA-ASCII-Art
15-780 Final Project. A Llama-3.2-1B-Instruct model fine-tuned to generate ASCII art.
## Training Details
### Training Data
The model was trained on the [ASCII-Art](https://huggingface.co/datasets/AvaLovelace/ASCII-Art) dataset.
### Training Procedure
The model was fine-tuned using LoRA and AdamW optimization. Learning rate followed a cosine decay with warmup.
#### Training Hyperparameters
- **Training regime:** bf16 mixed precision
- **Epochs:** 10
- **Batch size:** 2
- **Max learning rate:** 5e-4
- **Learning rate warmup steps:** 100
- **LoRA rank:** 32
- **LoRA alpha:** 16
- **LoRA dropout:** 0.05
#### Speeds, Sizes, Times [optional]
Fine-tuning took approximately 1 hour on one NVIDIA RTX A6000 (48GB).
### Framework versions
- PEFT 0.15.2 |