Post
663
I made a speedrun leaderboard for LoRA fine-tuning. One frozen task (Qwen2.5-1.5B to 57% on GSM8K), one GPU, fastest training run wins. Every record gets re-run 3x with fresh seeds on identical hardware before it counts, so no self-reported numbers.
The baseline was 11:57 three days ago. Someone already got it down to 1:44, with data pruning and a chunked cross-entropy that never materializes the logits.
Attempting is free (Modal's monthly credits cover full runs), and the second track (SmolLM2 + SQuAD) is still sitting at its naive baseline — easy first record for someone.
vineeth98/lora-speedrun
The baseline was 11:57 three days ago. Someone already got it down to 1:44, with data pruning and a chunked cross-entropy that never materializes the logits.
Attempting is free (Modal's monthly credits cover full runs), and the second track (SmolLM2 + SQuAD) is still sitting at its naive baseline — easy first record for someone.
vineeth98/lora-speedrun
- 6 replies
