---
base_model:
- unsloth/gemma-3-12b-it
- NewEden/Gemma-LN-Lora
library_name: transformers
tags:
- mergekit
- merge

---
Hparams:

- LR: 1e-5
- Weight Decay: 0.02
- grad-clip: 0.2
- R64, A32, Rslora, 0.1 drop out, 16bit lora (qlora is cope.)

pretrained on a buncha LNs, books, etc.
Used for a base for Pascal.