Delta-Vector
/

Pascal-Base-12B

Image-Text-to-Text

text-generation-inference

Model card Files Files and versions

Delta-Vector commited on 27 days ago

Commit

2d6fe0f

·

verified ·

1 Parent(s): 3fa2a76

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -13,7 +13,7 @@ Hparams:
 - LR: 1e-5
 - Weight Decay: 0.02
 - grad-clip: 0.2
-- R64, A32, Rslora.
 pretrained on a buncha LNs, books, etc.
 Used for a base for Pascal.

 - LR: 1e-5
 - Weight Decay: 0.02
 - grad-clip: 0.2
+- R64, A32, Rslora, 0.1 drop out, 16bit lora (qlora is cope.)
 pretrained on a buncha LNs, books, etc.
 Used for a base for Pascal.