tomg-group-umd/Gemstone-768x45

Text Generation • Updated 2 days ago • 3

Note Our main training runs (no postfix) are trained for 350 billion tokens with checkpoints stored are revisions at every 2 billion tokens. We also have ablations over learning rate (postfix: "_lr_ablation") and cooldown periods (postfix "_cooldown" still uploading).