BEE-spoke-data/smol_llama-101M-GQA
Text Generation
•
0.1B
•
Updated
•
3.78k
•
28
LLM training
small-scale pretraining experiments of mine
-
-
BEE-spoke-data/smol_llama-220M-GQA
Text Generation • 0.2B • Updated • 3.66k • 12
BEE-spoke-data/smol_llama-220M-GQA-fineweb_edu
Text Generation • 0.2B • Updated • 21 • 1Note smol_llama-220M-GQA CPT on fineweb-edu for 10 billion tokens
-
BEE-spoke-data/smol_llama-81M-tied
Text Generation • 0.1B • Updated • 914 • 6 -
BEE-spoke-data/mega-ar-126m-4k
Text Generation • 0.1B • Updated • 3.13k • 4 -
BEE-spoke-data/verysmol_llama-v11-KIx2
Text Generation • 0.1B • Updated • 2.48k • 4 -
pszemraj/pythia-31m-KI_v1-2048-scratch
Text Generation • 0.0B • Updated • 840 -
BEE-spoke-data/bert-plus-L8-4096-v1.0
Fill-Mask • 0.1B • Updated • 27 -
BEE-spoke-data/mega-encoder-small-16k-v1
Fill-Mask • 0.1B • Updated • 11 • 4
BEE-spoke-data/NanoLlama-GQA-L10-A32_KV8-v13-KI
Text Generation • 0.2B • Updated • 19 • 2Note this is a mid-training checkpoint of what is now smol_llama-220M
-
pszemraj/jamba-900M-v0.13-KIx2
Text Generation • 0.9B • Updated • 13 • 4