Hugging Face's logo
poe's picture
6 7 270

poe

wop
andyolivers's profile picture Pacific-Prime's profile picture Fishtiks's profile picture
·
https://cosmos-vb.netlify.app/

AI & ML interests

AI research AGI

Recent Activity

reacted to reaperdoesntknow's post with 👍 1 day ago
We present a methodology for training small language models on CPU at FP32 precision that achieves capability-per-dollar efficiency orders of magnitude beyond GPU-based training. Across15modelsspanningfournovelarchitecturefamilies—MixtureofAttentions(MoA),cross- architecture fusion (Qemma), swarm intelligence (SAGI), and metric-space causal language models (DiscoverLM)—total compute cost was $24 on a single AMD EPYC 9454P proces- sor. We introduce seven methodological pillars: (1) FP32 precision preservation, with exper- iments demonstrating 5,810×single-operation error and 23,225×compounding error ratio for FP16 at network depth; (2) sparse cognitive architectures where 0.02–7% of parameters activate per token, matching CPU branching rather than GPU SIMD; (3) developmental curriculum training progressing from language to logic to transfer to depth; (4) continuous belt-fed data ingestion eliminating truncation waste; (5) hardware-native optimization for AMD Zen 4 via AOCL/OpenMP/NUMA-aware allocation; (6) self-regulating thermodynamic governance with emergent temperature measurement grounded in L2-star discrepancy; and (7) open-standard compute (AVX2 SIMD at FP32) free of proprietary vendor dependency. We argue that trans- formers were designed for GPU hardware rather than mathematical optimality, and that archi- tectures designed for geometric correctness—metric-space attention, triangle inequality enforce- ment, sparse expert routing—naturally favor CPU execution. For sub-2B parameter models, CPU training produces more capable models at a fraction of the cost.
liked a model 2 days ago
Efficient-Large-Model/Fast_dLLM_v2_1.5B
liked a Space 3 days ago
shariqfarooq/ZoeDepth
View all activity

Organizations

OpenAgent's profile picture Agentic General Inteligence's profile picture