Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Paper
• 2404.02258 • Published
• 107
LLM foundations
-
-
Textbooks Are All You Need
Paper • 2306.11644 • Published • 154 -
Jamba: A Hybrid Transformer-Mamba Language Model
Paper • 2403.19887 • Published • 112
Large Language Models Struggle to Learn Long-Tail Knowledge
Paper • 2211.08411 • Published • 3Note This paper shows that LLMs should not be used as libraries! They are not knowledge engines, but rather (feeble) reasoning engines.
-
Language Models are Few-Shot Learners
Paper • 2005.14165 • Published • 19 -
Large Language Models are Zero-Shot Reasoners
Paper • 2205.11916 • Published • 3 -
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
Paper • 2411.04905 • Published • 127 -
Hierarchical Reasoning Model
Paper • 2506.21734 • Published • 48 -
Less is More: Recursive Reasoning with Tiny Networks
Paper • 2510.04871 • Published • 509