DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Paper
•
2401.06066
•
Published
•
43
MOE
-
-
Mixtral of Experts
Paper • 2401.04088 • Published • 159 -
Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
Paper • 2401.02994 • Published • 48 -
LLM Augmented LLMs: Expanding Capabilities through Composition
Paper • 2401.02412 • Published • 36 -
Scaling Laws for Fine-Grained Mixture of Experts
Paper • 2402.07871 • Published • 11 -
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
Paper • 2402.01739 • Published • 26 -
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
Paper • 2401.15947 • Published • 49