Linear Transformers with Learnable Kernel Functions are Better In-Context Models
Paper
•
2402.10644
•
Published
•
82
Transformers & Attention
-
-
Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
Paper • 2401.04658 • Published • 28 -
KAN: Kolmogorov-Arnold Networks
Paper • 2404.19756 • Published • 113 -
Your Transformer is Secretly Linear
Paper • 2405.12250 • Published • 159 -
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Paper • 2405.12981 • Published • 34 -
Block Transformer: Global-to-Local Language Modeling for Fast Inference
Paper • 2406.02657 • Published • 41