The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
Paper
•
2504.17768
•
Published
•
14
Piotr Nawrot
pnawrot
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
about 2 months ago
Sample-efficient Integration of New Modalities into Large Language
Models
upvoted
a
paper
5 months ago
Bootstrapping World Models from Dynamics Models in Multimodal Foundation
Models
upvoted
a
paper
5 months ago
Inference-Time Hyper-Scaling with KV Cache Compression
Organizations
None yet
authored
6
papers
6 months ago
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models
Paper
•
2307.06440
•
Published
•
3
Hierarchical Transformers Are More Efficient Language Models
Paper
•
2110.13711
•
Published
•
1
nanoT5: A PyTorch Framework for Pre-training and Fine-tuning T5-style Models with Limited Resources
Paper
•
2309.02373
•
Published
•
1
Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference
Paper
•
2403.09636
•
Published
•
3
Efficient Transformers with Dynamic Token Pooling
Paper
•
2211.09761
•
Published