DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Paper
•
2410.10819
•
Published
•
8
Jiaming Tang
Sakits
AI & ML interests
Efficient Machine Learning
Recent Activity
updated
a model
about 5 hours ago
Sakits/so101_clean_table_merged_pi0_unfreeze_all_020000
published
a model
about 5 hours ago
Sakits/so101_clean_table_merged_pi0_unfreeze_all_020000
updated
a dataset
about 5 hours ago
Sakits/eval_clean_table
Organizations
authored
a
paper
11 months ago
authored
3
papers
about 1 year ago
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Paper
•
2306.00978
•
Published
•
10
Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
Paper
•
2406.10774
•
Published
•
3
OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization
Paper
•
2304.07493
•
Published
•
1