DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Paper
•
2410.10819
•
Published
•
8
Jiaming Tang
Sakits
AI & ML interests
Efficient Machine Learning
Recent Activity
updated
a model
about 20 hours ago
Sakits/so101_kick_the_ball_goal_static_pi05_with_state_proj_async_32_random_state_cond_010000
published
a model
about 20 hours ago
Sakits/so101_kick_the_ball_goal_static_pi05_with_state_proj_async_32_random_state_cond_010000
updated
a model
1 day ago
Sakits/so101_kick_the_ball_goal_static_pi05_with_state_proj_async_32_random_state_cond_002000
Organizations
authored
a
paper
12 months ago
authored
3
papers
about 1 year ago
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Paper
•
2306.00978
•
Published
•
11
Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
Paper
•
2406.10774
•
Published
•
3
OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization
Paper
•
2304.07493
•
Published
•
1