Weigao Sun

weigao266's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes

upvoted 2 collections 5 months ago

Qwen2

Collection

Qwen2 language models, including pretrained and instruction-tuned models of 5 sizes, including 0.5B, 1.5B, 7B, 57B-A14B, and 72B. • 39 items • Updated Sep 18 • 346

SSMs

Collection

A collection of Mamba-2-based research models with 8B parameters trained on 3.5T tokens for comparison with Transformers. • 5 items • Updated Oct 1 • 26

upvoted 3 papers 7 months ago

CO2: Efficient Distributed Training with Full Communication-Computation Overlap

Paper • 2401.16265 • Published Jan 29 • 1

Linear Attention Sequence Parallelism

Paper • 2404.02882 • Published Apr 3 • 3

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Paper • 2401.04658 • Published Jan 9 • 25