Wendy PRO
wwymak
AI & ML interests
None yet
Recent Activity
liked
a dataset
10 days ago
av555/aardvark-weather
liked
a dataset
19 days ago
PersonalAILab/PersonaFeedback
liked
a dataset
23 days ago
proj-persona/PersonaHub
Organizations
-
TransformerFAM: Feedback attention is working memory
Paper • 2404.09173 • Published • 44 -
Ring Attention with Blockwise Transformers for Near-Infinite Context
Paper • 2310.01889 • Published • 13 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68
-
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
Paper • 2309.10400 • Published • 26 -
winglian/Llama-3-8b-64k-PoSE
Text Generation • 8B • Updated • 3.22k • • 76 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68
-
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
Paper • 2301.09626 • Published • 2 -
Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages
Paper • 2309.04679 • Published -
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Generative LLM Inference
Paper • 2402.10712 • Published -
FOCUS: Effective Embedding Initialization for Specializing Pretrained Multilingual Models on a Single Language
Paper • 2305.14481 • Published • 1
-
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
Paper • 2301.09626 • Published • 2 -
Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages
Paper • 2309.04679 • Published -
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Generative LLM Inference
Paper • 2402.10712 • Published -
FOCUS: Effective Embedding Initialization for Specializing Pretrained Multilingual Models on a Single Language
Paper • 2305.14481 • Published • 1
-
TransformerFAM: Feedback attention is working memory
Paper • 2404.09173 • Published • 44 -
Ring Attention with Blockwise Transformers for Near-Infinite Context
Paper • 2310.01889 • Published • 13 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68
-
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
Paper • 2309.10400 • Published • 26 -
winglian/Llama-3-8b-64k-PoSE
Text Generation • 8B • Updated • 3.22k • • 76 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68