Tien Dung's picture

Tien Dung

tiendung

·

tiendung

AI & ML interests

None yet

Recent Activity

liked a model 2 days ago

SparseLLM/BlockFFN-3B-SFT

liked a model 10 days ago

turboderp/ERNIE-4.5-300B-A47B-PT-exl3

reacted to Jaward's post with 😎 14 days ago

I played around with the new RXTX paper (XX^T) and was able to train nanogpt with 4x4 RXTX matmuls in both attention layer and optimizer🤕 It just works (well I had to add some guardrails) but still saves 5% of memory usage: The Patch: - Computes attention scores with a 4x4 blockwise RXTX matmuls (no pytorch dot prod) - Handles arbitrary sequence lengths by padding to the nearest multiple of 4. - An RXTX variant of shampoo with params reshaped into 4x4 blocks during each optimizer step. - Uses 5% less ops Code: https://github.com/Jaykef/ai-algorithms/blob/main/nanogpt-rxtx.ipynb Paper: https://arxiv.org/pdf/2505.09814

View all activity

Organizations

tiendung 's models 16

tiendung/gemma-2-9b__extend_vocab

9B • Updated Oct 28, 2024 • 3

tiendung/gemma2reranking

9B • Updated Oct 11, 2024 • 3

tiendung/bge-reranking-m3_bf16

0.6B • Updated Aug 7, 2024 • 4

tiendung/bge-embedding-m3_bf16

0.6B • Updated Aug 7, 2024 • 2

tiendung/gemma2embedding

9B • Updated Aug 7, 2024 • 4

tiendung/gemma1reranking

3B • Updated Aug 7, 2024 • 4

tiendung/cc-vi_segdedup

Updated Aug 16, 2023 • 1

tiendung/pygmalion-6b-20-percent-soda_2e_merged

Text Generation • Updated Jul 21, 2023 • 13 • 1

tiendung/open_llama_3b-8k_visyll

Text Generation • Updated Jun 30, 2023 • 12

tiendung/tiny_starcoder_py-vi06

Text Generation • Updated Jun 13, 2023 • 10

tiendung/c4_vi_filtered

Updated Jun 1, 2023 • 1

tiendung/pygmalion-6b_20-percent-soda_2e

Updated May 27, 2023 • 1

tiendung/baize_gpt-j-6B_90k-3

Updated Apr 15, 2023 • 1

tiendung/hoaiht_vietnamese-alpaca-lora-gpt-j

Updated Mar 31, 2023

tiendung/symato

Updated Mar 14, 2023 • 2

tiendung/symato-nvidia-vn

Updated Mar 9, 2023