Hugging Face's logo

BlinkDL
/
rwkv-7-world

Text Generation
PyTorch
causal-lm
rwkv
Model card Files Community
BlinkDL commited on
Commit
bde4af2
·
verified ·
1 Parent(s): b9edcaf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -37,7 +37,11 @@ For developers: https://github.com/BlinkDL/RWKV-LM
37
 
38
  Chat demo: https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_CHAT.py
39
 
40
- MMLU rwkv7-v3-1b5 44.84% (rwkv6-v2.1-1b6 26.34%): https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/rwkv_mmlu_eval.py
 
 
 
 
41
 
42
  0.1B = L12 D768 // 0.4B = L24 D1024 // 1.5B = L24 D2048 // ~3B = L32 D2560 // ~7B = L32 D4096
43
 
 
37
 
38
  Chat demo: https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_CHAT.py
39
 
40
+ MMLU eval: https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v7/rwkv_mmlu_eval.py
41
+
42
+ rwkv7-v3-2.9b 54.56% (rwkv6-v2.1-3.1b 32.38%)
43
+
44
+ rwkv7-v3-1.5b 44.84% (rwkv6-v2.1-1.6b 26.34%)
45
 
46
  0.1B = L12 D768 // 0.4B = L24 D1024 // 1.5B = L24 D2048 // ~3B = L32 D2560 // ~7B = L32 D4096
47