T145 commited on
Commit
327e2d9
·
verified ·
1 Parent(s): cbb3acc

Adding Evaluation Results

Browse files

This is an automated PR created with [this space](https://huggingface.co/spaces/T145/open-llm-leaderboard-results-to-modelcard)!

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

Please report any issues here: https://huggingface.co/spaces/T145/open-llm-leaderboard-results-to-modelcard/discussions

Files changed (1) hide show
  1. README.md +116 -0
README.md CHANGED
@@ -1,3 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  <p align="center" width="100%">
2
  </p>
3
 
@@ -409,3 +510,18 @@ journal={arXiv preprint arXiv:2412.03187},
409
  year={2024}
410
  }
411
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ model-index:
3
+ - name: FuseChat-Llama-3.1-8B-Instruct
4
+ results:
5
+ - task:
6
+ type: text-generation
7
+ name: Text Generation
8
+ dataset:
9
+ name: IFEval (0-Shot)
10
+ type: wis-k/instruction-following-eval
11
+ split: train
12
+ args:
13
+ num_few_shot: 0
14
+ metrics:
15
+ - type: inst_level_strict_acc and prompt_level_strict_acc
16
+ value: 72.05
17
+ name: averaged accuracy
18
+ source:
19
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=FuseAI%2FFuseChat-Llama-3.1-8B-Instruct
20
+ name: Open LLM Leaderboard
21
+ - task:
22
+ type: text-generation
23
+ name: Text Generation
24
+ dataset:
25
+ name: BBH (3-Shot)
26
+ type: SaylorTwift/bbh
27
+ split: test
28
+ args:
29
+ num_few_shot: 3
30
+ metrics:
31
+ - type: acc_norm
32
+ value: 30.85
33
+ name: normalized accuracy
34
+ source:
35
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=FuseAI%2FFuseChat-Llama-3.1-8B-Instruct
36
+ name: Open LLM Leaderboard
37
+ - task:
38
+ type: text-generation
39
+ name: Text Generation
40
+ dataset:
41
+ name: MATH Lvl 5 (4-Shot)
42
+ type: lighteval/MATH-Hard
43
+ split: test
44
+ args:
45
+ num_few_shot: 4
46
+ metrics:
47
+ - type: exact_match
48
+ value: 7.02
49
+ name: exact match
50
+ source:
51
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=FuseAI%2FFuseChat-Llama-3.1-8B-Instruct
52
+ name: Open LLM Leaderboard
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: GPQA (0-shot)
58
+ type: Idavidrein/gpqa
59
+ split: train
60
+ args:
61
+ num_few_shot: 0
62
+ metrics:
63
+ - type: acc_norm
64
+ value: 7.38
65
+ name: acc_norm
66
+ source:
67
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=FuseAI%2FFuseChat-Llama-3.1-8B-Instruct
68
+ name: Open LLM Leaderboard
69
+ - task:
70
+ type: text-generation
71
+ name: Text Generation
72
+ dataset:
73
+ name: MuSR (0-shot)
74
+ type: TAUR-Lab/MuSR
75
+ args:
76
+ num_few_shot: 0
77
+ metrics:
78
+ - type: acc_norm
79
+ value: 6.15
80
+ name: acc_norm
81
+ source:
82
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=FuseAI%2FFuseChat-Llama-3.1-8B-Instruct
83
+ name: Open LLM Leaderboard
84
+ - task:
85
+ type: text-generation
86
+ name: Text Generation
87
+ dataset:
88
+ name: MMLU-PRO (5-shot)
89
+ type: TIGER-Lab/MMLU-Pro
90
+ config: main
91
+ split: test
92
+ args:
93
+ num_few_shot: 5
94
+ metrics:
95
+ - type: acc
96
+ value: 30.37
97
+ name: accuracy
98
+ source:
99
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=FuseAI%2FFuseChat-Llama-3.1-8B-Instruct
100
+ name: Open LLM Leaderboard
101
+ ---
102
  <p align="center" width="100%">
103
  </p>
104
 
 
510
  year={2024}
511
  }
512
  ```
513
+
514
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
515
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/FuseAI__FuseChat-Llama-3.1-8B-Instruct-details)!
516
+ Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=FuseAI%2FFuseChat-Llama-3.1-8B-Instruct&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
517
+
518
+ | Metric |Value (%)|
519
+ |-------------------|--------:|
520
+ |**Average** | 25.64|
521
+ |IFEval (0-Shot) | 72.05|
522
+ |BBH (3-Shot) | 30.85|
523
+ |MATH Lvl 5 (4-Shot)| 7.02|
524
+ |GPQA (0-shot) | 7.38|
525
+ |MuSR (0-shot) | 6.15|
526
+ |MMLU-PRO (5-shot) | 30.37|
527
+