DeepSeek-R1-Distill-Llama-8B-GGUF / scores /DeepSeek-R1-Distill-Llama-8B-iq3_m.hsw
eaddario's picture
Generate Perplexity, KLD, ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores
afa61ba verified
raw
history blame
12.5 kB
common_init_from_params: setting dry_penalty_last_n to ctx_size = 768
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
system_info: n_threads = 6 (n_threads_batch = 6) / 12 | Metal : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | ACCELERATE = 1 | AARCH64_REPACK = 1 |
hellaswag_score : loaded 10042 tasks from prompt.
================================= is_spm = 0
hellaswag_score : selecting 750 randomized tasks.
hellaswag_score : calculating hellaswag score over selected tasks.
task acc_norm
1 0.00000000
2 50.00000000
3 66.66666667
4 75.00000000
5 80.00000000
6 83.33333333
7 71.42857143
8 75.00000000
9 77.77777778
10 70.00000000
11 72.72727273
12 66.66666667
13 69.23076923
14 71.42857143
15 73.33333333
16 75.00000000
17 70.58823529
18 72.22222222
19 68.42105263
20 70.00000000
21 71.42857143
22 72.72727273
23 73.91304348
24 70.83333333
25 68.00000000
26 65.38461538
27 66.66666667
28 67.85714286
29 68.96551724
30 66.66666667
31 67.74193548
32 65.62500000
33 66.66666667
34 67.64705882
35 68.57142857
36 69.44444444
37 67.56756757
38 68.42105263
39 69.23076923
40 67.50000000
41 68.29268293
42 66.66666667
43 67.44186047
44 68.18181818
45 66.66666667
46 67.39130435
47 68.08510638
48 66.66666667
49 67.34693878
50 68.00000000
51 68.62745098
52 67.30769231
53 66.03773585
54 66.66666667
55 65.45454545
56 64.28571429
57 64.91228070
58 65.51724138
59 66.10169492
60 66.66666667
61 67.21311475
62 67.74193548
63 68.25396825
64 68.75000000
65 69.23076923
66 68.18181818
67 68.65671642
68 69.11764706
69 69.56521739
70 70.00000000
71 69.01408451
72 69.44444444
73 68.49315068
74 68.91891892
75 68.00000000
76 68.42105263
77 68.83116883
78 69.23076923
79 69.62025316
80 70.00000000
81 70.37037037
82 70.73170732
83 71.08433735
84 71.42857143
85 71.76470588
86 72.09302326
87 72.41379310
88 72.72727273
89 71.91011236
90 72.22222222
91 72.52747253
92 71.73913043
93 70.96774194
94 71.27659574
95 71.57894737
96 71.87500000
97 72.16494845
98 72.44897959
99 72.72727273
100 73.00000000
101 73.26732673
102 72.54901961
103 71.84466019
104 72.11538462
105 72.38095238
106 72.64150943
107 71.96261682
108 72.22222222
109 71.55963303
110 71.81818182
111 72.07207207
112 72.32142857
113 72.56637168
114 72.80701754
115 73.04347826
116 73.27586207
117 72.64957265
118 72.88135593
119 73.10924370
120 72.50000000
121 72.72727273
122 72.95081967
123 73.17073171
124 73.38709677
125 73.60000000
126 73.80952381
127 74.01574803
128 73.43750000
129 72.86821705
130 72.30769231
131 72.51908397
132 72.72727273
133 72.93233083
134 73.13432836
135 73.33333333
136 73.52941176
137 73.72262774
138 73.91304348
139 73.38129496
140 73.57142857
141 73.75886525
142 73.94366197
143 74.12587413
144 74.30555556
145 74.48275862
146 74.65753425
147 74.82993197
148 75.00000000
149 75.16778523
150 75.33333333
151 74.83443709
152 74.34210526
153 74.50980392
154 74.02597403
155 74.19354839
156 73.71794872
157 73.24840764
158 73.41772152
159 73.58490566
160 73.75000000
161 73.91304348
162 74.07407407
163 73.61963190
164 73.78048780
165 73.93939394
166 74.09638554
167 74.25149701
168 74.40476190
169 73.96449704
170 74.11764706
171 74.26900585
172 74.41860465
173 73.98843931
174 73.56321839
175 73.14285714
176 72.72727273
177 72.88135593
178 73.03370787
179 72.62569832
180 72.77777778
181 72.37569061
182 72.52747253
183 72.67759563
184 72.28260870
185 72.43243243
186 72.04301075
187 71.65775401
188 71.27659574
189 70.89947090
190 71.05263158
191 71.20418848
192 71.35416667
193 70.98445596
194 71.13402062
195 70.76923077
196 70.40816327
197 70.05076142
198 69.69696970
199 69.84924623
200 70.00000000
201 70.14925373
202 69.80198020
203 69.45812808
204 69.60784314
205 69.75609756
206 69.41747573
207 69.56521739
208 69.23076923
209 69.37799043
210 69.52380952
211 69.66824645
212 69.81132075
213 69.95305164
214 70.09345794
215 69.76744186
216 69.90740741
217 69.58525346
218 69.72477064
219 69.86301370
220 70.00000000
221 69.68325792
222 69.81981982
223 69.95515695
224 70.08928571
225 70.22222222
226 69.91150442
227 69.60352423
228 69.73684211
229 69.86899563
230 69.56521739
231 69.26406926
232 68.96551724
233 68.66952790
234 68.80341880
235 68.51063830
236 68.22033898
237 68.35443038
238 68.48739496
239 68.61924686
240 68.75000000
241 68.87966805
242 69.00826446
243 69.13580247
244 68.85245902
245 68.57142857
246 68.69918699
247 68.42105263
248 68.54838710
249 68.67469880
250 68.80000000
251 68.92430279
252 69.04761905
253 68.77470356
254 68.89763780
255 68.62745098
256 68.75000000
257 68.48249027
258 68.60465116
259 68.72586873
260 68.84615385
261 68.96551724
262 68.70229008
263 68.44106464
264 68.56060606
265 68.67924528
266 68.42105263
267 68.53932584
268 68.65671642
269 68.77323420
270 68.88888889
271 68.63468635
272 68.75000000
273 68.49816850
274 68.61313869
275 68.36363636
276 68.47826087
277 68.59205776
278 68.70503597
279 68.81720430
280 68.92857143
281 69.03914591
282 69.14893617
283 69.25795053
284 69.36619718
285 69.12280702
286 68.88111888
287 68.98954704
288 69.09722222
289 69.20415225
290 69.31034483
291 69.07216495
292 68.83561644
293 68.94197952
294 69.04761905
295 69.15254237
296 69.25675676
297 69.36026936
298 69.46308725
299 69.23076923
300 69.33333333
301 69.43521595
302 69.53642384
303 69.30693069
304 69.40789474
305 69.50819672
306 69.60784314
307 69.38110749
308 69.48051948
309 69.57928803
310 69.67741935
311 69.77491961
312 69.87179487
313 69.96805112
314 69.74522293
315 69.84126984
316 69.93670886
317 70.03154574
318 69.81132075
319 69.90595611
320 70.00000000
321 69.78193146
322 69.56521739
323 69.65944272
324 69.75308642
325 69.84615385
326 69.93865031
327 70.03058104
328 70.12195122
329 70.21276596
330 70.30303030
331 70.39274924
332 70.18072289
333 70.27027027
334 70.05988024
335 70.14925373
336 70.23809524
337 70.02967359
338 69.82248521
339 69.91150442
340 70.00000000
341 70.08797654
342 70.17543860
343 70.26239067
344 70.05813953
345 69.85507246
346 69.65317919
347 69.74063401
348 69.54022989
349 69.34097421
350 69.42857143
351 69.51566952
352 69.60227273
353 69.68838527
354 69.77401130
355 69.85915493
356 69.94382022
357 70.02801120
358 70.11173184
359 70.19498607
360 70.00000000
361 69.80609418
362 69.88950276
363 69.97245179
364 70.05494505
365 70.13698630
366 70.21857923
367 70.29972752
368 70.38043478
369 70.46070461
370 70.54054054
371 70.61994609
372 70.69892473
373 70.50938338
374 70.58823529
375 70.66666667
376 70.74468085
377 70.55702918
378 70.37037037
379 70.44854881
380 70.52631579
381 70.60367454
382 70.41884817
383 70.49608355
384 70.57291667
385 70.64935065
386 70.72538860
387 70.54263566
388 70.36082474
389 70.43701799
390 70.51282051
391 70.33248082
392 70.40816327
393 70.48346056
394 70.55837563
395 70.63291139
396 70.70707071
397 70.78085642
398 70.85427136
399 70.92731830
400 70.75000000
401 70.82294264
402 70.64676617
403 70.47146402
404 70.29702970
405 70.12345679
406 70.19704433
407 70.27027027
408 70.34313725
409 70.41564792
410 70.48780488
411 70.31630170
412 70.38834951
413 70.46004843
414 70.53140097
415 70.36144578
416 70.43269231
417 70.26378897
418 70.33492823
419 70.16706444
420 70.23809524
421 70.07125891
422 70.14218009
423 70.21276596
424 70.28301887
425 70.35294118
426 70.42253521
427 70.25761124
428 70.32710280
429 70.16317016
430 70.23255814
431 70.06960557
432 69.90740741
433 69.74595843
434 69.81566820
435 69.65517241
436 69.49541284
437 69.33638444
438 69.17808219
439 69.24829157
440 69.09090909
441 69.16099773
442 69.00452489
443 69.07449210
444 69.14414414
445 69.21348315
446 69.28251121
447 69.35123043
448 69.41964286
449 69.48775056
450 69.55555556
451 69.40133038
452 69.46902655
453 69.31567329
454 69.38325991
455 69.45054945
456 69.51754386
457 69.58424508
458 69.43231441
459 69.49891068
460 69.56521739
461 69.63123644
462 69.69696970
463 69.76241901
464 69.82758621
465 69.67741935
466 69.74248927
467 69.80728051
468 69.65811966
469 69.50959488
470 69.36170213
471 69.42675159
472 69.49152542
473 69.55602537
474 69.62025316
475 69.68421053
476 69.53781513
477 69.39203354
478 69.45606695
479 69.51983299
480 69.58333333
481 69.43866944
482 69.50207469
483 69.56521739
484 69.42148760
485 69.27835052
486 69.34156379
487 69.40451745
488 69.26229508
489 69.32515337
490 69.38775510
491 69.45010183
492 69.51219512
493 69.57403651
494 69.63562753
495 69.69696970
496 69.75806452
497 69.81891348
498 69.87951807
499 69.73947896
500 69.80000000
501 69.86027944
502 69.92031873
503 69.98011928
504 70.03968254
505 70.09900990
506 70.15810277
507 70.21696252
508 70.27559055
509 70.33398821
510 70.39215686
511 70.45009785
512 70.31250000
513 70.37037037
514 70.42801556
515 70.48543689
516 70.34883721
517 70.40618956
518 70.46332046
519 70.52023121
520 70.38461538
521 70.24952015
522 70.30651341
523 70.36328872
524 70.41984733
525 70.28571429
526 70.15209125
527 70.20872865
528 70.26515152
529 70.32136106
530 70.37735849
531 70.43314501
532 70.48872180
533 70.35647280
534 70.41198502
535 70.46728972
536 70.52238806
537 70.57728119
538 70.63197026
539 70.50092764
540 70.55555556
541 70.60998152
542 70.66420664
543 70.53406998
544 70.58823529
545 70.64220183
546 70.69597070
547 70.74954296
548 70.62043796
549 70.67395264
550 70.54545455
551 70.41742287
552 70.28985507
553 70.34358047
554 70.39711191
555 70.27027027
556 70.32374101
557 70.19748654
558 70.07168459
559 70.12522361
560 70.17857143
561 70.23172906
562 70.10676157
563 70.15985790
564 70.21276596
565 70.08849558
566 69.96466431
567 70.01763668
568 70.07042254
569 69.94727592
570 69.82456140
571 69.87740806
572 69.93006993
573 69.98254799
574 70.03484321
575 70.08695652
576 70.13888889
577 70.19064125
578 70.24221453
579 70.12089810
580 70.17241379
581 70.05163511
582 70.10309278
583 70.15437393
584 70.20547945
585 70.08547009
586 69.96587031
587 70.01703578
588 69.89795918
589 69.77928693
590 69.66101695
591 69.71235195
592 69.59459459
593 69.47723440
594 69.36026936
595 69.41176471
596 69.46308725
597 69.51423786
598 69.56521739
599 69.44908180
600 69.50000000
601 69.55074875
602 69.60132890
603 69.65174129
604 69.70198675
605 69.75206612
606 69.63696370
607 69.68698517
608 69.73684211
609 69.78653530
610 69.83606557
611 69.88543372
612 69.77124183
613 69.65742251
614 69.54397394
615 69.59349593
616 69.48051948
617 69.52998379
618 69.57928803
619 69.46688207
620 69.51612903
621 69.56521739
622 69.45337621
623 69.50240770
624 69.55128205
625 69.60000000
626 69.48881789
627 69.53748006
628 69.58598726
629 69.47535771
630 69.36507937
631 69.25515055
632 69.14556962
633 69.03633491
634 69.08517350
635 69.13385827
636 69.18238994
637 69.23076923
638 69.27899687
639 69.32707355
640 69.21875000
641 69.26677067
642 69.31464174
643 69.36236392
644 69.40993789
645 69.45736434
646 69.50464396
647 69.55177743
648 69.59876543
649 69.49152542
650 69.53846154
651 69.58525346
652 69.63190184
653 69.67840735
654 69.72477064
655 69.77099237
656 69.81707317
657 69.86301370
658 69.75683891
659 69.65098634
660 69.54545455
661 69.59152799
662 69.63746224
663 69.68325792
664 69.72891566
665 69.77443609
666 69.81981982
667 69.86506747
668 69.76047904
669 69.65620329
670 69.70149254
671 69.74664680
672 69.79166667
673 69.83655275
674 69.88130564
675 69.92592593
676 69.97041420
677 70.01477105
678 70.05899705
679 70.10309278
680 70.14705882
681 70.04405286
682 70.08797654
683 70.13177160
684 70.17543860
685 70.21897810
686 70.26239067
687 70.30567686
688 70.34883721
689 70.39187228
690 70.43478261
691 70.47756874
692 70.52023121
693 70.41847042
694 70.46109510
695 70.50359712
696 70.54597701
697 70.58823529
698 70.63037249
699 70.52932761
700 70.57142857
701 70.61340942
702 70.65527066
703 70.69701280
704 70.73863636
705 70.78014184
706 70.82152975
707 70.86280057
708 70.90395480
709 70.94499295
710 70.98591549
711 70.88607595
712 70.92696629
713 70.96774194
714 71.00840336
715 70.90909091
716 70.94972067
717 70.99023710
718 70.89136490
719 70.93184979
720 70.97222222
721 71.01248266
722 71.05263158
723 71.09266943
724 70.99447514
725 71.03448276
726 71.07438017
727 71.11416781
728 71.15384615
729 71.19341564
730 71.23287671
731 71.27222982
732 71.17486339
733 71.21418827
734 71.25340599
735 71.15646259
736 71.19565217
737 71.09905020
738 71.00271003
739 71.04194858
740 71.08108108
741 71.12010796
742 71.15902965
743 71.19784657
744 71.10215054
745 71.00671141
746 71.04557641
747 70.95046854
748 70.98930481
749 71.02803738
750 71.06666667