WaveCut commited on
Commit
5c98e9e
·
verified ·
1 Parent(s): 384cf0e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -36
README.md CHANGED
@@ -18,40 +18,53 @@ pipeline_tag: text-generation
18
 
19
  - [HF model](https://huggingface.co/Vikhrmodels/Vikhr-Gemma-2B-instruct)
20
 
21
- ## Perplexity Wikitext-2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
- | Model | Perplexity |
24
- |:--------------------------------|:--------------------|
25
- | Vikhr-Gemma-2B-instruct-Q4_K | 10.4374 +/- 0.07339 |
26
- | Vikhr-Gemma-2B-instruct-Q4_0 | 10.6480 +/- 0.07452 |
27
- | Vikhr-Gemma-2B-instruct-Q8_0 | 10.1209 +/- 0.07105 |
28
- | Vikhr-Gemma-2B-instruct-Q4_1 | 10.5574 +/- 0.07476 |
29
- | Vikhr-Gemma-2B-instruct-F32 | 10.1191 +/- 0.07099 |
30
- | Vikhr-Gemma-2B-instruct-Q6_K | 10.1503 +/- 0.07117 |
31
- | Vikhr-Gemma-2B-instruct-IQ2_S | 14.0745 +/- 0.10367 |
32
- | Vikhr-Gemma-2B-instruct-BF16 | 10.1189 +/- 0.07098 |
33
- | Vikhr-Gemma-2B-instruct-IQ2_M | 12.7723 +/- 0.09234 |
34
- | Vikhr-Gemma-2B-instruct-Q2_K | 12.8851 +/- 0.09332 |
35
- | Vikhr-Gemma-2B-instruct-IQ4_XS | 10.3770 +/- 0.07299 |
36
- | Vikhr-Gemma-2B-instruct-Q5_0 | 10.2551 +/- 0.07251 |
37
- | Vikhr-Gemma-2B-instruct-Q5_K | 10.1975 +/- 0.07184 |
38
- | Vikhr-Gemma-2B-instruct-IQ1_S | 42.6888 +/- 0.36888 |
39
- | Vikhr-Gemma-2B-instruct-IQ4_NL | 10.3767 +/- 0.07310 |
40
- | Vikhr-Gemma-2B-instruct-IQ3_S | 10.9100 +/- 0.07725 |
41
- | Vikhr-Gemma-2B-instruct-Q3_K_S | 11.6028 +/- 0.08333 |
42
- | Vikhr-Gemma-2B-instruct-Q2_K_S | 14.7951 +/- 0.10960 |
43
- | Vikhr-Gemma-2B-instruct-F16 | 10.1191 +/- 0.07099 |
44
- | Vikhr-Gemma-2B-instruct-Q4_K_M | 10.4374 +/- 0.07339 |
45
- | Vikhr-Gemma-2B-instruct-Q5_K_M | 10.1975 +/- 0.07184 |
46
- | Vikhr-Gemma-2B-instruct-Q5_1 | 10.2348 +/- 0.07208 |
47
- | Vikhr-Gemma-2B-instruct-Q4_K_S | 10.4924 +/- 0.07386 |
48
- | Vikhr-Gemma-2B-instruct-Q5_K_S | 10.2098 +/- 0.07198 |
49
- | Vikhr-Gemma-2B-instruct-IQ3_XS | 11.1461 +/- 0.07907 |
50
- | Vikhr-Gemma-2B-instruct-Q3_K | 10.7416 +/- 0.07606 |
51
- | Vikhr-Gemma-2B-instruct-Q3_K_M | 10.7416 +/- 0.07606 |
52
- | Vikhr-Gemma-2B-instruct-IQ3_M | 10.7035 +/- 0.07573 |
53
- | Vikhr-Gemma-2B-instruct-Q3_K_L | 10.6242 +/- 0.07506 |
54
- | Vikhr-Gemma-2B-instruct-IQ2_XS | 15.8229 +/- 0.11909 |
55
- | Vikhr-Gemma-2B-instruct-IQ1_M | 29.5443 +/- 0.24168 |
56
- | Vikhr-Gemma-2B-instruct-IQ3_XXS | 11.6931 +/- 0.08282 |
57
- | Vikhr-Gemma-2B-instruct-IQ2_XXS | 18.8375 +/- 0.14581 |
 
18
 
19
  - [HF model](https://huggingface.co/Vikhrmodels/Vikhr-Gemma-2B-instruct)
20
 
21
+ ## Perplexity
22
+ Veles Results:
23
+ | Model | Perplexity |
24
+ |:--------|:-------------------|
25
+ | Q4_K | 4.7254 +/- 0.03867 |
26
+ | Q4_0 | 4.8067 +/- 0.03922 |
27
+ | Q8_0 | 4.6042 +/- 0.03751 |
28
+ | Q4_1 | 4.7798 +/- 0.03933 |
29
+ | F32 | 4.6013 +/- 0.03749 |
30
+ | Q6_K | 4.6244 +/- 0.03760 |
31
+ | BF16 | 4.6015 +/- 0.03749 |
32
+ | Q2_K | 5.6819 +/- 0.04737 |
33
+ | Q5_0 | 4.6876 +/- 0.03855 |
34
+ | Q5_K | 4.6428 +/- 0.03789 |
35
+ | Q3_K_S | 5.1485 +/- 0.04257 |
36
+ | Q2_K_S | 6.3124 +/- 0.05359 |
37
+ | F16 | 4.6013 +/- 0.03749 |
38
+ | Q4_K_M | 4.7254 +/- 0.03867 |
39
+ | Q5_K_M | 4.6428 +/- 0.03789 |
40
+ | Q5_1 | 4.6518 +/- 0.03794 |
41
+ | Q4_K_S | 4.7631 +/- 0.03916 |
42
+ | Q5_K_S | 4.6509 +/- 0.03803 |
43
+ | Q3_K | 4.8339 +/- 0.03965 |
44
+ | Q3_K_M | 4.8339 +/- 0.03965 |
45
+ | Q3_K_L | 4.7981 +/- 0.03934 |
46
 
47
+ Wikitext-2 Results:
48
+ | Model | Perplexity |
49
+ |:--------|:--------------------|
50
+ | Q4_K | 10.4374 +/- 0.07339 |
51
+ | Q4_0 | 10.6480 +/- 0.07452 |
52
+ | Q8_0 | 10.1209 +/- 0.07105 |
53
+ | Q4_1 | 10.5574 +/- 0.07476 |
54
+ | F32 | 10.1191 +/- 0.07099 |
55
+ | Q6_K | 10.1503 +/- 0.07117 |
56
+ | BF16 | 10.1189 +/- 0.07098 |
57
+ | Q2_K | 12.8851 +/- 0.09332 |
58
+ | Q5_0 | 10.2551 +/- 0.07251 |
59
+ | Q5_K | 10.1975 +/- 0.07184 |
60
+ | Q3_K_S | 11.6028 +/- 0.08333 |
61
+ | Q2_K_S | 14.7951 +/- 0.10960 |
62
+ | F16 | 10.1191 +/- 0.07099 |
63
+ | Q4_K_M | 10.4374 +/- 0.07339 |
64
+ | Q5_K_M | 10.1975 +/- 0.07184 |
65
+ | Q5_1 | 10.2348 +/- 0.07208 |
66
+ | Q4_K_S | 10.4924 +/- 0.07386 |
67
+ | Q5_K_S | 10.2098 +/- 0.07198 |
68
+ | Q3_K | 10.7416 +/- 0.07606 |
69
+ | Q3_K_M | 10.7416 +/- 0.07606 |
70
+ | Q3_K_L | 10.6242 +/- 0.07506 |