File size: 2,370 Bytes
151493b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------- | --------- | --------- | -------- | ------------ | -------------- |
| IQ1_S | 489 | 355.5202 | 14.40 | 9.80 | 8.19 |
| IQ1_M | 515 | 159.7892 | 15.17 | 21.80 | 3.59 |
| IQ2_XXS | 560 | 113.7759 | 16.49 | 30.62 | 2.62 |
| IQ2_XS | 597 | 66.9608 | 17.58 | 52.03 | 1.44 |
| IQ2_S | 633 | 55.5589 | 18.65 | 62.70 | 1.18 |
| IQ2_M | 668 | 49.2839 | 19.67 | 70.69 | 1.04 |
| Q2_K_S | 683 | 56.3132 | 20.12 | 61.86 | 1.22 |
| Q2_K | 718 | 52.9062 | 21.15 | 65.85 | 1.15 |
| IQ3_XXS | 733 | 40.4841 | 21.59 | 86.05 | 0.85 |
| IQ3_XS | 793 | 37.9592 | 23.36 | 91.78 | 0.78 |
| Q3_K_S | 821 | 40.8887 | 24.18 | 85.20 | 0.86 |
| IQ3_S | 822 | 37.3800 | 24.21 | 93.20 | 0.77 |
| IQ3_M | 836 | 35.4048 | 24.62 | 98.40 | 0.72 |
| Q3_K_M | 881 | 36.7423 | 25.95 | 94.82 | 0.77 |
| Q3_K_L | 935 | 37.9153 | 27.54 | 91.88 | 0.80 |
| IQ4_XS | 972 | 37.3032 | 28.63 | 93.39 | 0.79 |
| IQ4_NL | 1018 | 37.4248 | 29.99 | 93.09 | 0.79 |
| Q4_0 | 1019 | 36.6871 | 30.01 | 94.96 | 0.77 |
| Q4_K_S | 1021 | 35.4185 | 30.07 | 98.36 | 0.74 |
| Q4_K_M | 1065 | 35.2520 | 31.37 | 98.82 | 0.73 |
| Q4_1 | 1108 | 35.9411 | 32.64 | 96.93 | 0.75 |
| Q5_K_S | 1200 | 35.3946 | 35.35 | 98.43 | 0.74 |
| Q5_0 | 1203 | 36.2193 | 35.43 | 96.18 | 0.76 |
| Q5_K_M | 1225 | 34.7903 | 36.08 | 100.14 | 0.72 |
| Q5_1 | 1292 | 35.1618 | 38.06 | 99.08 | 0.73 |
| Q6_K | 1396 | 34.7554 | 41.12 | 100.24 | 0.72 |
| Q8_0 | 1806 | 34.9765 | 53.19 | 99.60 | 0.73 |
| F16 | 3395 | 34.8374 | 100 | 100 | 0.73 |
|