| | Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate | | |
| | ------ | --------- | ------- | -------- | ------------ | -------------- | | |
| | IQ1_S | 1815 | 29.3739 | 12.49 | 49.92 | 0.53 | | |
| | IQ1_M | 1947 | 23.4611 | 13.40 | 62.50 | 0.42 | | |
| | IQ2_XXS | 2167 | 23.8257 | 14.91 | 61.54 | 0.46 | | |
| | IQ2_XS | 2354 | 20.5413 | 16.20 | 71.38 | 0.39 | | |
| | IQ2_S | 2475 | 19.3763 | 17.03 | 75.67 | 0.36 | | |
| | IQ2_M | 2651 | 22.3007 | 18.24 | 65.75 | 0.44 | | |
| | Q2_K_S | 2702 | 17.5446 | 18.59 | 83.57 | 0.31 | | |
| | Q2_K | 2876 | 16.9426 | 19.79 | 86.54 | 0.29 | | |
| | IQ3_XXS | 2970 | 16.2668 | 20.44 | 90.14 | 0.29 | | |
| | IQ3_XS | 3191 | 16.1443 | 21.96 | 90.82 | 0.29 | | |
| | Q3_K_S | 3330 | 17.0364 | 22.92 | 86.07 | 0.29 | | |
| | IQ3_S | 3337 | 16.1048 | 22.96 | 91.04 | 0.29 | | |
| | IQ3_M | 3408 | 15.8128 | 23.45 | 92.72 | 0.28 | | |
| | Q3_K_M | 3631 | 15.2580 | 24.99 | 96.10 | 0.26 | | |
| | Q3_K_L | 3899 | 15.1997 | 26.83 | 96.46 | 0.26 | | |
| | IQ4_XS | 4023 | 14.9385 | 27.68 | 98.15 | 0.25 | | |
| | IQ4_NL | 4232 | 14.9257 | 29.12 | 98.24 | 0.25 | | |
| | Q4_0 | 4238 | 15.2621 | 29.17 | 96.07 | 0.26 | | |
| | Q4_K_S | 4251 | 14.8852 | 29.25 | 98.50 | 0.26 | | |
| | Q4_K_M | 4466 | 14.8666 | 30.73 | 98.63 | 0.26 | | |
| | Q4_1 | 4647 | 14.8789 | 31.98 | 98.54 | 0.26 | | |
| | Q5_K_S | 5068 | 14.7449 | 34.88 | 99.44 | 0.25 | | |
| | Q5_0 | 5081 | 14.7425 | 34.97 | 99.46 | 0.25 | | |
| | Q5_K_M | 5192 | 14.7327 | 35.73 | 99.52 | 0.25 | | |
| | Q5_1 | 5490 | 14.7293 | 37.78 | 99.55 | 0.25 | | |
| | Q6_K | 5964 | 14.6907 | 41.04 | 99.81 | 0.25 | | |
| | Q8_0 | 7723 | 14.6686 | 53.15 | 99.96 | 0.25 | | |
| | F16 | 14531 | 14.6625 | 100 | 100 | 0.25 | | |