perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	1815	29.3739	12.49	49.92	0.53
IQ1_M	1947	23.4611	13.40	62.50	0.42
IQ2_XXS	2167	23.8257	14.91	61.54	0.46
IQ2_XS	2354	20.5413	16.20	71.38	0.39
IQ2_S	2475	19.3763	17.03	75.67	0.36
IQ2_M	2651	22.3007	18.24	65.75	0.44
Q2_K_S	2702	17.5446	18.59	83.57	0.31
Q2_K	2876	16.9426	19.79	86.54	0.29
IQ3_XXS	2970	16.2668	20.44	90.14	0.29
IQ3_XS	3191	16.1443	21.96	90.82	0.29
Q3_K_S	3330	17.0364	22.92	86.07	0.29
IQ3_S	3337	16.1048	22.96	91.04	0.29
IQ3_M	3408	15.8128	23.45	92.72	0.28
Q3_K_M	3631	15.2580	24.99	96.10	0.26
Q3_K_L	3899	15.1997	26.83	96.46	0.26
IQ4_XS	4023	14.9385	27.68	98.15	0.25
IQ4_NL	4232	14.9257	29.12	98.24	0.25
Q4_0	4238	15.2621	29.17	96.07	0.26
Q4_K_S	4251	14.8852	29.25	98.50	0.26
Q4_K_M	4466	14.8666	30.73	98.63	0.26
Q4_1	4647	14.8789	31.98	98.54	0.26
Q5_K_S	5068	14.7449	34.88	99.44	0.25
Q5_0	5081	14.7425	34.97	99.46	0.25
Q5_K_M	5192	14.7327	35.73	99.52	0.25
Q5_1	5490	14.7293	37.78	99.55	0.25
Q6_K	5964	14.6907	41.04	99.81	0.25
Q8_0	7723	14.6686	53.15	99.96	0.25
F16	14531	14.6625	100	100	0.25