perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	301	32.6390	31.75	69.27	0.59
IQ1_M	303	29.9721	31.96	75.43	0.53
IQ2_XXS	306	28.2728	32.28	79.97	0.52
IQ2_XS	309	26.5764	32.59	85.07	0.48
IQ2_S	310	26.3276	32.70	85.88	0.48
IQ2_M	313	25.6364	33.02	88.19	0.46
Q2_K_S	315	25.6897	33.23	88.01	0.46
IQ3_XXS	318	24.6070	33.54	91.88	0.44
Q3_K_S	322	24.5354	33.97	92.15	0.44
IQ3_S	322	24.0335	33.97	94.07	0.43
IQ3_XS	322	24.0335	33.97	94.07	0.43
Q2_K	322	24.4565	33.97	92.45	0.44
IQ3_M	326	23.8652	34.39	94.74	0.43
IQ4_XS	333	23.6261	35.13	95.70	0.42
IQ4_NL	336	23.6102	35.44	95.76	0.42
Q4_0	336	23.8547	35.44	94.78	0.42
Q3_K_M	338	23.5268	35.65	96.10	0.42
Q3_K_L	352	23.3214	37.13	96.95	0.42
Q4_1	357	23.3396	37.66	96.87	0.42
Q4_K_S	367	23.2008	38.71	97.45	0.41
Q5_0	379	23.1914	39.98	97.49	0.41
Q4_K_M	379	23.1306	39.98	97.75	0.41
Q5_K_S	393	22.8146	41.46	99.10	0.41
Q5_1	399	22.8381	42.09	99.00	0.41
Q5_K_M	400	22.7761	42.19	99.27	0.41
Q6_K	482	22.6302	50.84	99.91	0.40
Q8_0	506	22.6359	53.38	99.88	0.40
F16	948	22.6094	100	100	0.40