perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	489	88.4250	14.40	23.35	1.76
IQ1_M	516	53.8278	15.19	38.35	1.03
IQ2_XXS	560	45.5693	16.49	45.31	0.93
IQ2_XS	598	32.6813	17.61	63.17	0.62
IQ2_S	633	28.5477	18.64	72.32	0.54
IQ2_M	669	31.8272	19.70	64.87	0.63
Q2_K_S	683	28.7707	20.11	71.76	0.54
Q2_K	718	27.6342	21.14	74.71	0.51
IQ3_XXS	733	23.5511	21.58	87.66	0.44
IQ3_XS	793	22.9887	23.35	89.81	0.42
Q3_K_S	821	28.0462	24.17	73.61	0.53
IQ3_S	822	22.9268	24.20	90.05	0.42
IQ3_M	836	22.3167	24.62	92.51	0.41
Q3_K_M	881	22.5727	25.94	91.46	0.41
Q3_K_L	935	22.3758	27.53	92.27	0.41
IQ4_XS	972	21.3273	28.62	96.80	0.38
IQ4_NL	1018	21.3234	29.98	96.82	0.38
Q4_0	1019	22.5210	30.00	91.67	0.41
Q4_K_S	1022	21.1717	30.09	97.51	0.38
Q4_K_M	1065	21.0532	31.36	98.06	0.38
Q4_1	1109	21.1492	32.66	97.62	0.38
Q5_K_S	1201	20.7883	35.37	99.31	0.37
Q5_0	1203	20.8643	35.42	98.95	0.37
Q5_K_M	1226	20.7488	36.10	99.50	0.37
Q5_1	1293	20.7773	38.07	99.37	0.37
Q6_K	1396	20.6994	41.11	99.74	0.37
Q8_0	1807	20.6659	53.21	99.90	0.37
F16	3396	20.6457	100	100	0.37