perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	489	436.7686	14.40	8.68	9.93
IQ1_M	515	188.5540	15.17	20.11	4.19
IQ2_XXS	560	131.2225	16.49	28.90	2.98
IQ2_XS	597	76.8887	17.58	49.32	1.62
IQ2_S	633	61.0257	18.65	62.14	1.26
IQ2_M	668	54.0953	19.67	70.10	1.11
Q2_K_S	683	62.3147	20.12	60.85	1.32
Q2_K	718	54.6519	21.15	69.39	1.14
IQ3_XXS	733	44.2555	21.59	85.69	0.90
IQ3_XS	793	41.2959	23.36	91.83	0.82
Q3_K_S	821	42.9523	24.18	88.29	0.87
IQ3_S	822	41.0938	24.21	92.28	0.82
IQ3_M	836	40.2980	24.62	94.10	0.80
Q3_K_M	881	41.7783	25.95	90.77	0.86
Q3_K_L	935	42.7356	27.54	88.73	0.88
IQ4_XS	972	40.5138	28.63	93.60	0.83
IQ4_NL	1018	40.9880	29.99	92.52	0.84
Q4_0	1019	40.6287	30.01	93.33	0.84
Q4_K_S	1021	39.5842	30.07	95.80	0.81
Q4_K_M	1065	39.5447	31.37	95.89	0.80
Q4_1	1108	39.0552	32.64	97.10	0.79
Q5_K_S	1200	39.0222	35.35	97.18	0.80
Q5_0	1203	39.4241	35.43	96.19	0.80
Q5_K_M	1225	38.6085	36.08	98.22	0.79
Q5_1	1292	38.5234	38.06	98.43	0.78
Q6_K	1396	38.1520	41.12	99.39	0.77
Q8_0	1806	37.9817	53.19	99.84	0.77
F16	3395	37.9208	100	100	0.77