perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	489	355.5202	14.40	9.80	8.19
IQ1_M	515	159.7892	15.17	21.80	3.59
IQ2_XXS	560	113.7759	16.49	30.62	2.62
IQ2_XS	597	66.9608	17.58	52.03	1.44
IQ2_S	633	55.5589	18.65	62.70	1.18
IQ2_M	668	49.2839	19.67	70.69	1.04
Q2_K_S	683	56.3132	20.12	61.86	1.22
Q2_K	718	52.9062	21.15	65.85	1.15
IQ3_XXS	733	40.4841	21.59	86.05	0.85
IQ3_XS	793	37.9592	23.36	91.78	0.78
Q3_K_S	821	40.8887	24.18	85.20	0.86
IQ3_S	822	37.3800	24.21	93.20	0.77
IQ3_M	836	35.4048	24.62	98.40	0.72
Q3_K_M	881	36.7423	25.95	94.82	0.77
Q3_K_L	935	37.9153	27.54	91.88	0.80
IQ4_XS	972	37.3032	28.63	93.39	0.79
IQ4_NL	1018	37.4248	29.99	93.09	0.79
Q4_0	1019	36.6871	30.01	94.96	0.77
Q4_K_S	1021	35.4185	30.07	98.36	0.74
Q4_K_M	1065	35.2520	31.37	98.82	0.73
Q4_1	1108	35.9411	32.64	96.93	0.75
Q5_K_S	1200	35.3946	35.35	98.43	0.74
Q5_0	1203	36.2193	35.43	96.18	0.76
Q5_K_M	1225	34.7903	36.08	100.14	0.72
Q5_1	1292	35.1618	38.06	99.08	0.73
Q6_K	1396	34.7554	41.12	100.24	0.72
Q8_0	1806	34.9765	53.19	99.60	0.73
F16	3395	34.8374	100	100	0.73