| ====== Perplexity statistics ====== | |
| Mean PPL(Q) : 19.112490 ± 0.165621 | |
| Mean PPL(base) : 14.009216 ± 0.118474 | |
| Cor(ln(PPL(Q)), ln(PPL(base))): 91.50% | |
| Mean ln(PPL(Q)/PPL(base)) : 0.310627 ± 0.003536 | |
| Mean PPL(Q)/PPL(base) : 1.364280 ± 0.004823 | |
| Mean PPL(Q)-PPL(base) : 5.103274 ± 0.074552 | |
| ====== KL divergence statistics ====== | |
| Mean KLD: 0.547460 ± 0.001918 | |
| Maximum KLD: 18.895590 | |
| 99.9% KLD: 6.605354 | |
| 99.0% KLD: 3.488673 | |
| 99.0% KLD: 3.488673 | |
| Median KLD: 0.329876 | |
| 10.0% KLD: 0.015589 | |
| 5.0% KLD: 0.004167 | |
| 1.0% KLD: 0.000469 | |
| Minimum KLD: 0.000003 | |
| ====== Token probability statistics ====== | |
| Mean Δp: -5.840 ± 0.049 % | |
| Maximum Δp: 99.980% | |
| 99.9% Δp: 71.276% | |
| 99.0% Δp: 39.178% | |
| 95.0% Δp: 15.935% | |
| 90.0% Δp: 7.096% | |
| 75.0% Δp: 0.211% | |
| Median Δp: -0.554% | |
| 25.0% Δp: -9.068% | |
| 10.0% Δp: -27.946% | |
| 5.0% Δp: -43.354% | |
| 1.0% Δp: -75.395% | |
| 0.1% Δp: -95.604% | |
| Minimum Δp: -99.970% | |
| RMS Δp : 19.434 ± 0.071 % | |
| Same top p: 67.651 ± 0.123 % | |