Updated inference recommendations

README.md

@@ -177,18 +177,14 @@ Based on the listed rankings as of 4/12/24, is the top-rank 8B model.
 
 # Inference Settings
 
-Personal recommendations are:
+Personal recommendations are to use an [i1-Q4_K_M](https://www.reddit.com/r/LocalLLaMA/comments/1ck76rk/weightedimatrix_vs_static_quants/) quant with these settings:
 
 ```
-mirostat = 2
-mirostat_eta = 0.1
-mirostat_tau = 4.24
 num_ctx = 4096
-repeat_penalty = 1.4
+repeat_penalty = 1.2
 temperature = 0.85
-seed = 42
 top_k = 0
-top_p = 0.95
+top_p = 1
 ```
 
-After cross-referencing [this paper on mobile LLMs](https://openreview.net/pdf?id=ahVsd1hy2W) and [this paper on balancing model parameters](https://arxiv.org/html/2408.13586v1).
+Other recommendations can be found on [this paper on mobile LLMs](https://openreview.net/pdf?id=ahVsd1hy2W) and [this paper on balancing model parameters](https://arxiv.org/html/2408.13586v1).