Update README.md
Browse files
README.md
CHANGED
|
@@ -31,11 +31,12 @@ This training regimen, coupled with advanced techniques like manual learning rat
|
|
| 31 |
# How to install
|
| 32 |
|
| 33 |
On Huggingface, press the `Use this model` dropdown menu, and choose `Ollama`, then in the drop down menu, choose your quantization, following this GPU VRAM chart:
|
|
|
|
| 34 |
```
|
| 35 |
F16 = 20+ GB
|
| 36 |
Q8_0 = 12+ GB
|
| 37 |
-
Q5_K_M = 8
|
| 38 |
-
Q4_K_M = 8+ GB
|
| 39 |
Q3_K_M = 6-8 GB on Minecraft LOW settings
|
| 40 |
Q2_K_M = 6 GB on Minecraft LOW settings
|
| 41 |
```
|
|
|
|
| 31 |
# How to install
|
| 32 |
|
| 33 |
On Huggingface, press the `Use this model` dropdown menu, and choose `Ollama`, then in the drop down menu, choose your quantization, following this GPU VRAM chart:
|
| 34 |
+
*All of these values assume a context window size of 8192 or less*
|
| 35 |
```
|
| 36 |
F16 = 20+ GB
|
| 37 |
Q8_0 = 12+ GB
|
| 38 |
+
Q5_K_M = >8 GB
|
| 39 |
+
Q4_K_M = 8+ GB
|
| 40 |
Q3_K_M = 6-8 GB on Minecraft LOW settings
|
| 41 |
Q2_K_M = 6 GB on Minecraft LOW settings
|
| 42 |
```
|