DavidAU
/

Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF

Model card Files Files and versions Community

DavidAU commited on Dec 11, 2024

Commit

c0a3e72

·

verified ·

1 Parent(s): a37a3d2

Update README.md

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -113,7 +113,9 @@ You can set the number of experts in LMStudio (https://lmstudio.ai) at the "load
 For Text-Generation-Webui (https://github.com/oobabooga/text-generation-webui)  you set the number of experts at the loading screen page.
 For server.exe / Llama-server.exe (Llamacpp - https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md )
-add the following (CLI): "--override-kv llama.expert_used_count=int:6"
 (no quotes, where "6" is the number of experts to use)

 For Text-Generation-Webui (https://github.com/oobabooga/text-generation-webui)  you set the number of experts at the loading screen page.
 For server.exe / Llama-server.exe (Llamacpp - https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md )
+add the following to the command line to start the "llamacpp server" (CLI):
+"--override-kv llama.expert_used_count=int:6"
 (no quotes, where "6" is the number of experts to use)