General enquiries on model options & deployment

#2
by SpaceHunterInf - opened

Hi David,

Thanks again for the release, and for your continued work for the community.

I had a few quick questions/requests:

  1. Non-reasoning variant: Would it be feasible to train or release a faster “non-thinking” model (e.g., Gemma 3 27B–style) optimized for instruction following/roleplay without heavy reasoning?

  2. Roleplay use case: In your experience, which models currently do best for empathetic roleplay? Lately many feel less warm; the older Lyra-4 family still seems stronger to me. If I’m configuring things wrong, I’d love any tips (settings, prompts, system messages).

  3. Deployment docs: I usually serve via text-generation-webui / llama.cpp. The repo instructions look a bit out-of-date, could you revisit them or suggest a better deployment path?

Happy to share configs/logs if helpful. Also, if you have Patreon or Kickstarter, I’d be glad to support.

Thanks again

Owner

Hey;

Try these:
https://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF
https://huggingface.co/DavidAU/Mistral-MOE-4X7B-Dark-MultiVerse-Uncensored-Enhanced32-24B-gguf

Suggest Lmstudio and/or Koboldcpp (direct or via Silly Tavern).

These are MOES, non-thinking -> but you can dial up the power up/down by expert activation.
Default for both is "2" experts ;

If you want a LOT of firepower:
https://huggingface.co/DavidAU/Qwen3-Coder-42B-A3B-Instruct-TOTAL-RECALL-MASTER-CODER-M-512k-ctx
(quants links on this page)
(this is all use cases actually, not just coding)

For thinking - very short blocks ; this one:
https://huggingface.co/DavidAU/Qwen3-Jan-V1-4B-Grand-Horror-Day1-to-Day7-Evolved-Imatrix-GGUF

Try days 4-7 ; this model is exceptional, very fast, and "gets deep into character".
It has extraordinary instruction following abilities - perfect for RP.

There are ways to "block" the think blocks (in apps), so it does not ruin the RP.

FOR RP Prompts see [drop in the system prompt -> GO ; sometimes you need to add a few "notes" for better control]:
https://docsbot.ai/prompts/tags?tag=Roleplay

Sign up or log in to comment