Usable context length?
#1
by
belisarius
- opened
Whats the maximum context this can take before producing nonsense?
The context window for Mixtral 4x7B is technically unlimited, but it operates with a 4K sliding window. Mixtral claims it's 32k but each expert was trained on 8k.
I run it with LMStudio set to 8k with the rolling context on, that and using Novelcrafter with custom prompts and the codex system I regularly get 16k or more but YMMV.