Bias entry for chat completions?
Has anyone made a bias entry for chat completions that can be imported into sillytavern? With some models, using that is unavoidable.
Not that I know of. And I don't see myself making one, or converting my KobolCPP list. Logit bias is a pretty unreliable way to deal with word bans, it can end up causing some collateral banning of unrelated phrases since some words share token overlaps. I wouldn't recommend it.
The bans are never bans, so even if there is collateral, the model will find some way to say what it wants. 100% it reduces this slop by some margin.
Ahh well.. i'll do it myself and if so, Ill post it up here.
Still more destructive than I would like. But please, go ahead, share it on a repo, and I can give you a boost by linking it in my index if it works well enough.
Ok.. i got them all into a json.. https://files.catbox.moe/sm9akh.json
Now to test and see what it does. Only -25 bias so shouldn't cause too many issues.
I have been slow for the last month, sorry.
I just checked your list, and yeah, -25 seems like a safe value in some quick tests. With sampler settings that don't flatten the token probabilities too much, it shouldn't affect much the AI's coherence.
I created a new sampling section on my index and shared your list there. https://rentry.org/Sukino-Findings#sampler-settings
If you ever make a repository with it to make updates, post in this thread that I will change the link there.
In the meantime, Avani also made a logit bias list, you might want to check it out.
Cheers.