Please Unban EOS Token by default

[WolframRavenwolf]

With --unbantokens being deprecated, I think it's time to unban the EOS token by default. llama.cpp already does that, with banning of the EOS token a command line argument (--ignore-eos), as does oobabooga's text-generation-webui ("Ban the eos_token" off by default).

By unbanning the EOS token by default, we'd get koboldcpp to be consistent with the software it's based on as well as with another popular option, and with modern models the EOS token is best practice for fine-tuning to avoid rambling, hallucinations and conversations derailing. The "The KoboldCpp FAQ and Knowledgebase" mentions that as well, although IMHO it doesn't go far enough to recommend using this option (especially considering many coherency issues that were solved by unbanning the EOS token, as a quick Google search showed).

I'm pushing for this change in the backend now that the command line option is deprecated because frontends might prefer to stick with the defaults of the inference software, and overriding that from the model startup batchfile (the way I've been doing it thus far) won't be possible anymore. So I hope you agree with my reasoning why this change makes sense and make "unbantokens" the default once the command line argument is removed.

(By the way, stopping strings aren't a good alternative to the EOS token: Since the EOS token never appears in the input, it's never affected by repetition penalty. Stopping strings however often use the user's name and a colon, which are tokens that can and most likely will appear in the input multiple times (especially the user name), so repetition penalty will eventually restrict them and cause messages to get longer and less coherent as the conversation continues, until the chat derails completely till it hits max new tokens.)

[LostRuins]

The good part is that with --unbantokens being offloaded to the API, it is possible to toggle token banning on and off on demand at runtime depending on use case, which is what Lite does by itself now

I agree that allowing EOS as the default if not specified in the API payload does make sense. Of course no client should be using the default if they can avoid it - and this may be subject to change at a later date, so I would still recommend you send use_default_badwordsids = false in your payloads to be safer.

[WolframRavenwolf]

That's a very clever implementation!

So I propose the backend's default being to allow the EOS token. And frontends can handle their settings themselves, perhaps even by implementing an auto-selection like you did, which was a genius move.