detector: StringDetector matchtype fullmatch

[leondz]

Summary

Detector for if generator output is a specific string

Add configurable option to strip() generator output first

Add configurable option to still match even if output truncates before entire match is found (to handle max_tokens clipping and similar)

Useful for e.g. shield models, or probes that specify a complete expectedoutput string

[Eric-Hacker]

It seems that some of this was just added with the startswith StringDetector. The strip() option would be a good thing to add.

The option to do a partial match can be controlled by the detector by specifying a shorter string to be detected with startswith.

The Shields detector is designed to support the match string(s) and matchtype as optional parameters so it is ready to be run time configured for whatever the expected output would be. I thought about adding more docs to show this, but thought that might get to be too much? It could also be part of a tutorial or something.

If a fullmatch is still needed, then the Shields detector should be updated to support it. Even if not, I think Shields should use strip() as I already ran into one situation where that would have been helpful. If agreed, assign the issue to me along with exactly which way to go. And by exactly, I mean if fullmatch is somehow going to have an option partial then please provide a better name that doesn't sounds so silly because I can't think of one. 😏