Add native support for toxicity detection guardrail microservice #1258
Conversation
for more information, see https://pre-commit.ci
daniel-de-leon-user293
requested review from
lvliang-intel,
letonghan,
ftian1 and
chensuyue
as code owners
Signed-off-by: Daniel Deleon <[email protected]>
Signed-off-by: Daniel Deleon <[email protected]>
comps/guardrails/src/toxicity_detection/opea_toxicity_detection_microservice.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
LGTM!
Please make sure to address @lvliang-intel 's comment.
Thanks.
Signed-off-by: Daniel Deleon <[email protected]>
Signed-off-by: Daniel Deleon <[email protected]>
Signed-off-by: Daniel Deleon <[email protected]>
for more information, see https://pre-commit.ci
|
@daniel-de-leon-user293,
|
@lvliang-intel do you know what would make those links invalid? |
@chensuyue, |
I guess it's due to the unstable connection of huggingface, it's pass now. |
Description
After the re-architecture, Intel/toxic-prompt-roberta was removed from the toxicity detection microservice. This PR brings back the free, token-less, lightweight model as the default option for users.
The new component name, used by default if no component name is passed, is
OPEA_NATIVE_TOXICITY. Users who have a Prediction Guard API key must now setTOXICITY_DETECTION_COMPONENT_NAMEtoPREDICTIONGUARD_TOXICITY_DETECTION.The reason for choosing to default to
Intel/toxic-promp-robertais that it reduces friction for new users. Acquiring a Prediction Guard API key adds an extra step for new users who may be trying to use OPEA under the assumption it is free and open-source.Issues
n/aType of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
None
Tests
Ran README manually and toxicity detection test script
test_guardrails_bias_detection_toxicdetection.pySigned-off-by: Daniel Deleon [email protected]