Harness that orders probes for expected attack success rate (ASR) according to bag

[leondz]

Create a harness that orders probes so that the ones with highest known attack success rate are run first

Summary

Currently harnesses order probes alphabetically.

Create a harness that can look at the calibration data and order probes with the known highest attack success rates first.

Possible config variables here:

• Not every probe is in the data. We should give a threshold for when to run probes on which there's no data. Setting it to 100% makes these run first, to 0% makes them run at the end (maybe with other 0% probes), and other values leave them in the middle.
• Maybe we want to skip probes that aren't mentioned in the attack data (and log this)
• Probes in calibration data may have more than one detector run on them. Which ASR do we choose? Options include max, min, mean, median; default max, I think
• Do we want to give the calibration data path a config var? Default is data/calibration/calibration.json

Basic example

$ garak -m test -p encoding.InjectAscii85,encoding.InjectBase64
...
  queue of probes: encoding.InjectBase64, encoding.InjectAscii85

Note non-alphabetic ordering of probes

[leondz]

Hi @clivinn-shla81092 ,

If you'd like to contribute, please

a. open a pull request
b. add documentation for the contributed code
c. add tests for new functions/classes

We have a guide to contributing here - I hope it's useful!