You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Establish a pattern for probes that can queue prompts based on existing output.
Motivation
Currently most probes build a single batch of prompts. We want to focus on adaptive approaches which may involve looking at LLM responses to some initial prompts, or even to the detection results over those initial prompts. While a few probes do this today, it's time to establish a pattern for how to do this efficiently as we integrate more adaptive probes.
Basic example
Let's construct base classes or an infrastructure plan for:
Probes that enqueue more prompts based on generator output from prior prompts
Probes that enqueue more prompts based on detector results over prior prompts
There are already some examples in atkgen and topic` respectively.
The text was updated successfully, but these errors were encountered:
Summary
Establish a pattern for probes that can queue prompts based on existing output.
Motivation
Currently most probes build a single batch of prompts. We want to focus on adaptive approaches which may involve looking at LLM responses to some initial prompts, or even to the detection results over those initial prompts. While a few probes do this today, it's time to establish a pattern for how to do this efficiently as we integrate more adaptive probes.
Basic example
Let's construct base classes or an infrastructure plan for:
There are already some examples in
atkgen and
topic` respectively.The text was updated successfully, but these errors were encountered: