Skip to content

Commit

Permalink
refact: action -> tool
Browse files Browse the repository at this point in the history
  • Loading branch information
evilsocket committed Feb 7, 2025
1 parent 01490e4 commit 5702ca8
Show file tree
Hide file tree
Showing 58 changed files with 413 additions and 434 deletions.
14 changes: 7 additions & 7 deletions docs/tasklets.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,15 +136,15 @@ For more information about the default namespaces see [the namespaces documentat

### Custom Tools

Additional tools can be defined in the tasklet's `functions` section, and each is a group of actions that can be used by the agent, defining a `name`, `description` and a `tool` field with the command to be executed:
Additional tools can be defined in the tasklet's `tool_box` section, and each is a group of tools that can be used by the agent, defining a `name`, `description` and a `tool` field with the command to be executed:

```yaml
# ... snippet ...

functions:
tool_box:
- name: News
decription: You will use this action to read the recent news.
actions:
tools:
- name: read_news
description: "To read the recent news:"
# the output of this command will be returned to the agent
Expand All @@ -157,9 +157,9 @@ If the agent must provide arguments to the tool, it is possible to define an exa

```yaml
# ... snippet ...
functions:
tool_box:
- name: Environment
actions:
tools:
- name: report_finding
description: When you are ready to report findings, use this tool for each finding.
example_payload: >
Expand All @@ -178,10 +178,10 @@ If the tool requires named arguments it is possible to define them in the `args`

```yaml
# ... snippet ...
functions:
tool_box:
- name: Conversation
description: You will use these actions to create conversational entries.
actions:
tools:
- name: talk
description: "To have one of the characters say a specific sentence:"
example_payload: hi, how are you doing today?
Expand Down
16 changes: 8 additions & 8 deletions docs/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ prompt: prepare a list of ingredients for $food
guidance:
- Once you have made a list of ingredients, use the create_list_of_ingredients tool to confirm the decision.
functions:
tool_box:
- name: Tools
actions:
tools:
- name: create_list_of_ingredients
description: "To provide the ingredients one per line as an organized list:"
store_to: ingredients
Expand All @@ -75,9 +75,9 @@ guidance:
- Use the describe_preparation_steps tool to describe each step in the preparation.
- Once you have described each step in the preparation of the pie set your task as complete.
functions:
tool_box:
- name: Tools
actions:
tools:
- name: describe_preparation_steps
description: "To provide the preparation steps one per line as an organized list:"
store_to: steps
Expand Down Expand Up @@ -108,9 +108,9 @@ prompt: >
guidance:
- Once you have made an estimation, use the estimate_time tool to confirm the decision.
functions:
tool_box:
- name: Tools
actions:
tools:
- name: estimate_time
description: "To provide the time it will take to prepare the food:"
store_to: preparation_time
Expand Down Expand Up @@ -142,9 +142,9 @@ prompt: >
guidance:
- Once you have completed the task, use the rewrite tool to confirm the decision.
functions:
tool_box:
- name: Tools
actions:
tools:
- name: rewrite
description: "To confirm your version of the recipe:"
store_to: report
Expand Down
12 changes: 5 additions & 7 deletions examples/ab_problem/eval.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,11 @@ def solve(xs):
def get_solution(message):
if (
message["type"] == "agent"
and "data" in message
and message["data"] is not None
and message["data"][1] is not None
and message["data"][1]["action"] == "solution"
and message["data"][1]["payload"] is not None
and "tool_call" in message["data"]
and message["data"]["tool_call"] is not None
and message["data"]["tool_call"]["tool_name"] == "solution"
):
return message["data"][1]["payload"].strip().split(" ")
return message["data"]["tool_call"]["argument"].strip().split(" ")

return None

Expand All @@ -90,7 +88,7 @@ def get_solution(message):
solution = None

# find the most recent solution in the chat history
for message in reversed(state["chat"]["history"]["conversation"]):
for message in reversed(state["chat"]["history"]["messages"]):
solution = get_solution(message)
if solution is not None:
if actual == solution:
Expand Down
6 changes: 3 additions & 3 deletions examples/ab_problem/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,10 @@ evaluator:
- python3
- eval.py

functions:
tool_box:
- name: Solve
description: You will use these actions to provide final answer to the problem.
actions:
description: You will use these tools to provide the final answer to the problem.
tools:
- name: solution
description: "To provide final result:"
example_payload: B#
Expand Down
11 changes: 4 additions & 7 deletions examples/ai_news/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,11 @@ prompt: >
guidance:
- You will first use the read_news command and then write the bullet points to your memory.

functions:
tool_box:
- name: News
decription: You will use this action to read the recent news.
actions:
decription: You will use this tools to read the recent news.
tools:
- name: read_news
description: "To read the recent news:"
max_shown_output: 4096
tool: curl -s getnews.tech/ai,nocolor



tool: curl -s getnews.tech/ai,nocolor
4 changes: 2 additions & 2 deletions examples/code_auditor/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ guidance:
prompt: >
find vulnerabilities in source code in $TARGET_PATH and report your findings.
functions:
tool_box:
- name: Report
actions:
tools:
- name: report_finding
description: Use this tool to report your findings.
example_payload: >
Expand Down
4 changes: 2 additions & 2 deletions examples/code_auditor_with_judge/judge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ guidance:
prompt: >
confirm by using the judge tool whether or not this is a vulnerability: $STDIN
functions:
tool_box:
- name: Judge
actions:
tools:
- name: confirm_vulnerability
description: Use this tool to confirm whether or not the finding is a vulnerability.
complete_task: true
Expand Down
4 changes: 2 additions & 2 deletions examples/code_auditor_with_judge/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ guidance:
prompt: >
find vulnerabilities in source code in $TARGET_PATH and report your findings.
functions:
tool_box:
- name: Report
actions:
tools:
- name: judge_finding
description: Use this tool to ask an external expert to judge if a finding is a vulnerability or not. If the expert confirms it is a vulnerability, use the report_finding tool to report it.
example_payload: >
Expand Down
4 changes: 2 additions & 2 deletions examples/docker-agent/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ guidance:
- Continue providing commands and processing outputs until you believe you have completed the task.
- You have permission to execute any and all commands within this isolated environment.

functions:
tool_box:
- name: Environment
description: You have permission to execute any and all commands within this isolated environment.
actions:
tools:
- name: execute_command
description: The command will be executed on your host and the output will be returned to you.
example_payload: "ls -la"
Expand Down
13 changes: 0 additions & 13 deletions examples/eval_test/eval.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,5 @@
if "42" in raw:
exit(42)

state = json.loads(raw)

# uncomment this to validate the output of a tool in the history
"""
# in this case we're looping the chat history, we could just do substring matching really ...
for message in state["chat"]["history"]["conversation"]:
if message["type"] == "feedback":
invocation = message["data"][1]
if invocation is not None:
if invocation["action"] == "solution" and "42" in invocation["payload"]:
exit(42)
"""

# add a feedback message to the chat history
print("try thinking about a funny book reference to answer")
4 changes: 2 additions & 2 deletions examples/eval_test/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ evaluator:

# tools are not needed here, the evaluator will just check the chat history

# functions:
# tool_box:
# - name: Solve
# description: You will use these actions to provide the answer to the problem.
# actions:
# tools:
# - name: solution
# description: "To provide the answer to the problem:"
# example_payload: foobar
Expand Down
4 changes: 2 additions & 2 deletions examples/fuzzer/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ guidance:
- Try to often change the first characters of the payload in order to possibly trigger different execution paths.

# the agent toolbox
functions:
tool_box:
# divided in action groups
- name: Fuzzing
description: You will use these actions to inspect the source code of the executable, and interact with the fuzzing session.
actions:
tools:
# let's give the model a way to check the source :D
- name: inspect
description: "To visualize the original source code of the executable:"
Expand Down
4 changes: 2 additions & 2 deletions examples/kali_pentester/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ guidance:
- If you need to use the command 'sudo' before something, determine if you are root and only use sudo if you are not.
- Run commands one by one and avoid using && or other strategies to run multiple commands at once.

functions:
tool_box:
- name: Commands
actions:
tools:
- name: command
description: "To execute a bash command on the Kali Linux computer:"
example_payload: whoami
Expand Down
4 changes: 2 additions & 2 deletions examples/life/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ guidance:
- Create life forms that can build and use structures.
- Create life forms that can build and use machines.

functions:
tool_box:
- name: World
actions:
tools:
- name: observe
description: "To observe the world:"
max_shown_output: 10000
Expand Down
4 changes: 2 additions & 2 deletions examples/recipe_workflow/create_list_of_ingredients.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ prompt: prepare a list of ingredients for $food
guidance:
- Once you have made a list of ingredients, use the create_list_of_ingredients tool to confirm the decision.

functions:
tool_box:
- name: Tools
actions:
tools:
- name: create_list_of_ingredients
description: "To provide the ingredients one per line as an organized list:"
store_to: ingredients
Expand Down
4 changes: 2 additions & 2 deletions examples/recipe_workflow/describe_preparation_steps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ guidance:
- Use the describe_preparation_steps tool to describe each step in the preparation.
- Once you have described each step in the preparation of the pie set your task as complete.

functions:
tool_box:
- name: Tools
actions:
tools:
- name: describe_preparation_steps
description: "To provide the preparation steps one per line as an organized list:"
store_to: steps
Expand Down
4 changes: 2 additions & 2 deletions examples/recipe_workflow/estimate_time.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ prompt: >
guidance:
- Once you have made an estimation, use the estimate_time tool to confirm the decision.

functions:
tool_box:
- name: Tools
actions:
tools:
- name: estimate_time
description: "To provide the time it will take to prepare the food:"
store_to: preparation_time
Expand Down
4 changes: 2 additions & 2 deletions examples/recipe_workflow/rewrite_nicely.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ prompt: >
guidance:
- Once you have completed the task, use the rewrite tool to confirm the decision.

functions:
tool_box:
- name: Tools
actions:
tools:
- name: rewrite
description: "To confirm your version of the recipe:"
store_to: report
Expand Down
4 changes: 2 additions & 2 deletions examples/screenshot/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ guidance:

prompt: Take a screenshot and describe it.

functions:
tool_box:
- name: Desktop
actions:
tools:
- name: take_screenshot
description: Take a screenshot of the current screen.
tool: ./screenshot.py
Expand Down
4 changes: 2 additions & 2 deletions examples/software_dev_workflow/chief_product_officer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ prompt: Prepare a list of high level requirements for $product
guidance:
- Once you have made a list of requirements, use the create_list_of_requirements tool to confirm the decision.

functions:
tool_box:
- name: Tools
actions:
tools:
- name: create_list_of_requirements
description: "To provide the requirements one per line as an organized list:"
store_to: requirements
Expand Down
4 changes: 2 additions & 2 deletions examples/software_dev_workflow/chief_technology_officer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ prompt: >
guidance:
- Once you have made a list of technologies, use the create_list_of_technologies tool to confirm the decision.

functions:
tool_box:
- name: Tools
actions:
tools:
- name: create_list_of_technologies
description: "To provide the technologies one per line as an organized list:"
store_to: tech_stack
Expand Down
4 changes: 2 additions & 2 deletions examples/software_dev_workflow/manager.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ prompt: >
guidance:
- Once you have made a list of tasks, use the create_list_of_tasks tool to confirm the decision.

functions:
tool_box:
- name: Tools
actions:
tools:
- name: create_list_of_tasks
description: "To provide the tasks one per line as an organized list:"
store_to: tasks
Expand Down
4 changes: 2 additions & 2 deletions examples/ssh_agent/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ guidance:
timeout: 120s

# the agent toolbox
functions:
tool_box:
# divided in namespaces
- name: Commands
actions:
tools:
- name: ssh
# explains to the model when to use this action
description: "To execute a bash command on the remote host via SSH:"
Expand Down
4 changes: 2 additions & 2 deletions examples/web_fingerprint/task.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ guidance:
- If a page returns a "not found" error stop requesting it and try another page.
- Use exclusively the report_findings tool to report your findings.

functions:
tool_box:
- name: Report
actions:
tools:
- name: report_finding
description: Use this tool to report EACH of your findings.
example_payload: >
Expand Down
Loading

0 comments on commit 5702ca8

Please sign in to comment.