Skip to content

Commit

Permalink
[Agent] (Potentially) improve Editing using diff (#2685)
Browse files Browse the repository at this point in the history
* add replace-based block edit & preliminary test case fix

* further fix the insert behavior

* make edit only work on first occurence

* bump codeact version since we now use new edit agentskills

* update prompt for new agentskills

* update integration tests

* make run_infer.sh executable

* remove code block for edit_file

* update integration test for prompt changes

* default to not use hint for eval

* fix insert emptyfile bug

* throw value error when `to_replace` is empty

* make `_edit_or_insert_file` return string so we can try to fix some linter errors (best attempt)

* add todo

* update integration test

* fix sandbox test for this PR
  • Loading branch information
xingyaoww authored Jul 2, 2024
1 parent 6a0ffc5 commit 41ddba8
Show file tree
Hide file tree
Showing 47 changed files with 2,689 additions and 1,367 deletions.
2 changes: 1 addition & 1 deletion agenthub/codeact_agent/codeact_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def get_in_context_example() -> str:


class CodeActAgent(Agent):
VERSION = '1.6'
VERSION = '1.7'
"""
The Code Act Agent is a minimalist agent.
The agent works by passing the model a list of action-observation pairs and prompting the model to take the next step.
Expand Down
22 changes: 17 additions & 5 deletions agenthub/codeact_agent/prompt.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
COMMAND_DOCS = (
'\nApart from the standard Python library, the assistant can also use the following functions (already imported) in <execute_ipython> environment:\n'
f'{_AGENT_SKILLS_DOCS}'
"Please note that THE `edit_file` and `append_file` FUNCTIONS REQUIRE PROPER INDENTATION. If the assistant would like to add the line ' print(x)', it must fully write that out, with all those spaces before the code! Indentation is important and code that is not indented correctly will fail and require fixing before it can be run."
"Please note that THE `edit_file` and `insert_content_at_line` FUNCTIONS REQUIRE PROPER INDENTATION. If the assistant would like to add the line ' print(x)', it must fully write that out, with all those spaces before the code! Indentation is important and code that is not indented correctly will fail and require fixing before it can be run."
)

# ======= SYSTEM MESSAGE =======
Expand Down Expand Up @@ -75,7 +75,12 @@ def index():
if __name__ == '__main__':
app.run(port=5000)\"\"\"
edit_file('app.py', start=1, end=1, content=EDITED_CODE)
insert_content_at_line(
'app.py',
1,
EDITED_CODE,
)
</execute_ipython>
USER:
Expand All @@ -90,6 +95,7 @@ def index():
8|
9|if __name__ == '__main__':
10| app.run(port=5000)
11|
[File updated. Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
ASSISTANT:
Expand Down Expand Up @@ -199,7 +205,7 @@ def index():
</execute_ipython>
USER:
[File: /workspace/app.py (10 lines total)]
[File: /workspace/app.py (11 lines total)]
1|from flask import Flask
2|app = Flask(__name__)
3|
Expand All @@ -210,16 +216,21 @@ def index():
8|
9|if __name__ == '__main__':
10| app.run(port=5000)
11|
ASSISTANT:
I should edit the file to display the numbers in a table format. I should include correct indentation. Let me update the file:
<execute_ipython>
edit_file('app.py', start=7, end=7, content=" return '<table>' + ''.join([f'<tr><td>{i}</td></tr>' for i in numbers]) + '</table>'")
edit_file(
'app.py',
to_replace=" return str(numbers)",
new_content=" return '<table>' + ''.join([f'<tr><td>{i}</td></tr>' for i in numbers]) + '</table>'",
)
</execute_ipython>
USER:
Observation:
[File: /workspace/app.py (10 lines total after edit)]
[File: /workspace/app.py (11 lines total after edit)]
1|from flask import Flask
2|app = Flask(__name__)
3|
Expand All @@ -230,6 +241,7 @@ def index():
8|
9|if __name__ == '__main__':
10| app.run(port=5000)
11|
[File updated. Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
ASSISTANT:
Expand Down
Empty file modified evaluation/swe_bench/scripts/run_infer.sh
100644 → 100755
Empty file.
Loading

0 comments on commit 41ddba8

Please sign in to comment.