Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Replace pexpect with libtmux in BashSession #4881

Merged
merged 285 commits into from
Jan 3, 2025

Conversation

xingyaoww
Copy link
Collaborator

@xingyaoww xingyaoww commented Nov 10, 2024

This PR replaces the pexpect-based shell interaction with a more robust libtmux implementation, providing better reliability and functionality for command execution in OpenHands.

Key Improvements

  • Replaced pexpect with libtmux for more reliable shell interaction
  • Added proper handling of command errors, interactive commands, and timeouts
  • Improved output capture and error detection
  • Added automatic resource scaling for retries during evaluation
  • Added comprehensive test suite to verify behavior

Technical Details

  • Removed keep_prompt parameter from CmdRunAction as it's no longer needed with tmux
  • Added tmux installation to all CI workflows (Ubuntu and macOS)
  • Updated backend architecture diagram to reflect the new implementation
  • Enhanced error handling with better timeout management
  • Added resource scaling for retries in evaluation (up to 4x resources for failed attempts)

End-user Benefits

  • More reliable command execution and output capture
  • Better handling of interactive commands
  • Improved error detection and reporting
  • Automatic retry with increased resources for failed commands

Development Notes

Collaborated with OpenHands:


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:31bdc15-nikolaik   --name openhands-app-31bdc15   docker.all-hands.dev/all-hands-ai/openhands:31bdc15
  -p 3000:3000 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --add-host host.docker.internal:host-gateway \
  -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:752f341-nikolaik \
  --name openhands-app-752f341 \
  docker.all-hands.dev/all-hands-ai/openhands:752f341

@diwu-sf
Copy link
Contributor

diwu-sf commented Nov 15, 2024

Wow, this is really happening! Thank you so much!
With tmux, I suspect it will be easier to support "multi-terminals" like what Devin has, and have higher fidelity terminal "screen" captures to render in frontend.

@xingyaoww
Copy link
Collaborator Author

Finally figured out the resolve rate degradation issue! Now we get comparable performance on SWE-Bench Verified: 238 (tmux, this PR) vs 241 (reproduce-commit from 53% verified) resolved.

image

I will clean up the code and try to get this finally merged :D

@xingyaoww xingyaoww marked this pull request as ready for review January 3, 2025 15:42
@xingyaoww xingyaoww requested a review from rbren January 3, 2025 17:38
@xingyaoww
Copy link
Collaborator Author

Very excited to finally merge this almost 2-month-old PR with nearly 300 commits 😄

attach a screenshot showing this is working
image

@xingyaoww xingyaoww merged commit ec70af9 into main Jan 3, 2025
18 checks passed
@xingyaoww xingyaoww deleted the feature/tmux-shell branch January 3, 2025 21:22
@diwu-sf
Copy link
Contributor

diwu-sf commented Jan 3, 2025

Congrats on landing this one!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants