Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Cannot recover from "Agent stuck in loop" #5480

Closed
1 task done
xingyaoww opened this issue Dec 9, 2024 · 27 comments · Fixed by #5597
Closed
1 task done

[Bug]: Cannot recover from "Agent stuck in loop" #5480

xingyaoww opened this issue Dec 9, 2024 · 27 comments · Fixed by #5597
Labels
bug Something isn't working severity:medium Affecting multiple users

Comments

@xingyaoww
Copy link
Collaborator

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Describe the bug and reproduction steps

Once we stuck in "agent stuck in loop", we cannot send subsequent messages to the agent anymore.

image

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response
@xingyaoww xingyaoww added the bug Something isn't working label Dec 9, 2024
@xingyaoww
Copy link
Collaborator Author

@openhands-agent any idea?

@openhands-agent
Copy link
Contributor

OpenHands started fixing the issue! You can monitor the progress here.

@mamoodi mamoodi added the severity:medium Affecting multiple users label Dec 9, 2024
@openhands-agent
Copy link
Contributor

An attempt was made to automatically fix this issue, but it was unsuccessful. A branch named 'openhands-fix-issue-5480' has been created with the attempted changes. You can view the branch here. Manual intervention may be required.

@enyst
Copy link
Collaborator

enyst commented Dec 10, 2024

This proposal is rather tough: main...openhands-fix-issue-5480#diff-1e082f415afb40de1d5c2a3fa327326225e1da28a142ad89c9524109f35fb36eR197

@openhands-agent Can you look at how traffic control is implemented, paused in the UI when necessary, and restarting later when the user does something? That can serve as example for this issue.

@openhands-agent
Copy link
Contributor

OpenHands started fixing the issue! You can monitor the progress here.

@openhands-agent
Copy link
Contributor

A potential fix has been generated and a draft PR #5500 has been created. Please review the changes.

@openhands-agent openhands-agent mentioned this issue Dec 10, 2024
Closed
enyst pushed a commit that referenced this issue Dec 14, 2024
@openhands-agent
fix: improve stuck detection in UI mode
1e739bd 
- Add UI mode awareness to stuck detection
- Only consider history after last user message in UI mode
- Keep existing behavior in headless mode
- Add comprehensive tests for both modes

Fix: #5480
@enyst enyst mentioned this issue Dec 14, 2024
Closed
enyst pushed a commit that referenced this issue Dec 14, 2024
@openhands-agent
fix: improve stuck detection in interactive mode
40bb318 
- Use headless_mode flag to determine stuck detection behavior
- In interactive mode (not headless), only consider history after last user message
- Keep existing behavior in headless mode
- Add comprehensive tests for both modes

Fix: #5480
@enyst enyst mentioned this issue Dec 14, 2024
Merged
1 task
@avi12
Copy link

avi12 commented Dec 15, 2024

I also faced this issue
image

@enyst
Copy link
Collaborator

enyst commented Dec 15, 2024

@avi12 It's fixed on main, please if you can, run openhands with :main instead of :0.15, or there will be a new release within days.

@avi12
Copy link

avi12 commented Dec 15, 2024

I'd appreciate it if you could update the documentation accordingly

@enyst
Copy link
Collaborator

enyst commented Dec 15, 2024

Sorry, I should clarify, the main branch is the development branch. We generally recommend to run actual releases, not the development branch. When there will be a new release - within days - we may update the documentation with that new version.

It's up to you, though, if you want to get the fix earlier: you can run openhands from main. But if you don't want the risk - there is a risk other things are bugged - then please don't. I'm sure it will be released soon, and releases are tested for major functionalities working ok.

@avi12
Copy link

avi12 commented Dec 15, 2024

Thank you for the information, it's worth mentioning both main and the latest version in the documentation so that the user can decide for himself

@avi12
Copy link

avi12 commented Dec 16, 2024

@enyst I'm on the main branch
image

@enyst
Copy link
Collaborator

enyst commented Dec 16, 2024

Was the Pause/Stop button available? I'm trying to reproduce the issue.

@avi12
Copy link

avi12 commented Dec 17, 2024

As far as I can tell, whenever the agent is oscillating between two prompts and can't get to a different prompt, at some point it will tell me that the agent got stuck in a loop, which makes sense
And then it will pause automatically so that I can send a prompt

@enyst
Copy link
Collaborator

enyst commented Dec 17, 2024

Right! But when you send a message, actually typed words and send to the agent, did it restart its actions?

@avi12
Copy link

avi12 commented Dec 17, 2024

Yep, only to get stuck again
image

@enyst
Copy link
Collaborator

enyst commented Dec 17, 2024

Ah, if I see this right... it did several more steps, and got stuck later again.

What model are you using?

@CitroenGames
Copy link

CitroenGames commented Dec 17, 2024 •
edited
Loading

image
image
this is the issue it says success but the command fails

@enyst
Copy link
Collaborator

enyst commented Dec 17, 2024

@CitroenGames There is another issue for that, please see here, and feel free to add your experience:
#5637

I would still love to hear what model / provider are you using, when you are getting a lot of stuck cases. It does matter! The model decides what steps to perform.

@avi12
Copy link

avi12 commented Dec 17, 2024

Ah, if I see this right... it did several more steps, and got stuck later again.

What model are you using?

https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF running via https://lmstudio.ai

@BradKML
Copy link

BradKML commented Dec 18, 2024 •
edited
Loading

I have also encountered this issue as well and could you try Phi-3/Phi-4 or DeepSeek Coder @avi12 I would like to see if this is an isolated problem #5310

@avi12
Copy link

avi12 commented Dec 18, 2024

@BradKML Funny enough, when used https://huggingface.co/matteogeniaccio/phi-4 I encountered #4586 (comment)

@avi12
Copy link

avi12 commented Dec 18, 2024

After trying out https://huggingface.co/lmstudio-community/Yi-Coder-9B-Chat-GGUF, I reproduced the error https://www.all-hands.dev/share?share_id=3a4846c320bba583aa836da5dcafe53712c08a42d8020f345edeaeab8b7c623f

@kasinadhsarma
Copy link

image
same got agent stuck loop again

@avi12
Copy link

avi12 commented Dec 24, 2024

@kasinadhsarma I think it partially has to do with the LLM that you choose, try the one that OH recommends, Claude 3.5 Sonnet

@BradKML
Copy link

BradKML commented Dec 25, 2024

@avi12 could it work with the OpenRouter version of Sonnet, or official only?

@avi12
Copy link

avi12 commented Dec 25, 2024

@avi12 could it work with the OpenRouter version of Sonnet, or official only?

IDK, I haven't gone that far

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working severity:medium Affecting multiple users
Projects
None yet
Milestone
No milestone
8 participants
@enyst @avi12 @xingyaoww @CitroenGames @BradKML @kasinadhsarma @mamoodi @openhands-agent