Allow configureing the litellm Timeout

[IMbackK]

I seams (from the documentation) that there is no way to configure the litellm timeout, this is a problem as i am trying to use openahands with a local openai compatible server (tabbyai and aphrodite). Sometimes this can be very slow to return a compleation request, in this case openhands will timeout even though there is nothing wrong, the backend just needs more time.

[mamoodi]

If you are running the docker command, you can add:
-e SANDBOX_TIMEOUT=360

or whatever timeout you require to the command, and I believe it will work.

If you are running through the development workflow,
In the config.toml:
[sandbox]
timeout = 360

Let me know if this resolves your issue. I know for example if you wanted to install Java which takes a long time you have to modify this command.

If I misunderstood litellm timeout I apologize.

[neubig]

Hey @mamoodi , maybe it's actually LLM_TIMEOUT?

OpenHands/openhands/core/schema/config.py

Line 37 in a753bab

LLM_TIMEOUT = 'LLM_TIMEOUT'

[mamoodi]

Ah thank you! Please try the above suggestion by passing in:
-e LLM_TIMEOUT=<your timeout>

to the docker command.

[IMbackK]

yes that seams to do the trick, i seams it might make sense to document the options provided by envvar via config.py

[mamoodi]

Yep we have an issue open for that! I believe someone is working on it. Thank you!