Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force Pyro to timeout quickly and introduce retries #1095

Closed
wants to merge 1 commit into from

Conversation

LukasPukenis
Copy link
Contributor

@LukasPukenis LukasPukenis commented Jan 31, 2025

We observe that sometimes pyro remote calls are with time gaps in between which should not happen. We guess it might be due to network issues. However the packets still arrive and might be due to TCP retransmission.

Pyro has a timeout property which is not exposed publicly. This property is set and looped over if timeout error is returned to fix the issue.

Problem

--describe problem being solved--

Solution

--describe selected solution--

☑️ Definition of Done checklist

  • Commit history is clean (requirements)
  • README.md is updated
  • Functionality is covered by unit or integration tests

@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 2f6dec1 to 7b94f4e Compare January 31, 2025 12:34
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 7b94f4e to aa38753 Compare January 31, 2025 14:38
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from aa38753 to 5eda99f Compare February 3, 2025 06:36
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 5eda99f to 9f607fd Compare February 3, 2025 08:01
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 9f607fd to 5bb7779 Compare February 3, 2025 10:58
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 5bb7779 to 9c0b45d Compare February 3, 2025 10:59
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 9c0b45d to 32d7752 Compare February 3, 2025 13:21
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 32d7752 to d3c5970 Compare February 3, 2025 13:47
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from d3c5970 to 386c250 Compare February 3, 2025 14:41
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 386c250 to 3a546a3 Compare February 3, 2025 14:41
@LukasPukenis LukasPukenis force-pushed the LLT-5975/pyro_timeouts branch from 3a546a3 to 105cdb4 Compare February 3, 2025 15:45
We observe that sometimes pyro remote calls are with
time gaps in between which should not happen. We guess
it might be due to network issues. However the packets
still arrive and might be due to TCP retransmission.

Pyro has a timeout property which is not exposed publicly.
This property is set and looped over if timeout error is
returned to fix the issue.

Signed-off-by: Lukas Pukenis <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant