Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intelligent anti-bufferboat in picomux #52

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

nullchinchilla
Copy link
Member

@nullchinchilla nullchinchilla commented Jan 4, 2025

Flow control in picomux is based on a basic per-stream sliding window approach:

  • the send window is initially set to INIT_WINDOW
  • sender decrements window for each frame sent, blocking when it reaches 0
  • receiver can grow the window by sending MORE frames

Generally, we want the window to be as small as possible, while still being big enough to fill the pipe. Oversized windows have two problems with "bufferbloat":

  1. It causes bufferbloat within the stream if the stream is being read from slowly. This is a problem if the stream itself contains concurrent interactions, such as with HTTP/2 or picomux itself.
  2. It worsens bufferbloat in the underlying transport. A window far exceeding the bandwidth-delay product, combined with network buffers also far exceeding the bandwidth-delay product, can cause massive standing queues in the network pipe.

Currently, there are heuristics for avoiding 1. (which is important because of how central picomux-over-picomux is to the whole Geph bridge system), but none for 2 other than attempting to use low-level socket options to limit TCP buffering on Linux. These options kinda work, but network-level TCP middleboxes can easily cause unfixable bufferbloat. Ensuring that every combination of underlying transport is bloat-free is also basically impossible.

This PR instead attempts to very generously estimate the BDP and limit the window to that:

  • bandwidth is estimated by how fast data is passing through the stream
  • the delay is simply assumed to be 500ms

This prevents causing more than 500ms or so of bufferbloat in the underlying transport, and on top of that the way bandwidth is estimated also prevents bufferbloat within the picomux stream by slow readers, so it kills two birds with one stone.

(We don't try to use the measured ping, since that can cause a bloat -> measured ping increases -> more bloat is allowed cycle)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant