NAT holepunching code?

[nettyso]

Hi @mcginty

The innernet blog post mentioned NAT holepunching:

We had some simple goals:

• Conveniences a typical WireGuard user wants: peer names, auto-updating peer lists, groups based on IP blocks, and automatic NAT holepunching.

I had just read Tailscale's great blog post about NAT holepunching mechanisms and learned of the many approaches to the problem. I was curious how innernet did it but only found this in the codebase:

innernet/server/src/api/mod.rs

Lines 8 to 14 in c01c2be

	/// Inject the collected endpoints from the WG interface into a list of peers.
	/// This is essentially what adds NAT holepunching functionality.
	pub fn inject_endpoints(session: &Session, peers: &mut Vec<Peer>) {
	for mut peer in peers {
	if peer.contents.endpoint.is_none() {
	if let Some(endpoint) = session.context.endpoints.read().get(&peer.public_key) {
	peer.contents.endpoint = Some(endpoint.to_owned().into());

Is there more implementation than this? I tried searching for "endpoint" in the codebase to find relevant NAT table code or port scanning code - I found persistent_keepalive_interval and spawn_endpoint_refresher but it seems to be only part of innernet-server and not the clients/peers...

I looked up if Wireguard does holepunching only found example code of how it could be done; with a note to never use the code haha.

#109 wants to implement more NAT code but does innernet do any NAT holepunching on its own right now?

Maybe the Tailscale blog hyped it up as a more difficult thing than it is - in Nebula's code it looks to be a simple for-loop that send 1 byte to every peer and then sleeps: https://github.com/slackhq/nebula/blob/785914071104c73515736aafd2b9d91501108b23/hostmap.go#L369

Thanks for clarification!

[mcginty]

hey @heyheyhello! Thanks for opening the issue, this is a great time to write a better explanation of where we're at and what the current limitations are.

Here's what innernet currently does:

1. Peers connect to the server via WireGuard (the server must have its WireGuard port open).
2. The server, via that inject_endpoints function you referenced, gossips the endpoint it sees for each peer to all their associated peers.
3. Peers then set their associated peers' endpoints to what the server told them.
4. Since persistent_keepalive is set for each peer to send a heartbeat packet every 25 seconds, peers behind NATs will be sending UDP holepunching packets (via the keepalive functionality).

We're able to avoid the need to use raw packets, since we are in fact talking to the server over WireGuard, so we don't need to use BPF to spoof the source port like they do in the holepunching example from the WireGuard repository.

Where this breaks currently is with, as that really fantastic Tailscale article details, "Hard" NATs. Innernet currently doesn't have any ICE-like functionality, and even more abstractly doesn't have any ability currently to try a list of IPs and see what connects. WireGuard itself (thankfully) doesn't have any concept of a stateful connection, so we need to implement the simplest reliable way to check if an endpoint one that allows a handshake to succeed.