Support external network connectivity for the pod via pod VM network #2273
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds support for external network connectivity for the pod via the pod VM network, instead of the worker node. This is useful for cloud-bursting cases, where you want the pod to directly access the external network without routing the traffic via the worker node. For example, if you are running a local K8s cluster on your laptop and want to spin up a large instance for AI experimentation in the cloud. The prerequisite is for the pod VM to have an additional interface with an IP. This interface will be moved to the pod network namespace and default routes adjusted so that pod network traverses via worker node, and any other traffic uses the secondary interface. The code extends the primary interface detection (ie, an interface having a default route attached) and adds support for secondary interface detection. Signed-off-by: Pradipta Banerjee <[email protected]>
Add basic support for external pod connectivity via pod VM network. This requires pod VM instance to have two NICs. When using public IP with multi-NIC instance, we need to use elastic IP and attach it to the specific network interface. We can't use AssociatePublicIPAddress as it's meant to be used only with single NIC instance. So what we do is create a new network interface, attach it to the instance and associate an Elastic IP with this network interface Signed-off-by: Pradipta Banerjee <[email protected]>
Bail out early if podNetworkConfig is nil. This is important as we set the ExternalNetViaPodVM via podNetworkConfig. Also fix the test case. Signed-off-by: Pradipta Banerjee <[email protected]>
This is useful for setting explicit routes in the pod namespace for the vxlan device Signed-off-by: Pradipta Banerjee <[email protected]>
yoheiueda
reviewed
Feb 13, 2025
Comment on lines
+200
to
+207
| route := &tunneler.Route{ | ||
| Dst: prefix, | ||
| // The gateway address is 0.0.0.0 | ||
| GW: netip.Addr{}, | ||
| Dev: podInterface, | ||
| Scope: netops.RouteScope(netlink.SCOPE_LINK), | ||
| Protocol: netops.RouteProtocol(unix.RTPROT_KERNEL), | ||
| } |
There was a problem hiding this comment.
GW should be the actual gateway address that is specified in the default route.
The findPrimaryInterface function that is called at Line 129 internally identifies the default route. I think we can modify the function to return the gateway address of the default route.
podInterface, gatewayAddr, err := findPrimaryInterface(podNS)
route := &tunneler.Route{
Dst: prefix,
GW: gatewayAddr,
Dev: podInterface,
}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.