Support UDP for LoadBalancer

[Sapd]

Description

Update primarily loadbalancer.go to support UDP

• Removed check preventing creation of UDP LoadBalancer
• Modified updateLoadBalancer to make port/protocol combination is used as key instead of only port
• Check Port/Protocol combination during NLB deletion
• Added healthcheck annotation to make possible to provide a custom healthcheck port
• Added example to readme

Fixes #79

Checklist

(For exoscale contributors)

• Changelog updated (under Unreleased block)
• Testing

Testing

Added a test variant for NLB creation with UDP

❯ gmake test
go.mk/version.mk:13: warning: overriding recipe for target 'get-version-tag'
/Users/denis/Documents/GitHub/exoscale-cloud-controller-manager/go.mk/version.mk:13: warning: ignoring old recipe for target 'get-version-tag'
go.mk/version.mk:13: warning: overriding recipe for target 'get-version-tag'
/Users/denis/Documents/GitHub/exoscale-cloud-controller-manager/go.mk/version.mk:13: warning: ignoring old recipe for target 'get-version-tag'
'/opt/homebrew/bin/go' test \
  -race \
  -timeout 15s \
   \
  github.com/exoscale/exoscale-cloud-controller-manager/exoscale
ok  	github.com/exoscale/exoscale-cloud-controller-manager/exoscale	5.832s

Manual tests done:

• Added and checked UDP services
• Modified service ports
  • Modifying 8080/tcp, 8080/udp to 8081/tcp, 8080/udp works out of the box
  • Modifying back to the same port/different protocol combination needs kubectl apply --server-side because of a long lasting kubectl bug:Umbrella Issue: (Known Issue) Service Duplicate Port Numbers + Patch Merge kubernetes/kubernetes#105610
  • Talked successfully to UDP service via netcat, both with externalTrafficPolicy Cluster (and provided healthcheckport in annotation) and with Local (without the sidecar container and annotation)
• Deletion of NLB
• Classical NLB setup with nginx ingress

[sauterp]

excellent work, thank you!
Since the change isn't trivial, can I ask you to run the acceptance tests https://github.com/exoscale/exoscale-cloud-controller-manager/tree/master/test
they are a bit more involved, so feel free to reach out if you need help.

[Sapd]

@sauterp
Added UDP test (tests wether it can connect to the NLB via UDP after the manifest was applied similar to the hello ingress external test)

Test output:

Test output

❯ pytest -v

================================================================================================================= test session starts ==================================================================================================================
platform darwin -- Python 3.11.10, pytest-8.3.4, pluggy-1.5.0 -- /opt/homebrew/opt/[email protected]/bin/python3.11
cachedir: .pytest_cache
rootdir: ---
plugins: anyio-3.6.2
collected 72 items

test/tests_0_environment/test_001_environment.py::test_type PASSED                                                                                                                                                                               [  1%]
test/tests_0_environment/test_001_environment.py::test_exoscale_credentials PASSED                                                                                                                                                               [  2%]
test/tests_0_environment/test_001_environment.py::test_executable_terraform PASSED                                                                                                                                                               [  4%]
test/tests_0_environment/test_001_environment.py::test_executable_kubectl PASSED                                                                                                                                                                 [  5%]
test/tests_0_environment/test_001_environment.py::test_executable_exocli PASSED                                                                                                                                                                  [  6%]
test/tests_1_control_plane/test_101_control_plane.py::test_control_plane_outputs PASSED                                                                                                                                                          [  8%]
test/tests_1_control_plane/test_102_control_plane_files.py::test_kubernetes_cni PASSED                                                                                                                                                           [  9%]
test/tests_1_control_plane/test_102_control_plane_files.py::test_kubeconfig_admin PASSED                                                                                                                                                         [ 11%]
test/tests_1_control_plane/test_102_control_plane_files.py::test_kubeconfig_ccm PASSED                                                                                                                                                           [ 12%]
test/tests_1_control_plane/test_102_control_plane_files.py::test_ccm_rbac PASSED                                                                                                                                                                 [ 13%]
test/tests_1_control_plane/test_102_control_plane_files.py::test_ccm_cloud_config PASSED                                                                                                                                                         [ 15%]
test/tests_1_control_plane/test_102_control_plane_files.py::test_ccm_main PASSED                                                                                                                                                                 [ 16%]
test/tests_1_control_plane/test_103_control_plane_start.py::test_k8s_version PASSED                                                                                                                                                              [ 18%]
test/tests_1_control_plane/test_103_control_plane_start.py::test_cni_started PASSED                                                                                                                                                              [ 19%]
test/tests_2_nodes/test_201_nodes.py::test_nodes_outputs PASSED                                                                                                                                                                                  [ 20%]
test/tests_2_nodes/test_202_nodes_files.py::test_kubeconfig_external_node SKIPPED (This test may only be performed for 'kubeadm' type (<-> external node kubeconfig/username))                                                                   [ 22%]
test/tests_2_nodes/test_202_nodes_files.py::test_k8s_manifests PASSED                                                                                                                                                                            [ 23%]
test/tests_3_ccm/test_300_init.py::test_cni_started PASSED                                                                                                                                                                                       [ 25%]
test/tests_3_ccm/test_301_k8s_state.py::test_k8s_nodes PASSED                                                                                                                                                                                    [ 26%]
test/tests_3_ccm/test_301_k8s_state.py::test_k8s_node_csrs PASSED                                                                                                                                                                                [ 27%]
test/tests_3_ccm/test_302_ccm_start.py::test_ccm_rbac PASSED                                                                                                                                                                                     [ 29%]
test/tests_3_ccm/test_302_ccm_start.py::test_ccm_start PASSED                                                                                                                                                                                    [ 30%]
test/tests_3_ccm/test_302_ccm_start.py::test_ccm_api_credentials_valid PASSED                                                                                                                                                                    [ 31%]
test/tests_3_ccm/test_302_ccm_start.py::test_ccm_node_csr_agent PASSED                                                                                                                                                                           [ 33%]
test/tests_3_ccm/test_302_ccm_start.py::test_ccm_started PASSED                                                                                                                                                                                  [ 34%]
test/tests_3_ccm/test_303_ccm_node_csr_validation.py::test_ccm_node_csrs_approved PASSED                                                                                                                                                         [ 36%]
test/tests_3_ccm/test_304_ccm_node_csr_invalid.py::test_ccm_node_csrs_invalid[k8s_csr_invalid_ip_address0] SKIPPED (This test may only be performed for 'kubeadm' type (<-> external node kubeconfig/username))                                  [ 37%]
test/tests_3_ccm/test_304_ccm_node_csr_invalid.py::test_ccm_node_csrs_invalid[k8s_csr_invalid_ip_address1] SKIPPED (This test may only be performed for 'kubeadm' type (<-> external node kubeconfig/username))                                  [ 38%]
test/tests_3_ccm/test_305_ccm_nodes.py::test_ccm_nodes_init XFAIL (TODO/BUG[58670]: CCM: provider ID cannot be empty)                                                                                                                            [ 40%]
test/tests_3_ccm/test_306_ccm_api_credentials_update.py::test_ccm_api_credentials_invalid PASSED                                                                                                                                                 [ 41%]
test/tests_3_ccm/test_306_ccm_api_credentials_update.py::test_ccm_api_credentials_valid PASSED                                                                                                                                                   [ 43%]
test/tests_3_ccm/test_307_k8s_nodes.py::test_k8s_nodes_spec XFAIL (TODO/BUG[58670]: CCM: provider ID cannot be empty)                                                                                                                            [ 44%]
test/tests_3_ccm/test_307_k8s_nodes.py::test_k8s_nodes_labels PASSED                                                                                                                                                                             [ 45%]
test/tests_3_ccm/test_307_k8s_nodes.py::test_k8s_nodes_addresses XFAIL (TODO: Investigate why this fails on SKS)                                                                                                                                 [ 47%]
test/tests_4_nlb/test_400_init.py::test_cni_started PASSED                                                                                                                                                                                       [ 48%]
test/tests_4_nlb/test_400_init.py::test_ccm_started PASSED                                                                                                                                                                                       [ 50%]
test/tests_4_nlb/test_401_nlb_hello_external.py::test_k8s_hello_external PASSED                                                                                                                                                                  [ 51%]
test/tests_4_nlb/test_401_nlb_hello_external.py::test_ccm_hello_external_services PASSED                                                                                                                                                         [ 52%]
test/tests_4_nlb/test_401_nlb_hello_external.py::test_cli_hello_external PASSED                                                                                                                                                                  [ 54%]
test/tests_4_nlb/test_401_nlb_hello_external.py::test_cli_hello_external_services PASSED                                                                                                                                                         [ 55%]
test/tests_4_nlb/test_401_nlb_hello_external.py::test_http_hello_external PASSED                                                                                                                                                                 [ 56%]
test/tests_4_nlb/test_402_nlb_ingress_nginx.py::test_k8s_ingress_nginx PASSED                                                                                                                                                                    [ 58%]
test/tests_4_nlb/test_402_nlb_ingress_nginx.py::test_ccm_ingress_nginx PASSED                                                                                                                                                                    [ 59%]
test/tests_4_nlb/test_402_nlb_ingress_nginx.py::test_cli_ingress_nginx PASSED                                                                                                                                                                    [ 61%]
test/tests_4_nlb/test_402_nlb_ingress_nginx.py::test_ccm_ingress_nginx_services PASSED                                                                                                                                                           [ 62%]
test/tests_4_nlb/test_402_nlb_ingress_nginx.py::test_ccm_ingress_nginx_loadbalancer PASSED                                                                                                                                                       [ 63%]
test/tests_4_nlb/test_402_nlb_ingress_nginx.py::test_k8s_ingress_nginx_loadbalancer PASSED                                                                                                                                                       [ 65%]
test/tests_4_nlb/test_402_nlb_ingress_nginx.py::test_cli_ingress_nginx_services PASSED                                                                                                                                                           [ 66%]
test/tests_4_nlb/test_403_nlb_hello_ingress.py::test_k8s_hello_ingress PASSED                                                                                                                                                                    [ 68%]
test/tests_4_nlb/test_403_nlb_hello_ingress.py::test_http_hello_ingress PASSED                                                                                                                                                                   [ 69%]
test/tests_4_nlb/test_404_nlb_echo_udp.py::test_k8s_udp_echo_external PASSED                                                                                                                                                                     [ 70%]
test/tests_4_nlb/test_404_nlb_echo_udp.py::test_udp_echo_service_creation PASSED                                                                                                                                                                 [ 72%]
test/tests_4_nlb/test_404_nlb_echo_udp.py::test_udp_echo_external_response PASSED                                                                                                                                                                [ 73%]
test/tests_5_nodes_pool_resize/test_500_init.py::test_cni_started PASSED                                                                                                                                                                         [ 75%]
test/tests_5_nodes_pool_resize/test_500_init.py::test_ccm_started PASSED                                                                                                                                                                         [ 76%]
test/tests_5_nodes_pool_resize/test_501_k8s_state.py::test_state_change[2] PASSED                                                                                                                                                                [ 77%]
test/tests_5_nodes_pool_resize/test_501_k8s_state.py::test_k8s_nodes[2] PASSED                                                                                                                                                                   [ 79%]
test/tests_5_nodes_pool_resize/test_501_k8s_state.py::test_k8s_node_csrs[2] PASSED                                                                                                                                                               [ 80%]
test/tests_5_nodes_pool_resize/test_502_ccm_nodes_deletion.py::test_ccm_node_deletion[2] SKIPPED (No existing node is expected to be deleted)                                                                                                    [ 81%]
test/tests_5_nodes_pool_resize/test_503_ccm_nodes_csr_validation.py::test_ccm_node_csrs_approved[2] PASSED                                                                                                                                       [ 83%]
test/tests_5_nodes_pool_resize/test_504_k8s_nodes.py::test_k8s_nodes[2] PASSED                                                                                                                                                                   [ 84%]
test/tests_5_nodes_pool_resize/test_505_nlb_hello.py::test_http_hello_external[2] PASSED                                                                                                                                                         [ 86%]
test/tests_5_nodes_pool_resize/test_505_nlb_hello.py::test_http_hello_ingress[2] PASSED                                                                                                                                                          [ 87%]
test/tests_5_nodes_pool_resize/test_501_k8s_state.py::test_state_change[1] PASSED                                                                                                                                                                [ 88%]
test/tests_5_nodes_pool_resize/test_501_k8s_state.py::test_k8s_nodes[1] PASSED                                                                                                                                                                   [ 90%]
test/tests_5_nodes_pool_resize/test_501_k8s_state.py::test_k8s_node_csrs[1] PASSED                                                                                                                                                               [ 91%]
test/tests_5_nodes_pool_resize/test_502_ccm_nodes_deletion.py::test_ccm_node_deletion[1] XPASS (TODO/BUG[58670]: CCM: provider ID cannot be empty)                                                                                               [ 93%]
test/tests_5_nodes_pool_resize/test_503_ccm_nodes_csr_validation.py::test_ccm_node_csrs_approved[1] SKIPPED (No new node is expected to be approved)                                                                                             [ 94%]
test/tests_5_nodes_pool_resize/test_504_k8s_nodes.py::test_k8s_nodes[1] PASSED                                                                                                                                                                   [ 95%]
test/tests_5_nodes_pool_resize/test_505_nlb_hello.py::test_http_hello_external[1] PASSED                                                                                                                                                         [ 97%]
test/tests_5_nodes_pool_resize/test_505_nlb_hello.py::test_http_hello_ingress[1] PASSED                                                                                                                                                          [ 98%]
test/tests_9_the_end/test_999_finalize.py::test_ccm_tail PASSED                                                                                                                                                                                  [100%]

I also added into the docs that export EXOSCALE_ZONE="ch-gva-2" is required for the tests. The zone is hardcoded in terraform and if someone has by accident set the variable to de-fra-1 NLB tests will fail (also on master without my changes) because everything will be deployed in ch-gva-2 in any case - but the CCM tries for some reason to find load balancers in frankfurt then.

[sauterp]

awesome stuff!