x509: cannot validate certificate for <WORKER_NODE_IP> because it doesn't contain any IP SANs

[MugenTwo]

My setup is:

1. Ubuntu server inside HyperV running MicroK8s (the host of the hyperv is the windows server). I followed the standard installation: https://microk8s.io/docs/getting-started
2. Windows server 2022 (which is the host of the hyperv that has the ubuntu server that has microk8s) running worker node installed following: https://microk8s.io/docs/add-a-windows-worker-node-to-microk8s
3. The cluster seems to be connected and I can run windows containers inside the windows worker node and linux containers in the ubuntu server microk8s (control plane + worker node).
4. This is what I see when I do a kubectl get nodes
   
5. However, there are cert issues whenever I do certain operations on pods running in the Windows worker node like kubectl log, kubectl port-forward:
   

Expected Behavior

There should NOT be cert errors.

Current Behavior

5. However, there are cert issues whenever I do certain operations on pods running in the Windows worker node like kubectl log, kubectl port-forward:
   

Possible Solution

I've tried to:

1. Initially I install Kubernetes services version 1.27.1, and I got the cert error.
2. So, I decided to uninstall this version 1.27.1 and then I installed 1.32.0 because my control plane (microk8s) had this version.
3. I tried deleting the cert: C:\var\lib\kubelet\pki and then restarting the kubelet.
4. I also checked how the kubelet was being run in the kubelet,out.log that is under "C:\k" and I printed the kubelet.exe parameters:
   
5. From what I read online the IP SAN might be dependant on the --node-ip, but the --node-ip in kubelet.exe args seems correct in my case.

None of which has fixed the issue

Steps to Reproduce (for bugs)

1. Ubuntu server inside HyperV running MicroK8s (the host of the hyperv is the windows server). I followed the standard installation: https://microk8s.io/docs/getting-started
2. Windows server 2022 (which is the host of the hyperv that has the ubuntu server that has microk8s) running worker node installed following: https://microk8s.io/docs/add-a-windows-worker-node-to-microk8s
3. Run any windows container in a pod.
4. Do a "kubectl log" on the pod.

Context

1. Ubuntu server inside HyperV running MicroK8s (the host of the hyperv is the windows server). I followed the standard installation: https://microk8s.io/docs/getting-started
2. Windows server 2022 (which is the host of the hyperv that has the ubuntu server that has microk8s) running worker node installed following: https://microk8s.io/docs/add-a-windows-worker-node-to-microk8s

Your Environment

• Calico version
  CALICO_VERSION="3.25.0"
• Calico dataplane (iptables, windows etc.)
• Orchestrator version (e.g. kubernetes, mesos, rkt):
• Operating System and version:
  Microk8s on ubuntu server running in HyperV (105 IP)
  Windows worker node (106 IP)
• Link to your project (optional):

[MugenTwo]

I'm linking this issue to the issue I created in MicroK8s: canonical/microk8s#4814
As I think it might have something to do with the integration with MicroK8s

[caseydavenport]

@MugenTwo what makes you believe this has something to do with Calico?

AFAIK, Calico isn't involved in the management of the certificate(s) in question. But my knowledge of the Windows side of things is a bit fuzzy. Perhaps @coutinhop can chime in here as well. But IMO this sounds like a problem with the way Kubernetes is installed on the cluster, and nothing to do with Calico.

[MugenTwo]

Hi @caseydavenport ! The reason is because the script to install and run kubelet and kube-proxy that is recommended in the Microk8s guide is done via calico scripts: https://github.com/projectcalico/calico/tree/master/node/windows-packaging/CalicoWindows/kubernetes

Here is the command from the Microk8s guide (https://microk8s.io/docs/add-a-windows-worker-node-to-microk8s):
c:\k\install-calico-windows.ps1 -ReleaseBaseURL "https://github.com/projectcalico/calico/releases/download/v3.25.0" -ReleaseFile "calico-windows-v3.25.0.zip" -KubeVersion "1.27.1" -DownloadOnly "yes" -ServiceCidr "10.152.183.0/24" -DNSServerIPs "10.152.183.10"