How is the DNS resolution on the development machine exactly working?

[schwiftysmith]

Hey everyone,

I am facing some network issues on WSL within a VPN which keep blocking me using mirrord there. Maybe someone can help me.

Setup information

I installed a completely fresh WSL2 distribution (Ubuntu) to use this as new development environment. My cluster is remotely available on another PC. After adding the K8s config, I am able to run the kubectl commands as usual.

(Note: the Windows PC with the WSL installed is part of a VPN which required manual adjustsments in the WSL /etc/resolve.conf file to get internet access)

The used mirrord config looks like this:

{
  "accept_invalid_certificates": true,
  "target": {
    "path": {
      "pod": "my-app-0"
    },
    "namespace": "my-namespace"
  },
  "feature": {
    "network": {
      "incoming": "steal"
    }
  }
}

Every connection information of my application is defined by the running/working pod's (my-app-0) environment.

Problem statement

Once I start debugging my application which requires database access in a separate pod, the host cannot be resolved:

{"level":"debug","time":"2024-01-16T13:03:30+01:00","message":"DB Connecting to: postgres://postgres:5432"}
{"level":"error","error":"failed to connect to `host=postgres user=XXX database=XXX`: hostname resolving error (lookup postgres on YYY.YYY.YYY.YYY:53: no such host)","time":"2024-01-16T13:04:00+01:00","message":"DB Connection failed"}

The mirrord job/pod is created as expected within the cluster. Also, the environment variables including the connection information and credentials are successfully passed to my application debugger. However, the hostname postgres cannot be resolved. The IP YYY.YYY.YYY.YYY (which you can see in the logs) is the one of the first nameserver of the /etc/resolve.conf file which I had to configure in order to get internet access in the VPN. Obviously, that globally used name server cannot resolve the hostname.

(Side note: In an Ubuntu VM on the same machine, everything works properly out of the box. Obviously different network situation due to the isolated VM.)

Question

I tried plenty of scenarios to get this work. However, my application cannot establish a proper connection to another pod.

I am not sure, if this is an issue of mirrord or missing network configuration in WSL. Therefore, can you provide some detailed information how mirrord takes care that the application traffic is routed to the cluster's pods and how the lookup exactly is working.

Also, are there logs available of the mirrord locally (not sure if the agent pod's logs make sense when the routing is not working)?

Thanks a lot!

[aviramha]

Hi! I'm sorry about the issues you're having.
There shouldn't be different between Linux VM and WSL - mirrord works on the libc level (read more about it here.)
A possible issue could be that you're running a Go application or any application that is compiled statically, resulting in mirrord failing to load into the executable.
In case you are really using Go - in most cases if you install gcc sudo apt install gcc it does compile a dynamic binary, which should resolve that - or you can force it to make it dynamic - by using the build flags
go build -ldflags='-linkmode external'

Given that it works on one Linux machine and doesn't on the other, I'm pretty sure it's the case ^^

Read more about it here https://mirrord.dev/docs/overview/faq/#ive-run-my-program-with-mirrord-but-it-seems-to-have-no-effect

To get mirrord to output logs, you can use RUST_LOG=mirrord=trace as environment (RUST_LOG being key and mirrord=trace as the value)
It'd also help us understand if the issue is mirrord not being loaded correctly.

In any case, feel free to book support time with us, or join the Discord to help you troubleshoot :)

[schwiftysmith]

Thank you very much for the quick response @aviramha . Totally forgot about this on the fresh system. Now, it is working perfectly fine, thanks 👍