Not able to run in Android - Debian (Linux-Deploy) arm using **fwmark'd** packages.

[hugoj-goncalves]

Describe the problem

I can't run netbird in a arm device (Android using linux-deploy) with custom routing enabled.

To Reproduce

Steps to reproduce the behavior:

1. Install netbird using the install script provided for linux.
2. Try to run using netbird up
3. Logs below.

Expected behavior

It should work ;s

Are you using NetBird Cloud?

I'm using netbird cloud.

NetBird version

0.36.3

OS

root@localhost:/home/android# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Logs

root@localhost:/home/android# **netbird up -F -l debug**
2025-01-20T08:05:58-03:00 DEBG client/internal/login.go:94: connecting to the Management service https://api.netbird.io:443
2025-01-20T08:05:58-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T08:05:58-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T08:05:58-03:00 ERRO util/grpc/dialer.go:43: Failed to dial: d.Dialer.DialContext: dial tcp 35.186.199.111:443: connect: network is unreachable
2025-01-20T08:05:59-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T08:05:59-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T08:05:59-03:00 ERRO util/grpc/dialer.go:43: Failed to dial: d.Dialer.DialContext: dial tcp 35.186.199.111:443: connect: network is unreachable
2025-01-20T08:06:01-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T08:06:01-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T08:06:01-03:00 ERRO util/grpc/dialer.go:43: Failed to dial: d.Dialer.DialContext: dial tcp 35.186.199.111:443: connect: network is unreachable
^C


root@localhost:/home/android# **NB_USE_LEGACY_ROUTING=true NB_DISABLE_CUSTOM_ROUTING=true netbird up -F -l debug**
2025-01-20T08:06:55-03:00 DEBG client/internal/login.go:94: connecting to the Management service https://api.netbird.io:443
2025-01-20T08:06:55-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T08:06:55-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T08:06:55-03:00 INFO util/net/net_linux.go:61: Custom routing is disabled, skipping SO_MARK
2025-01-20T08:06:55-03:00 DEBG client/internal/login.go:64: connected to the Management service https://api.netbird.io:443
2025-01-20T08:06:58-03:00 DEBG client/internal/login.go:94: connecting to the Management service https://api.netbird.io:443
2025-01-20T08:06:58-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T08:06:58-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T08:06:58-03:00 INFO util/net/net_linux.go:61: Custom routing is disabled, skipping SO_MARK
2025-01-20T08:06:58-03:00 DEBG client/internal/login.go:64: connected to the Management service https://api.netbird.io:443
2025-01-20T08:07:00-03:00 INFO client/internal/connect.go:115: starting NetBird client version 0.36.3 on linux/arm
2025-01-20T08:07:00-03:00 DEBG client/internal/connect.go:170: connecting to the Management service api.netbird.io:443
2025-01-20T08:07:00-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T08:07:00-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T08:07:00-03:00 INFO util/net/net_linux.go:61: Custom routing is disabled, skipping SO_MARK
2025-01-20T08:07:00-03:00 DEBG client/internal/connect.go:178: connected to the Management service api.netbird.io:443
2025-01-20T08:07:02-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T08:07:02-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp signal.netbird.io:443
2025-01-20T08:07:02-03:00 INFO util/net/net_linux.go:61: Custom routing is disabled, skipping SO_MARK
2025-01-20T08:07:03-03:00 DEBG signal/client/grpc.go:83: connected to Signal Service: signal.netbird.io:443
2025-01-20T08:07:03-03:00 INFO client/internal/connect.go:247: connecting to the Relay service(s): rels://relay.netbird.io:443
2025-01-20T08:07:03-03:00 DEBG relay/client/manager.go:109: starting relay client manager with [rels://relay.netbird.io:443] relay servers
2025-01-20T08:07:03-03:00 DEBG relay/client/picker.go:45: pick server from list: [rels://relay.netbird.io:443]
^C


root@localhost:/home/android# **NB_SKIP_SOCKET_MARK=true netbird up -F -l debug**
2025-01-20T09:09:08-03:00 DEBG client/internal/login.go:94: connecting to the Management service https://api.netbird.io:443
2025-01-20T09:09:08-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T09:09:08-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T09:09:08-03:00 INFO util/net/env.go:25: NB_SKIP_SOCKET_MARK is set to true, skipping SO_MARK
2025-01-20T09:09:08-03:00 DEBG client/internal/login.go:64: connected to the Management service https://api.netbird.io:443
2025-01-20T09:09:09-03:00 DEBG client/internal/login.go:94: connecting to the Management service https://api.netbird.io:443
2025-01-20T09:09:09-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T09:09:09-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T09:09:09-03:00 INFO util/net/env.go:25: NB_SKIP_SOCKET_MARK is set to true, skipping SO_MARK
2025-01-20T09:09:09-03:00 DEBG client/internal/login.go:64: connected to the Management service https://api.netbird.io:443
2025-01-20T09:09:10-03:00 INFO client/internal/connect.go:115: starting NetBird client version 0.36.3 on linux/arm
2025-01-20T09:09:10-03:00 DEBG client/internal/connect.go:170: connecting to the Management service api.netbird.io:443
2025-01-20T09:09:10-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T09:09:10-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp api.netbird.io:443
2025-01-20T09:09:10-03:00 INFO util/net/env.go:25: NB_SKIP_SOCKET_MARK is set to true, skipping SO_MARK
2025-01-20T09:09:10-03:00 DEBG client/internal/connect.go:178: connected to the Management service api.netbird.io:443
2025-01-20T09:09:11-03:00 DEBG util/grpc/dialer.go:40: Using nbnet.NewDialer()
2025-01-20T09:09:11-03:00 DEBG util/net/dialer_dial.go:52: Dialing tcp signal.netbird.io:443
2025-01-20T09:09:11-03:00 INFO util/net/env.go:25: NB_SKIP_SOCKET_MARK is set to true, skipping SO_MARK
2025-01-20T09:09:11-03:00 DEBG signal/client/grpc.go:83: connected to Signal Service: signal.netbird.io:443
2025-01-20T09:09:11-03:00 INFO client/internal/connect.go:247: connecting to the Relay service(s): rels://relay.netbird.io:443
2025-01-20T09:09:11-03:00 DEBG relay/client/manager.go:109: starting relay client manager with [rels://relay.netbird.io:443] relay servers
2025-01-20T09:09:11-03:00 DEBG relay/client/picker.go:45: pick server from list: [rels://relay.netbird.io:443]
2025-01-20T09:09:11-03:00 INFO relay/client/picker.go:72: try to connecting to relay server: rels://relay.netbird.io:443

[lixmal]

I assume something drops fwmark'd packets, nothing we can do about.
NB_SKIP_SOCKET_MARK=true should be sufficient, though.

[hugoj-goncalves]

it does start with NB_SKIP_SOCKET_MARK=true

would it be usable this way? I have never heard about it before, but at a quick glance it should be responsible for adjusting the traffic to the correct peers I guess.

Edit: looks like it won't work as a routing peer without it. ;/