Missing packages from $GOPATH while trying to build deb for PureOS

[jeremiah]

Hello! My name is Jeremiah and I'm the Director of PureOS. We're packaging opensnitch for PureOS and have run into some small issues described below.

Issue: missing dependencies prevents build

Reproduce: run dpkg-buildpackage

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build904233071=/tmp/go-build -gno-record-gcc-switches"
        cd _build && go install -trimpath -v -p 8 github.com/evilsocket/opensnitch/daemon github.com/evilsocket/opensnitch/daemon/conman github.com/evilsocket/opensnitch/daemon/core github.com/evilsocket/opensnitch/daemon/dns github.com/evilsocket/opensnitch/daemon/firewall github.com/evilsocket/opensnitch/daemon/log github.com/evilsocket/opensnitch/daemon/netfilter github.com/evilsocket/opensnitch/daemon/netlink github.com/evilsocket/opensnitch/daemon/netstat github.com/evilsocket/opensnitch/daemon/procmon github.com/evilsocket/opensnitch/daemon/procmon/audit github.com/evilsocket/opensnitch/daemon/procmon/ebpf github.com/evilsocket/opensnitch/daemon/procmon/monitor github.com/evilsocket/opensnitch/daemon/rule github.com/evilsocket/opensnitch/daemon/statistics github.com/evilsocket/opensnitch/daemon/ui
src/github.com/evilsocket/opensnitch/daemon/procmon/watcher.go:8:2: cannot find package "github.com/evilsocket/ftrace" in any of:
        /usr/lib/go-1.15/src/github.com/evilsocket/ftrace (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/evilsocket/ftrace (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/conman/connection.go:17:2: cannot find package "github.com/evilsocket/opensnitch/daemon/ui/protocol" in any of:
        /usr/lib/go-1.15/src/github.com/evilsocket/opensnitch/daemon/ui/protocol (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/evilsocket/opensnitch/daemon/ui/protocol (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/firewall/config.go:10:2: cannot find package "github.com/fsnotify/fsnotify" in any of:
        /usr/lib/go-1.15/src/github.com/fsnotify/fsnotify (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/fsnotify/fsnotify (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/netfilter/packet.go:6:2: cannot find package "github.com/google/gopacket" in any of:
        /usr/lib/go-1.15/src/github.com/google/gopacket (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/google/gopacket (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/netfilter/queue.go:21:2: cannot find package "github.com/google/gopacket/layers" in any of:
        /usr/lib/go-1.15/src/github.com/google/gopacket/layers (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/google/gopacket/layers (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/procmon/ebpf/debug.go:12:2: cannot find package "github.com/iovisor/gobpf/elf" in any of:
        /usr/lib/go-1.15/src/github.com/iovisor/gobpf/elf (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/iovisor/gobpf/elf (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/procmon/ebpf/monitor.go:11:2: cannot find package "github.com/vishvananda/netlink" in any of:
        /usr/lib/go-1.15/src/github.com/vishvananda/netlink (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/vishvananda/netlink (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/netlink/socket_linux.go:11:2: cannot find package "github.com/vishvananda/netlink/nl" in any of:
        /usr/lib/go-1.15/src/github.com/vishvananda/netlink/nl (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/github.com/vishvananda/netlink/nl (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/ui/client.go:16:2: cannot find package "golang.org/x/net/context" in any of:
        /usr/lib/go-1.15/src/golang.org/x/net/context (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/golang.org/x/net/context (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/ui/client.go:17:2: cannot find package "google.golang.org/grpc" in any of:
        /usr/lib/go-1.15/src/google.golang.org/grpc (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/google.golang.org/grpc (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/ui/client.go:18:2: cannot find package "google.golang.org/grpc/connectivity" in any of:
        /usr/lib/go-1.15/src/google.golang.org/grpc/connectivity (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/google.golang.org/grpc/connectivity (from $GOPATH)
src/github.com/evilsocket/opensnitch/daemon/ui/client.go:19:2: cannot find package "google.golang.org/grpc/keepalive" in any of:
        /usr/lib/go-1.15/src/google.golang.org/grpc/keepalive (from $GOROOT)
        /media/jeremiah/Samsung_T51/Packaging/opensnitch/_build/src/google.golang.org/grpc/keepalive (from $GOPATH)
dh_auto_build: error: cd _build && go install -trimpath -v -p 8 github.com/evilsocket/opensnitch/daemon github.com/evilsocket/opensnitch/daemon/conman github.com/evilsocket/opensnitch/daemon/core github.com/evilsocket/opensnitch/daemon/dns github.com/evilsocket/opensnitch/daemon/firewall github.com/evilsocket/opensnitch/daemon/log github.com/evilsocket/opensnitch/daemon/netfilter github.com/evilsocket/opensnitch/daemon/netlink github.com/evilsocket/opensnitch/daemon/netstat github.com/evilsocket/opensnitch/daemon/procmon github.com/evilsocket/opensnitch/daemon/procmon/audit github.com/evilsocket/opensnitch/daemon/procmon/ebpf github.com/evilsocket/opensnitch/daemon/procmon/monitor github.com/evilsocket/opensnitch/daemon/rule github.com/evilsocket/opensnitch/daemon/statistics github.com/evilsocket/opensnitch/daemon/ui returned exit code 1
make: *** [debian/rules:13: binary] Error 1
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2

I expected the opensnitch package to build.

Screenshots
If applicable, add screenshots to help explain your problem.

Using PureOS Byzantium

[gustavo-iniguez-goya]

Hi @jeremiah !

As we depend now on iovisor/gobpf golang package the build process has changed a little bit. There're a few steps that you have to do before creating the package:

$ git clone ..
$ cd opensnitch
$ make protocol
$ cd daemon; go mod vendor; cd ..
$ dpkg-buildpackage

Don't forget that you need some dependencies before packaging it https://github.com/evilsocket/opensnitch/wiki/Compilation

I don't know if this will affect you, but until now I've been building the packages on Debian Sid. However since I don't know what version of golang 1.15/1.16/1.17 , the binaries now depend on GLIBC_2.32 pthread_sigmask. This forces the libc6 version to be 2.32, which is not available in many systems.

Using golang 1.15.9-6 from Debian Bullseye generates binaries compatible with libc6 >= 2.14.

[gustavo-iniguez-goya]

$ wget https://github.com/evilsocket/opensnitch/archive/refs/tags/v1.4.0.tar.gz
$ tar zxf v1.4.0.tar.gz
$ cp /tmp/opensnitch-arm64.o opensnitch-1.4.0/ebpf-prog/opensnitch.o
$ cd opensnitch-1.4.0
$ make protocol
$ cd daemon/
$ go mod vendor
$ cd ../..
$ tar zcf opensnitch_1.4.0.orig.tar.gz opensnitch-1.4.0/
$ cd opensnitch-1.4.0/
$ dpkg-buildpackage

Note that the opensnitch.o is precompiled, I've attached the modules compiled for 4 architecures with these sums:

6c1db0ca14c2f7548b9378a855c8362658fa35dc  opensnitch-arm64.o
5ece05a7f4fad65d3261b7b8c753974e3b569657  opensnitch-arm.o
4f440848aa043632ae5ad91efca34573bf8667ac  opensnitch-i386.o
5c585469bd305b79f7adbb18741f1fed9520901d  opensnitch-x86_64.o

If you want to compile them see here to know how:
https://github.com/evilsocket/opensnitch/tree/master/ebpf_prog
opensnitch-arm64.o.gz
opensnitch-armhf.o.gz
opensnitch-i386.o.gz
opensnitch-x86-64.o.gz

[jeremiah]

1. Thank you for the prompt, detailed answer @gustavo-iniguez-goya Let me digest what you've written to try to replicate package building. :-)

[jeremiah]

$ wget https://github.com/evilsocket/opensnitch/archive/refs/tags/v1.4.0.tar.gz
$ tar zxf v1.4.0.tar.gz
$ cp /tmp/opensnitch-arm64.o opensnitch-1.4.0/ebpf-prog/opensnitch.o

In this step above ^^ you're referring to the .o files you list below, right? Is there a way to compile the .o files on my machine? This is one of the requirements for having packages in our repos - they need to build locally as well as reproducibly via reprotest.

$ cd opensnitch-1.4.0
$ make protocol

Is it possible to include this stage in the debian/rules file?

$ cd daemon/
$ go mod vendor
$ cd ../..
$ tar zcf opensnitch_1.4.0.orig.tar.gz opensnitch-1.4.0/
$ cd opensnitch-1.4.0/
$ dpkg-buildpackage

Note that the opensnitch.o is precompiled, I've attached the modules compiled for 4 architecures with these sums:

6c1db0ca14c2f7548b9378a855c8362658fa35dc  opensnitch-arm64.o
5ece05a7f4fad65d3261b7b8c753974e3b569657  opensnitch-arm.o
4f440848aa043632ae5ad91efca34573bf8667ac  opensnitch-i386.o
5c585469bd305b79f7adbb18741f1fed9520901d  opensnitch-x86_64.o

If you want to compile them see here to know how:

https://github.com/evilsocket/opensnitch/tree/master/ebpf_prog

opensnitch-arm64.o.gz

opensnitch-armhf.o.gz

opensnitch-i386.o.gz

opensnitch-x86-64.o.gz

Thanks, this is useful.

[gustavo-iniguez-goya]

Is it possible to include this stage in the debian/rules file?

added!

[gustavo-iniguez-goya]

In this step above ^^ you're referring to the .o files you list below, right? Is there a way to compile the .o files on my machine? This is one of the requirements for having packages in our repos - they need to build locally as well as reproducibly via reprotest.

Oops, I think I didn't answer to this: yes, here's how: https://github.com/evilsocket/opensnitch/tree/master/ebpf_prog

[gustavo-iniguez-goya]

@jeremiah let me know if I can help you with anything else. Also if you finally package it for PureOS drop a comment here if you don't mind, I'd love to know about it! and help out with any problem that may appear.

[jeremiah]

Hi @gustavo-iniguez-goya! Thanks very much for your help. Purism is very interested in having OpenSnitch in PureOS and I'll continue to work on this, but right now we're blocked on the libc issue as well as not having the Go libraries already in Debian. I've communicated this to folks internally, who ask me regularly about OpenSnitch. FWIW Purism will blog about OpenSnitch and talk about how great it is if we can package it. I'll come back to you with my progress next week, still a bit overloaded here.