Support for dynamic dispatch

[dave-tucker]

It's possible that users of bpfman may want to implement their own dynamic dispatch - similar to how we create a dispatcher for XDP and TC programs.

This can be accomplished via either:

a. BPF_MAP_TYPE_PROG_ARRAY and tail calls
b. BPF_PROG_TYPE_EXT

From an API standpoint we'd need to figure out how we'd like to model this.

Prog Array

ISTR that we can update a BPF_MAP_TYPE_PROG_ARRAY from user space using bpf_map_update_elem.
The issue is that adding a program requires a file descriptor... and getting file descriptors from the bpf syscall requires privileges 😢

There are 2 ways we can handle this:

1. Manifest the necessary program pins inside a container's bpffs. This then puts the onus on the userland program to update the jump table, which I think seems reasonable. The API we'd need to design would be fairly easy to model from K8s (outside K8s if you have access to bpffs it doesn't matter).
2. Try and get bpfman to update the map. I'm discouraging this as we had concluded early on that letting bpfman manipulate maps was potentially a bad idea. The API for this would also get complicated - how do we reference other programs? what do we do if a program that's supposed to be in the map isn't loaded yet?

Extensions

I believe we already have an API for adding extensions. If not, it's easy as they are just treated like any other eBPF program. I don't see any concerns in how we'd model this in the K8s API.