k3s to facilitate MVP deployment

[daachi]

Objective

Use k3s to get an easy, local, light weight Kubernetes deployment that we can then test our workflow tools on, before deploying to our actual cluster.

Definition of Done

• minimal k3s cluster (server and at least one agent) up and running, ready to proceed to Deploy Prefect Server on K3s #32

Artifacts

• Deployment configuration used to create the cluster is in source control
• Notes on the design and implementation

[daachi]

Needs docs for Linux and OSX.

[daachi]

I was finally able to get a KVM created on which to run k3s, but otherwise have been unable to focus on making progress on this effort.

[amcnicho]

Thanks to @daachi for getting things configured so I could start using my workstation as a sandbox for on-prem experiments. I have found k3d a useful tool for starting to interact with a single-host k3s MVP deployment, at least for testing purposes.

I had to bump kubectl and helm from older versions I had (for compatibility with the default k3s cluster versions) but other than that the process to get started was fast and straightforward. Here are the software versions I've got running for now:

kubectl version

Client Version: v1.32.1
Kustomize Version: v5.5.0
Server Version: v1.31.4+k3s1

k3d version

k3d version v5.8.1
k3s version v1.31.4-k3s1 (default)

docker version

Client: Docker Engine - Community
 Version:           26.1.3
 API version:       1.45
Server: Docker Engine - Community
 Engine:
  Version:          26.1.3
  API version:      1.45 (minimum version 1.24)
 containerd:
  Version:          1.6.32
 runc:
  Version:          1.1.12
 docker-init:
  Version:          0.19.0

helm version

version.BuildInfo{Version:"v3.17.0", GitCommit:"301108edc7ac2a8ba79e4ebf5701b0b6ce6a31e4", GitTreeState:"clean", GoVersion:"go1.23.4"}

[amcnicho]

Created https://github.com/casangi/RADPS/wiki/Kubernetes to store "notes on the design and implementation", mirroring the wiki pages set up for Prefect tutorial and workflow design notes.

[amcnicho]

The same basic deployment of k3s (using default k3d) works on mac 14.7 (Apple M3 Pro) running Docker Desktop 27.4.0 and recent versions of k3d/kubectl/helm installed via macports.

Testing to make sure it works:
k3d cluster create --agents 2 --agents-memory=4GB --image=rancher/k3s:v1.31.4-k3s1

Confirmed that it's functional after installing a helm chart:
kubectl get nodes && kubectl get svc

NAME                       STATUS   ROLES                  AGE   VERSION
k3d-k3s-default-agent-0    Ready    <none>                 19m   v1.31.4+k3s1
k3d-k3s-default-agent-1    Ready    <none>                 19m   v1.31.4+k3s1
k3d-k3s-default-server-0   Ready    control-plane,master   20m   v1.31.4+k3s1
NAME                           TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
kubernetes                     ClusterIP   ##.##.##.#     <none>        443/TCP    20m
prefect-server                 ClusterIP   ##.##.##.###   <none>        4200/TCP   8m55s
prefect-server-postgresql      ClusterIP   ##.##.##.#     <none>        5432/TCP   8m55s
prefect-server-postgresql-hl   ClusterIP   None           <none>        5432/TCP   8m55s

This shows that configurable lightweight Kubernetes deployments can be quickly spun up for testing on developer machines.