diff --git a/kubernetes/README.md b/kubernetes/README.md new file mode 100644 index 0000000..45b0476 --- /dev/null +++ b/kubernetes/README.md @@ -0,0 +1,45 @@ +# Kubernetes + +Kubernetes provides container orchestration for our applications. The automated +management and monitoring provided by using kubernetes removes a lot of the +burden from the application developers. + +We use an all in one implementation of kubernetes called [k3s](https://k3s.io). +For instructions on initial cluster installation [read this +document](./installation) + +## Concepts + +### Applications + +Applications configuration is through the use of kubernetes manifests contained +in the +[https://github.com/metacpan/metacpan-k8s](https://github.com/metacpan/metacpan-k8s) +repository. + +Our manifests use a process named [kustomize](https://kustomize.io) that +provides the ability to patch manifests with environment and cluster specific +details. + +Kubernetes documentation on manifests is extensive [kubernetes +documentation](https://kubernetes.io/docs/tasks/configure-pod-container/) + +### Secrets + +Kubernetes secrets provide a secure way to store application configuration, +passwords, and tokens. The values of secrets are base64 encoded as part of the +manifests, this does not make their manifest form secure for inclusion in a +public repository. + +Raw secret manifests exist alongside the private configuration files. A process +called [kubeseal](https://sealed-secrets.netlify.app/) which encrypts the raw +secrets with the public key for a single cluster. The sealed secret can only be +decrypted by the cluster that it's sealed against. Because of this, sealed +secrets use the mentioned [kustomize](https://kustomize.io) process. + +### Networking + +Most hosting providers that provide single instances do not gaurantee that +public IP addresses are all on a single network. To provide a private +network for inter-cluster communication, a product called +[kilo](https://kilo.squat.ai/) that creates a wireguard based VPN is installed. diff --git a/kubernetes/installation.md b/kubernetes/installation.md new file mode 100644 index 0000000..3ec2cbc --- /dev/null +++ b/kubernetes/installation.md @@ -0,0 +1,60 @@ +# Installation + +## Naming + +Clusters names use a 2 letter abbreviation assigned to the hosting company (bm +-> ByteMark, lw -> LiquidWeb, hc -> Hivelocity) assign one for the new cluster. + +DNS records need to be created for each host within the host. Hostnames are: + +```text +<2 letter host>-mc-<2 digit server number>.metacpan.org +``` + +Create another record that provides round robin DNS for the cluster: + +```text +*.<2 letter host>.metacpan.org +``` + +## k3s + +k3s installation is straight forward, using an installtion script, that runs as +an unprivileged user. + +Each cluster needs a token for that cluster that allows other server +installations to join the cluster. + +1. Generate a token using `head -c48 /dev/urandom | base64 | tr -d "\n"` and store + in a secure location. + +2. Sign into server using ssh and an unprivileged account. + +3. Install k3s using the token generated above with the following command: + + ```bash + curl -sfL https://get.k3s.io \ + | K3S_TOKEN= \ + sh -s - server --cluster-init + ``` + +4. Sign into each additional server, and install k3s using the following + command: + + ```bash + curl -sfL https://get.k3s.io \ + | K3S_TOKEN= \ + sh -s - server --server https://:6443 + ``` + +## Remote Access + +Remote tools like [kubectl](https://kubernetes.io/docs/reference/kubectl/), +[k9s](https://k9scli.io/), and [kubeseal](https://sealed-secrets.netlify.app/) +require authentication with the cluster during use. The authentication uses a +certificate assigned to the cluster during its installation. + +On the first server installed copy the contents of the `sudo cat +/etc/rancher/k3s/k3s.yaml` to your `~/.kube/configs/` directory as `<2 letter +code>-mc`. Also copy this file to the metacpan-credential repository's `k8s/` +directory.