From 64896a4fb9ca7bd340adba5851ff93a101e101f5 Mon Sep 17 00:00:00 2001 From: Laura Nolan Date: Mon, 20 May 2024 14:33:09 +0100 Subject: [PATCH] Add extension to kafka-cron project - port to minikube --- projects/kafka-cron/README.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/projects/kafka-cron/README.md b/projects/kafka-cron/README.md index 84abcfbc1..a26a9d83e 100644 --- a/projects/kafka-cron/README.md +++ b/projects/kafka-cron/README.md @@ -242,6 +242,36 @@ How many containers being down can our system tolerate? What happens to the Kafka system logs and the metrics that our binaries export? Did our alerts fire? If not, consider how they could be improved - remember, the point of them is to tell us when something's wrong! + +### Porting your system from docker-compose to minikube + +In this project, and in several previous projects, you have used `docker-compose` to deploy our code, alongside dependencies (such as Kafka and +Zookeeper here). `docker-compose` is an extremely convenient tool for running a multi-part software stack locally (it also works well for running +integration tests agasint in a Continuous Integration workflow as part of your development process). However, in most deployments, we want to be able +to run our code and its dependencies across more than one host, in order to scale horizontally and to be robust to single-node failures. For this, +`Kubernetes` is a better tool. `Kubernetes`, like `docker-compose`, is a platform for running containerised applications, but where `docker-compose` +is focused on running a set of related containers on a single host, `Kubernetes` is optimized for running services across many hosts. + +Here are some introductions to Kubernetes: + * [What is Kubernetes](https://www.digitalocean.com/community/tutorials/an-introduction-to-kubernetes) + * [Kubernetes Basics](https://kubernetes.io/docs/tutorials/kubernetes-basics/) - explore the linked pages here + +You may have already used `minikube` in one of the previous projects. `Minikube` is a local version of Kubernetes, which we can use to learn +(rather than needing to incur the expense of cloud services such as [EKS](https://aws.amazon.com/eks/) for learning on.) + +Get your local `minikube setup` working bu following [minikube start](https://minikube.sigs.k8s.io/docs/start/). +Follow the steps to install the program and run the `hello-minikube` application. + +Once you have done this, you will need to convert your `docker-compose.yml` files to `kubernetes` deployment files. +There is a tool, [kompose](https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/) which may assist you. + +Test that your system works as it did on `docker-compose`. + +Learn your way around the `kubernetes` command-line tool, `kubectl` (see the [kubectl documentation](https://kubernetes.io/docs/reference/kubectl/)). +* How can you resize your service, i.e. change the number of running instances (pods)? +* How can you remove one instance of your service (a single pod)? +* How can you see the logs for your service? + ### Dealing with long-running jobs and load (challenging) What does our system do if someone submits a very long-running job? Try testing this with the `sleep` command.