Skip to content

Commit

Permalink
Add extension to kafka-cron project - port to minikube
Browse files Browse the repository at this point in the history
  • Loading branch information
lauranooooo committed May 20, 2024
1 parent 6bca88a commit 64896a4
Showing 1 changed file with 30 additions and 0 deletions.
30 changes: 30 additions & 0 deletions projects/kafka-cron/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,36 @@ How many containers being down can our system tolerate?

What happens to the Kafka system logs and the metrics that our binaries export? Did our alerts fire? If not, consider how they could be improved - remember, the point of them is to tell us when something's wrong!


### Porting your system from docker-compose to minikube

In this project, and in several previous projects, you have used `docker-compose` to deploy our code, alongside dependencies (such as Kafka and
Zookeeper here). `docker-compose` is an extremely convenient tool for running a multi-part software stack locally (it also works well for running
integration tests agasint in a Continuous Integration workflow as part of your development process). However, in most deployments, we want to be able
to run our code and its dependencies across more than one host, in order to scale horizontally and to be robust to single-node failures. For this,
`Kubernetes` is a better tool. `Kubernetes`, like `docker-compose`, is a platform for running containerised applications, but where `docker-compose`
is focused on running a set of related containers on a single host, `Kubernetes` is optimized for running services across many hosts.

Here are some introductions to Kubernetes:
* [What is Kubernetes](https://www.digitalocean.com/community/tutorials/an-introduction-to-kubernetes)
* [Kubernetes Basics](https://kubernetes.io/docs/tutorials/kubernetes-basics/) - explore the linked pages here

You may have already used `minikube` in one of the previous projects. `Minikube` is a local version of Kubernetes, which we can use to learn
(rather than needing to incur the expense of cloud services such as [EKS](https://aws.amazon.com/eks/) for learning on.)

Get your local `minikube setup` working bu following [minikube start](https://minikube.sigs.k8s.io/docs/start/).
Follow the steps to install the program and run the `hello-minikube` application.

Once you have done this, you will need to convert your `docker-compose.yml` files to `kubernetes` deployment files.
There is a tool, [kompose](https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/) which may assist you.

Test that your system works as it did on `docker-compose`.

Learn your way around the `kubernetes` command-line tool, `kubectl` (see the [kubectl documentation](https://kubernetes.io/docs/reference/kubectl/)).
* How can you resize your service, i.e. change the number of running instances (pods)?
* How can you remove one instance of your service (a single pod)?
* How can you see the logs for your service?

### Dealing with long-running jobs and load (challenging)

What does our system do if someone submits a very long-running job? Try testing this with the `sleep` command.
Expand Down

0 comments on commit 64896a4

Please sign in to comment.