Skip to content

Commit

Permalink
Merge remote-tracking branch
Browse files Browse the repository at this point in the history
  • Loading branch information
Uqqasha committed Sep 23, 2024
2 parents 688d3ae + d159af6 commit 5ea1939
Show file tree
Hide file tree
Showing 35 changed files with 850 additions and 4 deletions.
3 changes: 1 addition & 2 deletions contributions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,6 @@ General Intro about the course.
### Week 4

1. [_CRISP-ML(Q). The End-to-End Machine Learning Workflow_](https://github.com/KTH/devops-course/pull/2417)
1. [The importance of reliable testing methods](https://github.com/KTH/devops-course/pull/2398)
1. [Using Bicep to edit Azure resources as code](https://github.com/KTH/devops-course/pull/2398)
1. [Tensorboard: A suite of visualization tools to understand, debug, and optimize TensorFlow programs for ML experimentation](https://github.com/KTH/devops-course/pull/2442)
1. [Dynamic model rollbacks using MLflow](https://github.com/KTH/devops-course/pull/2452)
1. [CodeRabbit: Automated AI Code Reviews](https://github.com/KTH/devops-course/pull/2460)
Expand All @@ -61,4 +59,5 @@ General Intro about the course.
1. [Using Comet ML to analyze and compare the performance of ML models](https://github.com/KTH/devops-course/pull/2485)
1. [MLOps for Cyber-Physical Production Systems: Challenges and Solutions](https://github.com/KTH/devops-course/pull/2486)
1. [Quality Assurance in MLOps Setting: An Industrial Perspective](https://github.com/KTH/devops-course/pull/2479)
1. [The importance of reliable testing methods](https://github.com/KTH/devops-course/pull/2397)

1 change: 1 addition & 0 deletions contributions/demo/week5/aleliu-jakstrom
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

24 changes: 24 additions & 0 deletions contributions/demo/week5/aradberg-wiktord/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Assignment Proposal

## Title

Check-mate: Keep your OpenTofu configurations problem-free

## Names and KTH ID

- Anton Rådberg ([email protected])
- Wiktor Dobrosierdow ([email protected])

## Deadline
- Week 5

## Category
- Demo

## Description
This demo will demonstrate how Checkov can be used to analyze IaC configuration files to avoid misconfigurations and policy violations making it into production. The infrastructure will be built upon OpenTofu and GitHub actions will be used to run Checkov tests.

**Relevance**

Identifying and preventing security and compliance issues before deployment is important in IaC contexts even more so because of increased automation in DevOps contexts.

26 changes: 26 additions & 0 deletions contributions/demo/week5/arberl-sorger/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

Infrastructure as Code Using Ansible Playbooks

## Names and KTH ID

- Arber Limani ([email protected])
- Tom Sorger ([email protected])

## Deadline

- Week 7

## Category

- Demo

## Description

In this demo, we'll demonstrate how to use `Ansible` to automate the deployment of an application. This approach is useful when setting up multiple instances of the same application for development, testing, and production. The demo will show how `Ansible playbooks` can automatically configure all instances in a consistent manner, ensuring the same setup across every environment.

**Relevance**

This demo is highly relevant to DevOps because it showcases how automation tools like Ansible can streamline the deployment process, ensuring consistency across multiple environments. By automating infrastructure and configuration management, it reduces the risk of human error and allows teams to quickly deploy and scale applications in development, testing, and production environments.
26 changes: 26 additions & 0 deletions contributions/demo/week5/jmatsso-jtuv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

Using Pulumi Policy as Code to enforce rules on IaC managed cloud resources.

## Names and KTH ID

- Johannes Matsson ([email protected])
- Jonatan Tuvstedt ([email protected])

## Deadline

- Week 5

## Category

- Demo

## Description

The goal of this assignment is to demonstrate Pulumi's Policy as Code solution by showing a Policy for Cloud Storage (either Google cloud storage or AWS S3) in action. For this we will have a Pulumi CrossGuard policy for cloud storage and non compliant cloud storage resource created and managed by pulumi. We will first show the non compliant cloud storage in the cloud, then demonstrate that the CrossGuard Policy does not approve it. After that we will live fix the storage before finally showing it passing the Policy as Code check and being deployed.

**Relevance**

Infrastructure as code is an important topic in devops as being able to use code instead of manually managing disparate resources through online control centers is difficult, annoying and hard to have an overview of. But it is still easy to create unsafe or bad cloud resources with IaC, and therefore having a Policy as Code checks to enforce Policy compliance allows you to further integrate your IaC into your CI/CD pipeline.
26 changes: 26 additions & 0 deletions contributions/demo/week6/carlwang-jrybak/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

_Streamlining Python Dependency Management with Poetry_

## Names and KTH ID

- Carl Wang ([email protected])
- Jakub Rybak ([email protected])

## Deadline

- Week 6

## Category

- Demo

## Description

We will demonstrate how Poetry simplifies Python dependency management, from adding/removing dependencies to automatically updating configuration files, i.e. `pyproject.toml` and `poetry.lock`. Additionally, we will showcase more advanced features like dependency grouping, selective installation, and Poetry scripting to automate additional dependency related tasks. We'll also show how to export dependencies to `requirements.txt` for compatibility with other tools. The demo will highlight Poetry’s capabilities in dependency resolution and version constraints, paired with sample Docker deployment to showcase Poetry's seamless integration with containerization workflows.

**Relevance**

Automatic dependency management tools are widely used today, it is a crucial part of DevOps due to its efficiency in syncing environments across different machines. Tools like Maven and Gradle for Java are popular and industry standard. However, less people are aware that there are also such tools for Python, a more environment relied language. By leveraging Poetry's powerful dependency management capabilities and integrating it with containerization platforms, such as Docker, developers can ensure consistent and reproducible Python environments across development, testing, and production stages.
26 changes: 26 additions & 0 deletions contributions/demo/week6/melvinj-lindefor/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

Ensuring the integrity and source of software packages

## Names and KTH ID

- Martin Lindefors ([email protected])
- Melvin Jakobsson ([email protected])

## Deadline

- Week 6

## Category

- Demo

## Description

We intend to demonstrate how easy it can be to fall victim to attacks such as typosquatting or dependency confusion. We will do this by creating two bogus packages in `pip` & `npm` and highlight some weaknesses in those package managers. We will then show how to cryptographically verify the authenticity of packages using Sigstore & `npm audit`. Finally we will end with a note on the importance of verifying the origin of software, an often overlooked aspect of software development.

**Relevance**

Since the demo will include both package managers and software verification it is relevant for both of this weeks topics. In particular, verifying the origin of software packages is more relevant than ever because of the widespread nature of package managers and the comfortability of outsourcing code to these packages. Furthermore, there are several examples of attacks related to package managers.
26 changes: 26 additions & 0 deletions contributions/demo/week6/sofiaedv-ludvigch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

_Hardening GitHub Actions for increased security_

## Names and KTH ID

- Sofia Edvardsson ([email protected])
- Ludvig Christensen ([email protected])

## Deadline

- Week 6

## Category

- Demo

## Description

We plan to demonstrate how [StepSecurity Harden-Runner](https://github.com/step-security/harden-runner) can be implemented to harden the security of GitHub Actions runners. We will showcase how to mitigate and detect different attacks, such as supply chain attacks and workflow credential exfiltration. We will also briefly compare Harden-Runner to tools with similar functionality.

**Relevance**

DevOps principles, such as CI/CD, can by themselves introduce new attack vectors. A [recent example](https://adnanthekhan.com/2024/04/15/an-obscure-actions-workflow-vulnerability-in-googles-flank/) being a security researcher finding a vulnerability in the GitHub Actions workflow in an open source Google repository, which could have resulted in a supply chain attack. Our demo will showcase mitigations and detection of such attack vectors, making it relevant to DevOps, and more specifically DevSecOps.
26 changes: 26 additions & 0 deletions contributions/demo/week6/yilei-lawang/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

Comparing pnpm, npm and yarn

## Names and KTH ID

- Yilei Cheng ([email protected])
- Laicaiwei Wang ([email protected])

## Deadline

- Week 6

## Category

- Demo

## Description

In this demo, we will compare three popular JavaScript package managers: npm, Yarn, and pnpm. We’ll explore their core features, performance differences, and unique capabilities. Through live demonstrations, we’ll compare installation speeds, disk space usage, and other practical aspects to help you understand which tool might best suit your development needs. By the end of the session, you’ll have a clear perspective on the strengths and trade-offs of each package manager, enabling you to make an informed choice for your projects.

**Relevance**

Comparing npm, Yarn, and pnpm is crucial for understanding how different package managers handle dependency management. Each tool offers unique features and optimizations that affect installation speed, disk usage, and dependency resolution. Knowing these differences helps you choose the best tool for efficient and effective project management.
26 changes: 26 additions & 0 deletions contributions/executable-tutorial/arberl-mateusma/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

Nixpkgs and its use for reproducible environments for better local testing.

## Names and KTH ID

- Arber Limani ([email protected])
- Mateus Marinheiro ([email protected])

## Deadline

- Task 3

## Category

- Executable Tutorial

## Description

We will give a brief overview of the Nixpkgs syntax as well as its benefits while teaching how to apply it to real world scenarios. We will also provide a guide on how to change your environments to test your code locally before committing it to your repo.

**Relevance**

It is very important to test your code before you submit it, even if there are CI actions in place. As a team starts to grow, different developers with different machines with different environment states. It is therefore essential to ensure that even if machines have different configurations the result of testing should be the same.
34 changes: 34 additions & 0 deletions contributions/executable-tutorial/peremr-inigoaa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Assignment Proposal

## Title

Managing project dependencies with Devbox

## Names and KTH ID

- Íñigo Aréjula Aísa ([email protected])
- Pere Mateu Raventós ([email protected])

## Deadline

- Task 2

## Category

- executable-tutorial

## Description

Devbox is a command-line tool that lets you easily create isolated shells for development. You start by defining the list of packages required for your project, and Devbox creates an isolated, reproducible environment with those packages installed. We will create a tutorial that explains how to use Devbox to manage project dependencies. The tutorial will cover the following topics:
- Why devbox is useful
- How to install dependencies with devbox
- How to create scripts in devbox
- How to replicate the same environment in your CI/CD pipeline with GH Actions

**Relevance**

Devbox is relevant to DevOps as it simplifies creating reproducible development environments, ensuring consistency across local, testing, and production setups. This aligns with DevOps principles, improving collaboration, automation, and the reliability of CI/CD pipelines.

**Links to our tutorial**
Killercoda [https://killercoda.com/arejula27/scenario/devbox-tutorial](https://killercoda.com/arejula27/scenario/devbox-tutorial)
Github repository [https://github.com/arejula27/devbox-tutorial](https://github.com/arejula27/devbox-tutorial)
26 changes: 26 additions & 0 deletions contributions/executable-tutorial/smhanna-anouiser/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Assignment Proposal

## Title

Multibranch pipelines with ArgoCD

## Names and KTH ID

- Sam Maltin ([email protected])
- Amin Nouiser ([email protected])

## Deadline

- Task 1

## Category

- Executable tutorial

## Description

The tutorial will demonstrate how to set up and manage a multibranch pipeline for Kubernetes clusters using ArgoCD. We'll guide readers through creating a development branch for testing a web application and a production branch for stable releases. ArgoCD will be integrated with Git to automate the deployment process, ensuring that each commit triggers an update.

**Relevance**

As automation is a core principle of DevOps, this tutorial is highly relevant to good DevOps practices.
2 changes: 2 additions & 0 deletions contributions/feedback/atheers-miladsf/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,5 @@ Feedback on Scientific Paper [TinyMLOps: Operational Challenges for Widespread E
## Description

We want to provide a feedback for the scientfic paper presentation [TinyMLOps: Operational Challenges for Widespread Edge AI Adoption](https://github.com/KTH/devops-course/pull/2478)

We have add the feedback as comment on the original PR [here](https://github.com/KTH/devops-course/pull/2478#issuecomment-2353081693) and we have also added the feedback in **feedback.md**
58 changes: 58 additions & 0 deletions contributions/feedback/atheers-miladsf/feedback.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Feedback

By Atheer Salim, Milad Farahani

Number of Words: 608
## Code of Conduct

**I/We certify that generative AI, incl. ChatGPT, has not been used to write this feedback. Using generative AI without permission is considered academic misconduct.**

## High Level Overview

### Strengths
- The presentation covers critical issues about TinyMLOps that are highly relevant in the current day and age.
- The structure was well-thought-out, with coverage of relevant aspects of TinyMLOps.
- Good introduction with an engaging aspect of humor.
- Inclusion of technical part regarding TinyMLOps.
- There is a good emphasis on practical implications which offers insight into real-world applications.
- Excellent speech and tone.
- Great illustrations
- Good Take-Home Message


### Weaknesses
- Wrong date on the intro slides.
- The slides contain the name and slide number to the bottom right which is not accurate with the current slide.
- Clearly distinguish the content included in the paper you are presenting from content taken from other papers.
- Talk a bit more about the related papers in terms of their achievement.

## Additional Material

The paper [Implementing MLOps on Edge-Cloud Systems: A New Paradigm for Training at the Edge](https://uwspace.uwaterloo.ca/items/bc1f43a4-96dd-44d6-946f-b5cd19660647) proposes a MLOps architecture used to orchestrate edge-cloud model training and synchronization. The model has been tested on IoT devices, this material could be relevant to look at

The presentation briefly mentioned Federated Learning. A deeper overview of Federated Learning as opposed to Centralized Training could be valuable. One of the many sources providing good insight regarding Federated Learning is Google’s work. They clearly describe how Federated Learning could serve as a reference for understanding how updates can be managed across distributed devices. Here is one of the many sources: [Federated Learning: Collaborative Machine Learning without Centralized Training Data](https://research.google/blog/federated-learning-collaborative-machine-learning-without-centralized-training-data/)

## More Detailed Feedback

### General
The tone and the speech of the presenters were very good. But just like in machine learning, more training would result in an even better speech :)

The way the presenters covered the need to address challenges in edge AI deployment was done in a good way, making it understandable for the listener.

The technical part was explained in a manner that was coherent for the listener and not overwhelming. There were also readable code snippets included which extended the technical details even further.

### Introduction
In the first few slides of your presentation, it contains the date “21 September 2024” which does not align with the date of week 4 presentation time. Which should be “18 September 2024”

The touch of humor was very refreshing and multiple jokes were funny. Especially the AI standing for "Apple Intelligence" joke. Another entertaining visual was the hand pinching “TinyMLOps” indicating that it is “tiny”.

### Fragmented landscape Slide
When discussing ONNX it would be nice if you could show an example of the exported format that would make it easier to understand and clearly show the benefits it has.

When you mention Rune, it should be clearly highlighted that this was not presented in the paper you are presenting and that it comes from another paper. Furthermore, the citation for the paper (presenting Rune) is not visible since the text color matches the background, making it hard to see.

### Protecting Models Intellectual Property Slide
Could you elaborate further on what exactly “local access” means when you are talking about direct stealing, does this mean physical access, root access, or what exactly?

### Take Home Message Slide
In the end, it was evident that the take-home message was well-thought-out. It was broad enough for it to cover TinyMLOps but also specific enough to encapsulate the purpose of the paper and its findings.
22 changes: 22 additions & 0 deletions contributions/feedback/campione-sorger/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Assignment Proposal

## Title

Feedback on "nixpgs for reproducible environments"

## Names and KTH ID

- Marco Campione ([email protected])
- Tom Sorger ([email protected])

## Deadline

- Task 3

## Category

- Feedback

## Description

We would like to give feedback for [this executable tutorial](https://github.com/KTH/devops-course/pull/2455).
Loading

0 comments on commit 5ea1939

Please sign in to comment.