From e9962ced2d9d548b1a024356acab745f6df33c2b Mon Sep 17 00:00:00 2001 From: Guillaume Chervet Date: Mon, 5 Jun 2023 11:51:32 +0200 Subject: [PATCH] [no ci] docs(all): re-organize documentation --- CODE_OF_CONDUCT.md | 28 +++++ CONTRIBUTING.md | 21 ++++ ISSUE_TEMPLATE.md | 15 +++ PULL_REQUEST.md | 5 + README.md | 92 ++------------- Summit.md => documentation/Summit.md | 0 documentation/learning.md | 107 ++++++++++++++++++ project.md => documentation/project.md | 0 .../step_0_setup.md | 0 .../step_1_extraction.md | 0 .../step_2_label.md | 0 .../step_3_train.md | 0 .../step_4_share.md | 0 .../step_5_rest_api.md | 0 .../step_6_deploy.md | 0 .../step_7_integration_testing.md | 0 learning.md | 25 ---- 17 files changed, 184 insertions(+), 109 deletions(-) create mode 100644 CODE_OF_CONDUCT.md create mode 100644 CONTRIBUTING.md create mode 100644 ISSUE_TEMPLATE.md create mode 100644 PULL_REQUEST.md rename Summit.md => documentation/Summit.md (100%) create mode 100644 documentation/learning.md rename project.md => documentation/project.md (100%) rename step_0_setup.md => documentation/step_0_setup.md (100%) rename step_1_extraction.md => documentation/step_1_extraction.md (100%) rename step_2_label.md => documentation/step_2_label.md (100%) rename step_3_train.md => documentation/step_3_train.md (100%) rename step_4_share.md => documentation/step_4_share.md (100%) rename step_5_rest_api.md => documentation/step_5_rest_api.md (100%) rename step_6_deploy.md => documentation/step_6_deploy.md (100%) rename step_7_integration_testing.md => documentation/step_7_integration_testing.md (100%) delete mode 100644 learning.md diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..8ac48d85 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,28 @@ +# Contributor Code of Conduct + +As contributors and maintainers of this project, we pledge to respect all +people who contribute through reporting issues, posting feature requests, +updating documentation, submitting pull requests or patches, and other +activities. + +We are committed to making participation in this project a harassment-free +experience for everyone, regardless of level of experience, gender, gender +identity and expression, sexual orientation, disability, personal appearance, +body size, race, age, or religion. + +Examples of unacceptable behavior by participants include the use of sexual +language or imagery, derogatory comments or personal attacks, trolling, public +or private harassment, insults, or other unprofessional conduct. + +Project maintainers have the right and responsibility to remove, edit, or +reject comments, commits, code, wiki edits, issues, and other contributions +that are not aligned to this Code of Conduct. Project maintainers who do not +follow the Code of Conduct may be removed from the project team. + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported by opening an issue or contacting one or more of the project +maintainers. + +This Code of Conduct is adapted from the +[Contributor Covenant](http://contributor-covenant.org), version 1.0.0, +available at [http://contributor-covenant.org/version/1/0/0/](http://contributor-covenant.org/version/1/0/0/) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..fbc406d1 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,21 @@ +# Contributing to @axa-fr/react-oidc + +First, ensure you have the [latest `npm`](https://docs.npmjs.com/). + +To get started with the repository: + +```sh +git clone https://github.com/AxaGuilDEv/react-oidc.git +cd react-oidc/packages/react +npm install +npm start +``` +You are now ready to contribute! + +## Pull Request + +Please respect the following [PULL_REQUEST_TEMPLATE.md](./PULL_REQUEST_TEMPLATE.md) + +## Issue + +Please respect the following [ISSUE_TEMPLATE.md](./ISSUE_TEMPLATE.md) diff --git a/ISSUE_TEMPLATE.md b/ISSUE_TEMPLATE.md new file mode 100644 index 00000000..dd5f3245 --- /dev/null +++ b/ISSUE_TEMPLATE.md @@ -0,0 +1,15 @@ +### Issue and Steps to Reproduce + + + +### Versions + +### Screenshots + +#### Expected + +#### Actual + +### Additional Details + +- Installed packages: diff --git a/PULL_REQUEST.md b/PULL_REQUEST.md new file mode 100644 index 00000000..3bec15ba --- /dev/null +++ b/PULL_REQUEST.md @@ -0,0 +1,5 @@ +## A picture tells a thousand words + +## Before this PR + +## After this PR diff --git a/README.md b/README.md index f492010b..eedac88b 100644 --- a/README.md +++ b/README.md @@ -2,99 +2,18 @@ A real demo of Deep Learning project with preprocessing from development to production using code, ops and Machine Learning best practices. Production is a real time REST API. -Competences -- Code / Debug -- UnitTest, Test Driven Development principles -- Clean Code, Inversion Of Dependence, Functional Programing -- Build & Publish packages -- Git -- Linux Bash -- REST API -- Docker -- Kubernetes - -Tools -- Pycharm (https://www.jetbrains.com/pycharm/) -- Python, pip, PipEnv, Conda -- Git & Github & Github Action -- AzureML (https://azure.microsoft.com/fr-fr/free/students?WT.mc_id=DOP-MVP-5003370) -- Docker -- Kubernetes & OpenShift or Azure - -## 1. Introduction - -The name of our company: MLOps - -Today a team of indexers receive 10,000 PDF files a day containing either cats or dogs or something else. -They must manually open each PDF to classify them. - -We are going to automate this process. We will use a machine learning model to classify PDF files and expose it as a REST API. It will work in real time. - ![project_workflow.png](documentation%2Fproject_workflow.png) -![Supervised_Learning.PNG](documentation%2FSupervised_Learning.PNG) - -### Why MLOps ? - -Many projects do not ship to production due to many constraints: - - Expensive - - Complex Data Workflow - - More actors - -![Triangle_actors.PNG](documentation%2FTriangle_actors.PNG) - -![MLOps_boxes.PNG](documentation%2FMLOps_boxes.PNG) - -![Triangle_paint_points.PNG](documentation%2FTriangle_paint_points.PNG) - -### What is MLOps? - -MLOps is involved in the entire life cycle of an AI project. This is all that will allow your AI project to go into production and then keep your project in production. - -No MLOps practices, no production. - -![MLOps_is_more_about_human_culture.PNG](documentation%2FMLOps_is_more_about_human_culture.PNG) +## Getting Started -### Delivery of a MLOps project - -The deliverable of a project is not the AI model, it's how the AI model is generated. - -The model must be **reproducible**. - -The deliverable is: -- Versioned data -- Versioned code - -![MLOps_graal.PNG](documentation%2FMLOps_graal.PNG) - -## Project structure -We will use the following project structure. We will use mono-repository git in order to work together. It will help to retrain a model and deploy it in production from one manual action maximum. - -![code_organization.PNG](documentation%2Fcode_organization.PNG) - -## ML Libraries/Technologies choice - -Technologies choices have a huge impact on your development cost. -You cannot use a hundred of libraries and expect to have a good result in a short time. -It will be a nightmare to maintain and to debug. - -Enterprises use **Technologies RADAR** to manage and maintain their technologies choice : -https://github.com/axa-group/radar - -![ml_libraries_choices.PNG](documentation%2Fml_libraries_choices.PNG) - -## 2. Requirements - -Download and install: +Requirements, Download and install: - Pycharm: https://www.jetbrains.com/pycharm/ -- Python 3.11.x : https://www.python.org/downloads/ +- Python 3.11.1 : https://www.python.org/downloads/ - Git: https://git-scm.com/downloads - Docker-desktop: https://www.docker.com/products/docker-desktop/ - Postman: https://www.postman.com/downloads/ - NodeJS 18.x: https://nodejs.org/en/download/ -## 3. Getting Started - On windows your have to use "GitBash" only because all commands are linux bash commands. ```sh git clone https://www.github.com/guillaume-chervet/MLOpsPython @@ -115,3 +34,8 @@ docker-compose up # webapp is now available at : http://localhost:4000 # api is available at : http://localhost:4000/health ``` + +## Contribute + +- [How to run to contribute](./CONTRIBUTING.md) +- [Please respect our code of conduct](./CODE_OF_CONDUCT.md) diff --git a/Summit.md b/documentation/Summit.md similarity index 100% rename from Summit.md rename to documentation/Summit.md diff --git a/documentation/learning.md b/documentation/learning.md new file mode 100644 index 00000000..bd84bb37 --- /dev/null +++ b/documentation/learning.md @@ -0,0 +1,107 @@ +# MLOps Python + +Competences +- Code / Debug +- UnitTest, Test Driven Development principles +- Clean Code, Inversion Of Dependence, Functional Programing +- Build & Publish packages +- Git +- Linux Bash +- REST API +- Docker +- Kubernetes + +Tools +- Pycharm (https://www.jetbrains.com/pycharm/) +- Python, pip, PipEnv, Conda +- Git & Github & Github Action +- AzureML (https://azure.microsoft.com/fr-fr/free/students?WT.mc_id=DOP-MVP-5003370) +- Docker +- Kubernetes & OpenShift or Azure + +## 1. Introduction + +The name of our company: MLOps + +Today a team of indexers receive 10,000 PDF files a day containing either cats or dogs or something else. +They must manually open each PDF to classify them. + +We are going to automate this process. We will use a machine learning model to classify PDF files and expose it as a REST API. It will work in real time. + +![project_workflow.png](project_workflow.png) + +![Supervised_Learning.PNG](Supervised_Learning.PNG) + +### Why MLOps ? + +Many projects do not ship to production due to many constraints: + - Expensive + - Complex Data Workflow + - More actors + +![Triangle_actors.PNG](Triangle_actors.PNG) + +![MLOps_boxes.PNG](MLOps_boxes.PNG) + +![Triangle_paint_points.PNG](Triangle_paint_points.PNG) + +### What is MLOps? + +MLOps is involved in the entire life cycle of an AI project. This is all that will allow your AI project to go into production and then keep your project in production. + +No MLOps practices, no production. + +![MLOps_is_more_about_human_culture.PNG](MLOps_is_more_about_human_culture.PNG) + +### Delivery of a MLOps project + +The deliverable of a project is not the AI model, it's how the AI model is generated. + +The model must be **reproducible**. + +The deliverable is: +- Versioned data +- Versioned code + +![MLOps_graal.PNG](MLOps_graal.PNG) + +## Project structure +We will use the following project structure. We will use mono-repository git in order to work together. It will help to retrain a model and deploy it in production from one manual action maximum. + +![code_organization.PNG](code_organization.PNG) + +## ML Libraries/Technologies choice + +Technologies choices have a huge impact on your development cost. +You cannot use a hundred of libraries and expect to have a good result in a short time. +It will be a nightmare to maintain and to debug. + +Enterprises use **Technologies RADAR** to manage and maintain their technologies choice : +https://github.com/axa-group/radar + +![ml_libraries_choices.PNG](ml_libraries_choices.PNG) + + +## Step 0 : Setup +[step_0_setup.md](step_0_setup.md) + +## Step 1: Extract images from PDF files +[step_1_extraction.md](step_1_extraction.md) + +## Step 2: Label images +[step_2_label.md](step_2_label.md) + +## Step 3: Train AI model +[step_3_train.md](step_3_train.md) + +## Step 4: Share common code as libraries for production +[step_4_share.md](step_4_share.md) + +## Step 5: REST API +[step_5_rest_api.md](step_5_rest_api.md) + +## Step 6: Deploy +[step_6_deploy.md](step_6_deploy.md) + +## Step 7: Integration testing +[step_7_integration_testing.md](step_7_integration_testing.md) \ No newline at end of file diff --git a/project.md b/documentation/project.md similarity index 100% rename from project.md rename to documentation/project.md diff --git a/step_0_setup.md b/documentation/step_0_setup.md similarity index 100% rename from step_0_setup.md rename to documentation/step_0_setup.md diff --git a/step_1_extraction.md b/documentation/step_1_extraction.md similarity index 100% rename from step_1_extraction.md rename to documentation/step_1_extraction.md diff --git a/step_2_label.md b/documentation/step_2_label.md similarity index 100% rename from step_2_label.md rename to documentation/step_2_label.md diff --git a/step_3_train.md b/documentation/step_3_train.md similarity index 100% rename from step_3_train.md rename to documentation/step_3_train.md diff --git a/step_4_share.md b/documentation/step_4_share.md similarity index 100% rename from step_4_share.md rename to documentation/step_4_share.md diff --git a/step_5_rest_api.md b/documentation/step_5_rest_api.md similarity index 100% rename from step_5_rest_api.md rename to documentation/step_5_rest_api.md diff --git a/step_6_deploy.md b/documentation/step_6_deploy.md similarity index 100% rename from step_6_deploy.md rename to documentation/step_6_deploy.md diff --git a/step_7_integration_testing.md b/documentation/step_7_integration_testing.md similarity index 100% rename from step_7_integration_testing.md rename to documentation/step_7_integration_testing.md diff --git a/learning.md b/learning.md deleted file mode 100644 index 443eb8b2..00000000 --- a/learning.md +++ /dev/null @@ -1,25 +0,0 @@ -# Build solution from scratch - -## Step 0 : Setup -[step_0_setup.md](step_0_setup.md) - -## Step 1: Extract images from PDF files -[step_1_extraction.md](step_1_extraction.md) - -## Step 2: Label images -[step_2_label.md](step_2_label.md) - -## Step 3: Train AI model -[step_3_train.md](step_3_train.md) - -## Step 4: Share common code as libraries for production -[step_4_share.md](step_4_share.md) - -## Step 5: REST API -[step_5_rest_api.md](step_5_rest_api.md) - -## Step 6: Deploy -[step_6_deploy.md](step_6_deploy.md) - -## Step 7: Integration testing -[step_7_integration_testing.md](step_7_integration_testing.md) \ No newline at end of file