diff --git a/content/for-delivery-engineers/explanation/secrets.md b/content/for-delivery-engineers/explanation/secrets.md deleted file mode 100644 index 70a01929..00000000 --- a/content/for-delivery-engineers/explanation/secrets.md +++ /dev/null @@ -1,89 +0,0 @@ -# Pipeline Secrets - -SAAP Pipelines requires secrets for CI/CD workflow. Following are the secrets used, along with their usage details. - -## nexus-helm-auth - -**Purpose:** nexus-helm-auth secret is used in CI pipeline to push and pull helm charts from private nexus registry hosted on the cluster. This secret contains credentials for a machine-user to login into the registry. - -**Owner:** Stakater - -**Type:** Service Account (Shared user) - -**User for:** CI pipelines - -**Do not use for:** Logging in to registry from your local machine - -**Creation:** This secret is created at the time Nexus is set up. It is hard-coded in `nexus-pre-install` configMap, and created through `nexus-setup` job. - -**Rotation:** This secret needs to be added to Vault, which is then reconciled by External Secrets Operator, and a Kubernetes secret is created in cluster against it. - -**Stored in:** Keeper - -## nexus-docker-config - -**Purpose:** nexus-docker-config secret is used in CI pipeline to push and pull container images from private nexus registry hosted on the cluster. This secret contains credentials for a machine-user to login into the registry. - -**Owner:** Stakater - -**Type:** Service Account (Shared user) - -**User for:** CI pipelines - -**Do not use for:** Logging in to registry from your local machine - -**Creation:** This secret is created at the time Nexus is set up. It is hard-coded in `nexus-pre-install` configMap, and created through `nexus-setup` job. - -**Rotation:** This secret needs to be added to Vault, which is then reconciled by External Secrets Operator, and a Kubernetes secret is created in cluster against it. - -**Stored in:** Keeper - -**Sample Secret**: - -```yaml -kind: Secret -apiVersion: v1 -metadata: - name: nexus-docker-config - namespace: -immutable: false -data: - .dockerconfigjson: >- - - config: >- - -type: kubernetes.io/dockerconfigjson -``` - -Sample .dockerconfigjson: - -```json -{ - "auths": { - "https://nexus-docker-stakater-nexus.jlvwjls8.kubeapp.cloud": { - "auth": - }, - "https://nexus-docker-proxy-stakater-nexus.apps.jlvwjls8.kubeapp.cloud": { - "auth": - } - } - } -``` - -## git-auth - -**Purpose:** git-auth secret is used in CI pipeline. It's purpose is to clone git repositories into workspaces that are used in pipeline steps to perform any defined action. The secret contains a token that has access to defined user and repository permission (fine-grained token). - -**Owner:** Stakater - -**Type:** Service Account (Shared user) - -**User for:** CI pipelines - -**Do not use for:** Logging in to registry from your local machine - -**Creation:** This secret is created by GitHub organization admin. More details on how to create this can be seen [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) - -**Rotation:** This secret needs to be added to Vault, which is then reconciled by External Secrets Operator, and a Kubernetes secret is created in cluster against it. - -**Stored in:** Keeper diff --git a/content/for-delivery-engineers/overview.md b/content/for-delivery-engineers/overview.md index 944f9495..3b9138ec 100644 --- a/content/for-delivery-engineers/overview.md +++ b/content/for-delivery-engineers/overview.md @@ -1,6 +1,16 @@ # Overview The "For Delivery Engineer" section focuses on providing information tailored explicitly for delivery engineers involved in GitOps-based application delivery. +The primary goal of this section is to enable the developers to set up their CI-CD workflows. The delivery engineer is expected to have elevated permissions over their organization in their SCM provider to be able to create tokens and SSH keys. +In this section, we will create the two repositories that are required for our CI-CD workflow. Later we will generate Tokens and SSH keys for accessing these repositories. + +We manage GitOps with two different kinds of repository with different purpose enlisted below: + +- **`Apps GitOps Config`**: Used for delivering applications belonging to tenants. +- **`Infra GitOps Config`**: Used for delivering cluster scoped resources for application tenants or other services. + +You can pick any name for these two repositories as long as they explain the purpose well. + This section is divided into three main parts: Explanations, How-to-Guides, and Tutorials. ## Explanation @@ -11,8 +21,6 @@ The explanation contains all the required details that answer the obvious querie - [Stakater Opinionated GitOps Structure](./explanation/gitops-structure.md): This chapter introduces Stakater's opinionated GitOps structure. It explains how Stakater recommends organizing the GitOps repositories, directory structure, and naming conventions for managing infrastructure and application configurations with the deployment manifests. -- [Pipeline Secrets](./explanation/secrets.md): This chapter covers the important secrets that are necessary to have in the Stakater's deployment pipeline process. - - [Types of Environments](./explanation/types-of-environments.md): This chapter explores different environments involved in Stakater's application delivery. - [Stakater Tekton Chart](./explanation/stakater-tekton-chart.md): This chapter focuses on Stakater's Tekton Chart, which is a Helm chart designed to streamline the configurations and components needed by Tekton pipelines for building efficient CI/CD. diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/backup-restore.md b/content/for-developers/tutorials/inner-loop/add-backup-schedule/backup-restore.md new file mode 100644 index 00000000..1e63d8bb --- /dev/null +++ b/content/for-developers/tutorials/inner-loop/add-backup-schedule/backup-restore.md @@ -0,0 +1,124 @@ +# Creating a backup schedule + +Creating a backup schedule for an application using the OpenShift ADP (Application Data Protection) Operator involves several steps. This guide will walk you through the process, from installing the ADP Operator to configuring backup schedules. + +## Objectives + +- Create a backup schedule for your application +- Restore application + +## Key Results + +- Application successfully restored using the OADP operator + +## Tutorial + +### Add Namespace to Tenant + +You will need create a TENANT-system namespace for Back and Restore functionality to work. +When the system namespace is added, MTO deploys the OADP operator to the TENANT-system namespace. +Let's see how you can create this namespace. + +1. Open up the `infra-gitops-config` repository. + +1. Navigate to the folder containing the Tenant definitions. + +1. Now open the Tenant to which your application belongs. + +1. Under the namespaces, add the 'system' namespace. Commit the change + + ![tenant system namespace](images/tenant-system.png) + +1. After a few, the system namespace should start showing in SAAP. You will also see some pods running in the namespace + + ![system namespace pods](images/tenant-system-pods.png) + +### Create a Backup Schedule + +For creating a backup schedule, you will need to deploy the Schedule CR to the TENANT-system namespace +Let us deploy a Backup Schedule for our application + +1. Navigate to the application GitOps repository. In your application's environments, add a 'system' environment folder. + + ![system environment](images/system-folder.png) + +1. Next, add the ArgoCD application that points to the above folder. + + ![ArgoCD application](images/argocd-app.png) + +1. Now add the Backup Schedule in the system environment folder. Remember to replace the placeholder values. + +```yaml +apiVersion: velero.io/v1 +kind: Schedule +metadata: + name: [APPNAME]-backup + namespace: [TENANT]-system +spec: + schedule: * 11 * * * + template: + defaultVolumesToRestic: true + excludedResources: + - daemonsets + - statefulsets + - nodes + - apiservices + - events + - resourcequotas + - controllerrevisions.apps + includedNamespaces: + - [TENANT]-dev + includedResources: + - deployments + - services + - persistentvolumeclaims + - secrets + - configmaps + labelSelector: + matchLabels: + app.kubernetes.io/part-of: [APP-NAME] #Replace this with a label present on your application + snapshotVolumes: true + storageLocation: dpa-1 + ttl: 1h0m0s + +``` + + !!! note + In the labelSelector field, add a common label that is present on all the resources of your application + +1. Once ArgoCD syncs the changes, you should be able to see Schedule CR on the cluster. It should show as 'Enabled'. + + ![schedule](images/schedule.png) + +1. When the scheduled time arrives, a backup will be automatically created and stored in the designated storage location configured by the SAAP Admin. For the purpose of this tutorial, we are using AWS S3 buckets to store the backups. + +### Restore Application from Backup + +1. To restore the application, you will simply need to deploy the Restore CR to `tenant-system` namespace. + Here's a sample configuration: + +```yaml +apiVersion: velero.io/v1 +kind: Restore +metadata: + name: arsenal-dev-restore + namespace: arsenal-system +spec: + backupName: "" + includedNamespaces: + - [TENANT]-dev + itemOperationTimeout: 4h0m0s + restorePVs: true + scheduleName: [SCHEDULE-NAME] +``` + + Remember to replace the placeholder values. + + ![restore](images/restore-cr.png) + + !!! note + When restoring using a schedule, the `backupName` should be empty. OADP will automatically replace it with the latest backup. + +1. Once the Restore CR is created, you will see the Restore showing 'InProgress'. After the Backup is complete, the status will show as complete. + + ![restore](images/restore.png) diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/argocd-app.png b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/argocd-app.png new file mode 100644 index 00000000..21cd6cbb Binary files /dev/null and b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/argocd-app.png differ diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/restore-cr.png b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/restore-cr.png new file mode 100644 index 00000000..5021e438 Binary files /dev/null and b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/restore-cr.png differ diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/restore.png b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/restore.png new file mode 100644 index 00000000..dd48a614 Binary files /dev/null and b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/restore.png differ diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/schedule.png b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/schedule.png new file mode 100644 index 00000000..243f4a20 Binary files /dev/null and b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/schedule.png differ diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/system-folder.png b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/system-folder.png new file mode 100644 index 00000000..eba6a909 Binary files /dev/null and b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/system-folder.png differ diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/tenant-system-pods.png b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/tenant-system-pods.png new file mode 100644 index 00000000..358ee17e Binary files /dev/null and b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/tenant-system-pods.png differ diff --git a/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/tenant-system.png b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/tenant-system.png new file mode 100644 index 00000000..06260c5d Binary files /dev/null and b/content/for-developers/tutorials/inner-loop/add-backup-schedule/images/tenant-system.png differ diff --git a/theme_override/mkdocs.yml b/theme_override/mkdocs.yml index bda25732..388f1ce6 100644 --- a/theme_override/mkdocs.yml +++ b/theme_override/mkdocs.yml @@ -68,7 +68,6 @@ nav: - Explanation: - for-delivery-engineers/explanation/gitops-intro.md - for-delivery-engineers/explanation/gitops-structure.md - - for-delivery-engineers/explanation/secrets.md - for-delivery-engineers/explanation/types-of-environments.md - for-delivery-engineers/explanation/stakater-tekton-chart.md - for-delivery-engineers/explanation/faq.md @@ -113,6 +112,7 @@ nav: - for-developers/tutorials/inner-loop/add-grafana-dashboard/add-grafana-dashboard.md - for-developers/tutorials/inner-loop/scale-app/scale-app.md - for-developers/tutorials/inner-loop/validate-auto-reload/validate-auto-reload.md + - for-developers/tutorials/inner-loop/add-backup-schedule/backup-restore.md - for-developers/tutorials/inner-loop/add-pdb/add-pdb.md - for-developers/tutorials/inner-loop/add-network-policy/add-network-policy.md - for-developers/tutorials/inner-loop/tilt-zero-to-hero/step-by-step-guide.md