Provide single use-case based deployment manifests

[shaun-nx]

Use case
Customers have a need to be able to deploy and upgrade the NGINX Ingress Controller with automation tools such as ArgoCD, Flux, etc.
Being able to apply the necessary deployment files and CRDs to their cluster without the need to clone down the repo is necessary to make these operations smoother and involve less workarounds.
One common point of pain that we have heard is that updating the deployment and the CRDs using automation tools is less than stellar and has required additional work arounds as Helm has a less than optimal native process.

What
Customers have asked for:

• Single CRD yaml file

The project has the additional need for:

• Single, out-of-the-box, deployment yaml file for the Ingress Controller
• Use-case or cloud specific deployment yaml files (e.g. AWS, AKS, etc...)

Why
Ease of automation and less overall steps for deploying the Ingress Controller

How
Generate individual Manifests from the Helm templates pertaining to each use-case. Rather than maintaining both Helm and Manifest files separately.

Notes
Many of the requirements and use-cases can be extracted from this PR: #4278

Use cases to start with:

• NGINX OSS (deployment)
• NGINX OSS (daemon-set)
• NGINX Plus (deployment)
• NGINX Plus (daemon-set)

Idea to directory struct
To avoid confusion by keeping two "deployment" directories (i.e. deploy and deployment) we can keep deploy as the main top-level directory with directories for each use-case (as done in #4278) while also keeping a manifests (name not final) which contains the files that currently exist in deployments (e.g. rbacs, ingress-class, configmap, etc...)

### Tasks
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4628
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4678
- [ ] Provide single deployment yaml files for OSS and Plus that are generated by Helm
- [ ] Update documentation to make communicate the new and existing way of deploying the Ingress Controller
- [ ] Update Python tests to use single deployment manifests
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4396
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4434
- [ ] Update GCP marketplace to use helm template of ClusterRole

[github-actions]

Hi @shaun-nx thanks for reporting!

Be sure to check out the docs and the Contributing Guidelines while you wait for a human to take a look at this 🙂

Cheers!

[coolbry95]

Something that we do to avoid issues with CRDs and different versions of nginx-ingress is to deploy the CRDs once per cluster. At any given time we can be on at least two different versions of nginx-ingress.

This is our values.yaml file. It would be nice to just be able to disable all resources. Right now there are a few benign resources that are deployed.

nginx-ingress:
 controller:
   watchNamespace: $(POD_NAMESPACE)
   ingressClass: 
     name: nginx-default
   replicaCount: 0
   service:
     create: false
 rbac:
  create: false

For the other install of nginx-ingress we use --skip-crds helm flag to not install the CRDs.