feat(auth): add config option to deploy oauth2_proxy instead of openshift-oauth-proxy

[andrewazores]

Welcome to Cryostat! 👋

Before contributing, make sure you have:

• Read the contributing guidelines
• Linked a relevant issue which this PR resolves
• Linked any other relevant issues, PR's, or documentation, if any
• Resolved all conflicts, if any
• Rebased your branch PR on top of the latest upstream main branch
• Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
• Signed all commits: git commit -S -m "YOUR_COMMIT_MESSAGE"

---

Related to #710
Fixes #206

Description of the change:

This enables the Operator to deploy oauth2_proxy in place of openshift-oauth-proxy.

openshift-oauth-proxy is deployed by default when the Operator detects that it is installed on OpenShift. There is a new CRD option to disable this integration and force the use of oauth2_proxy instead. If installed on a non-OpenShift Kubernetes then oauth2_proxy will always be deployed.

If oauth2_proxy is deployed it will also respect the same htpasswd Basic auth configuration that I added in #802. If no htpasswd Basic auth is configured then oauth2_proxy will allow any requests to pass without authz.

Motivation for the change:

1. enable auth proxy functionality for installs on non-OpenShift clusters
2. enable end users to deploy Cryostat with htpasswd Basic, with or without OpenShift SSO login - they may not want to use cluster RBAC to control access to the Cryostat API and application, but rather to define separate htpasswd user accounts. Or they may wish to provide access to users who have cluster accounts as well as some users who don't
3. enable end users to deploy Cryostat without any authentication required at all. The configuration is locked down so integrating with other OAuth providers is not currently possible (not via the Operator anyway), but for quick one-off testing or small/private testing clusters, configuring auth may be an unwanted additional step

How to manually test:

1. Build and install Operator as usual
2. Create a CR:

apiVersion: operator.cryostat.io/v1beta2
kind: Cryostat
metadata:
  name: cryostat-sample
spec:
  enableCertManager: false
  authorizationOptions:
    disableOpenShiftSso: false
  trustedCertSecrets: []
  eventTemplates: []
  storageOptions:
    pvc:
      labels: {}
      annotations: {}
      spec: {}
  reportOptions:
    replicas: 0

3. ensure that the application comes up and is secured by OpenShift SSO as before
4. create a basic auth secret like in feat(auth): add configuration for htpasswd basic auth #802 (oc create secret generic basicauth --from-file=htpasswd.conf) patch the CR to add:

authorizationOptions:
  basicAuth:
    secretName: basicauth
    filename: htpasswd.conf

5. ensure that the application is redeployed and is now secured by OpenShift SSO, with the option of Basic auth as well
6. edit the CR to set disableOpenShiftSso: true
7. ensure that the application is redeployed and is now secured by OAuth2 Proxy using the same Basic auth
8. edit the CR to remove the basicAuth object
9. ensure that the application is redeployed and is now accessible without auth
10. bonus: install operator on a non-OpenShift cluster. The disableOpenShiftSso flag should now be ineffective and OAuth2 Proxy should be installed regardless. The basicAuth config option should still function as outlined in the previous steps.

[ebaron]

I think we may just want to handle this by using the IsOpenShift method instead of adding a CRD property. Downstream we want this code path to be inaccessible since we don't plan to build/release the OAuth2 proxy.

[andrewazores]

Hmm, okay. I can do that, but it does mean that once installed on OpenShift the only way to control who has access to the application will be around who has the right roles to pass the SAR (https://github.com/cryostatio/cryostat-operator/pull/799/files#diff-587612cdc407d5ab4abf28789670950c567419bc4ada045dda2e0c1070e74470R632). But I guess that's already how it works in 2.4 if you allow the OpenShift auth integration, so that's still at parity, right?

[ebaron]

Hmm, okay. I can do that, but it does mean that once installed on OpenShift the only way to control who has access to the application will be around who has the right roles to pass the SAR (https://github.com/cryostatio/cryostat-operator/pull/799/files#diff-587612cdc407d5ab4abf28789670950c567419bc4ada045dda2e0c1070e74470R632). But I guess that's already how it works in 2.4 if you allow the OpenShift auth integration, so that's still at parity, right?

Right

[ebaron]

Hmm, okay. I can do that, but it does mean that once installed on OpenShift the only way to control who has access to the application will be around who has the right roles to pass the SAR (https://github.com/cryostatio/cryostat-operator/pull/799/files#diff-587612cdc407d5ab4abf28789670950c567419bc4ada045dda2e0c1070e74470R632). But I guess that's already how it works in 2.4 if you allow the OpenShift auth integration, so that's still at parity, right?

Right

Isn't there also the Basic Auth option now too?

[andrewazores]

Yes, but enabling that in OpenShift gives you the option of logging in with OpenShift SSO or with Basic auth - it doesn't disable the OpenShift SSO part. So it's just another way to define users, ones who don't necessarily have OpenShift logins, but it doesn't provide a way to block access to OpenShift logins.

I don't think there is a way to configure the OpenShift OAuth Proxy to turn off the OpenShift SSO integration, but maybe there's a way to configure it to be just slightly broken on purpose... it would still present the login button to the user, but maybe it can be made to always fail at that stage. Not sure if that would be worthwhile.

[ebaron]

I suppose for full 2.4 parity we'd need the ability to customize the permissions used for the SAR.

[andrewazores]

Would it make sense to add that directly as structures within the CR's new AuthorizationOptions? That's a bit simpler and IMO nicer compared to 2.4's "configmap containing a properties file", I think.

[andrewazores]

Ah, I also just realized I forgot to fix this up:

cryostat-operator/internal/controllers/common/resource_definitions/resource_definitions.go

Line 638 in d455719

"--cookie-secret=REPLACEME",

I'll do that now.

[andrewazores]

I think the CR API looks a little clumsy, but all these cases are covered:

1. deployed on OpenShift, OpenShift SSO for interactive login and Bearer is enabled by default and Basic is not configured
2. deployed on OpenShift, Basic can be additionally configured using spec.authorizationOptions.basicAuth.[secretName|filename]
3. deployed on OpenShift, all auth can be disabled (SSO, and Basic if configured, and Bearer) using spec.authorizationOptions.openShiftSsoRoles.disabled: true
4. deployed on OpenShift, the SSO SubjectAccessReview and TokenReview can be configured using spec.authorizationOptions.openShiftSsoRoles.[subjectAccessReview|tokenReview]
5. deployed on non-OpenShift, all auth is disabled by default
6. deployed on non-OpenShift, Basic can be configured the same way as in point 2

What is not possible is deploying on OpenShift, disabling SSO, and enabling Basic only.

It's also a little clumsy that on OpenShift this defaults to enabling authz, but on non-OpenShift it defaults to deploying wide open. As suggested in #206 we could generate a secret and enable Basic auth by default and attach that to the CR status like the Grafana credentials in 2.4. This way the non-OpenShift user does need to have some kind of cluster access to be able to read the CR status or to read the secret contents in order to log in.

[ebaron]

Is TokenReview right here? That API is used to determine who a token belongs to isn't it? Also, do we need to have separate options for interactive vs non-interactive?

[andrewazores]

I guess it is actually TokenAccessReview, not just TokenReview. I'll rename it.

Also, do we need to have separate options for interactive vs non-interactive?

I think conceptually it makes sense to require the same access reviews either way, but unfortunately the way the proxy works is that you configure the interactive access as a list of resources to check for the SAR, whereas the Bearer auth access is only a single object with route prefixes mapped to TARs.

cryostatio/cryostat-helm#132 (comment)

So I think the most understandable model from the end user's point of view is probably to have a single AccessReview field, which is used for both TokenAccessReview on a catch-all / route prefix, and as a singleton list for SubjectAccessReview. This way there is just one particular role that any client needs (human user or programmatic client) to access any route, other than the health check endpoints.

The most flexible model is to have a list of SARs and a separate value for TAR but again with the catch-all / route prefix. This lets the CR represent all of the configuration possible on the proxy, but it does mean that you can do weird things like require more stringent permissions for browser access than for Bearer token access to the same endpoint.

[andrewazores]

FWIW over in the Helm chart, I have it as a single Bearer TokenAccessReview on the catch-all / prefix, and a list of SubjectAccessReviews. The SAR list defaults to a singleton of the same required permission as the TAR.

[ebaron]

Ah so the delegate URLs option doesn't support a set of permissions per path? I agree with the simplest option then. One SubjectAccessReview field with one ResourceAttributes struct, and this will get used for both openshift-sar and openshift-delegate-urls.

[andrewazores]

That's unfortunately correct, the delegate URLs option which enables Bearer token handling only takes a single permission per path prefix. But it also doesn't allow for inspecting the request at all beyond checking that the request path matches the path prefix, so we can't even differentiate between ex. GET and POST to apply different role mapping.

[ebaron]

Looks good! Just a couple more doc suggestions you can take or leave