Change operator cluster role to be read only by default

[atoulme]

Component(s)

auto-instrumentation

Is your feature request related to a problem? Please describe.

The operator cluster role as defined here:
https://github.com/open-telemetry/opentelemetry-helm-charts/blob/main/charts/opentelemetry-operator/templates/clusterrole.yaml
This role gives write access to many objects across all namespaces.

Describe the solution you'd like

This should be kept down to a minimum of privileges:

• Can it be filtered down to specific namespaces?
• Can we document why the operator requires the right to, say, delete services?

Describe alternatives you've considered

No response

Additional context

No response

[atoulme]

I have shown it is possible to reduce rights of the operator greatly with https://github.com/otel-warez/whitegloves-operator

[swiatekm]

Can it be filtered down to specific namespaces?

Most likely yes. If we're ok without getting automatic RBAC for components, we don't need to create ClusterRoles, and all the other resources are namespaced. The operator already has an option to be scoped to a set of selected namespaces.

Can we document why the operator requires the right to, say, delete services?

We should, yes. There's been some discussion about this in #3156. The delete permissions exist for two reasons:

• Sometimes resources are immutable, and we need to recreate them. Some changes to StatefulSets are like this. I think there's consensus that we should instead reject these kinds of changes and make the user recreate the whole Collector CR.
• Some resources are optional, in that their existence depends on an attribute of the Collector CR. For example, whether we create ServiceMonitors for a Collector is like that. If the user disables it after enabling it in the past, we should delete it. We discussed this during the SIG meeting on 05.12.2024, and reached consensus that we're ok not making this deletion a condition of a successful reconcile. So, if the operator doesn't have the necessary RBAC for deletion, it'd log a warning, perhaps add a message to the resource status, and carry on.

In short, there's a path to let the operator work without delete permissions, if at the cost of some functionality.