Handle support for secondary resource provided by CRD when not present at Operator start

[rwhundley]

Type of question

Best practices
How to implement a specific feature

Question

What did you do?

Our project needs to account for the presence/absence of various CRDs, and, depending upon the settings on our Operator's primary resource, the controller needs to handle those secondary resources. Examples include:

• OpenShift Routes - they should only be created when Routes are available on the Cluster
• CRDs/CRs that are defined and managed by other Operators using separate GVs
• CRDs/CRs that are defined and managed by other Operators using the same GV as our Operator's primary resource

As far as what we did, we wrote functions similar to this response to a previous issue here for the first and second bullets, but, for the third, we used a check of the APIResources listed for the GV to see if that GVK was registered. Using those, we have the following in the init in our main.go:

func init() {
	cfg, err := config.GetConfig()
	if err != nil {
		return
	}

	dc, err := discovery.NewDiscoveryClientForConfig(cfg)
	if err != nil {
		return
	}

	utilruntime.Must(clientgoscheme.AddToScheme(scheme))
	utilruntime.Must(oidcsecurityv1.AddToScheme(scheme))
	utilruntime.Must(certmgrv1.AddToScheme(scheme))

	if controllercommon.ClusterHasRouteGroupVersion(dc) {
		utilruntime.Must(routev1.AddToScheme(scheme))
	}
	if controllercommon.ClusterHasOperandRequestAPIResource(dc) {
		utilruntime.Must(operatorv1alpha1.AddODLMEnabledToScheme(scheme))
	} else {
		utilruntime.Must(operatorv1alpha1.AddToScheme(scheme))
	}
	if controllercommon.ClusterHasZenExtensionGroupVersion(dc) {
		utilruntime.Must(zenv1.AddToScheme(scheme))
	}
	//+kubebuilder:scaffold:scheme
}

In the SetupWithManager function for the controller, we have the following:

func (r *AuthenticationReconciler) SetupWithManager(mgr ctrl.Manager) error {
	builder := ctrl.NewControllerManagedBy(mgr).
		Owns(&corev1.ConfigMap{}).
		Owns(&corev1.Secret{}).
		Owns(&certmgr.Certificate{}).
		Owns(&batchv1.Job{}).
		Owns(&corev1.Service{}).
		Owns(&net.Ingress{}).
		Owns(&appsv1.Deployment{})

	if ctrlcommon.ClusterHasOpenShiftConfigGroupVerison(&r.DiscoveryClient) {
		builder.Owns(&routev1.Route{})
	}
	if ctrlcommon.ClusterHasZenExtensionGroupVersion(&r.DiscoveryClient) {
		builder.Owns(&zenv1.ZenExtension{})
	}
	if ctrlcommon.ClusterHasOperandRequestAPIResource(&r.DiscoveryClient) {
		builder.Owns(&operatorv1alpha1.OperandRequest{})
	}

	return builder.For(&operatorv1alpha1.Authentication{}).
		Complete(r)
}

What did you expect to see?

Ideally, if someone ended up installing the CRDs that this Operator would have skipped loading support for here, the Operator could detect this and load the missing GVs into its scheme. Alternatively, if there's a way for them to always be loaded, but not lead to the Operator exiting when it's not able to watch for them at the start, I would be happy with that as well.

What did you see instead? Under which circumstances?

So far, it seems like by adding them to the scheme in our main.go's init, the Manager will end up trying to watch for them, leading to exits due to failed watches. Makes sense - the resources it's trying to look for are not supported yet. I'd prefer a solution that doesn't mandate a Pod restart when the new APIs become supported.

Environment

Operator type:

/language go

Kubernetes cluster type:

Vanilla and OpenShift

$ operator-sdk version

operator-sdk version: "v1.37.0", commit: "819984d4c1a51c8ff2ef6c23944554148ace0752", kubernetes version: "1.29.0", go version: "go1.21.13", GOOS: "darwin", GOARCH: "arm64"

$ go version (if language is Go)

go version go1.23.1 darwin/arm64

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.1", GitCommit:"05d83eff7e17160e679898a2a5cd6019ec252c49", GitTreeState:"clean", BuildDate:"2023-06-07T16:46:24Z", GoVersion:"go1.19.9", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"28+", GitVersion:"v1.28.13-eks-a737599", GitCommit:"9183cd02caedacf6a14583843262d53d6244fc4a", GitTreeState:"clean", BuildDate:"2024-08-26T21:27:49Z", GoVersion:"go1.22.5", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.26) and server (1.28) exceeds the supported minor version skew of +/-1

Additional context

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale