GKE: DataPlane in Konnect never gets marked as ready

[mlavacca]

Current Behavior

When creating a DataPlane with a KonnectExtension all the expected resources are provisioned and properly works, but the DataPlane CR never gets marked as ready by KGO

Expected Behavior

When the DataPlane is properly configured and running, it should be marked as Ready by KGO

Additional Information

This issue has been observed in GKE. It should be tested if that happens in Kind or other environments as well.

[programmer04]

It occurs on a GKE Autopilot cluster. You can use config/samples/dataplane-konnect-extension.yaml to reproduce it.

The root cause is mutation of DataPlane's respective K8s Deployment by K8s GKE Autopilot cluster, see

...
- 				Resources: v1.ResourceRequirements{
- 					Limits: v1.ResourceList{
- 						s"cpu":               {i: resource.int64Amount{...}, s: "1", Format: "DecimalSI"},
- 						s"ephemeral-storage": {i: resource.int64Amount{...}, s: "1Gi", Format: "BinarySI"},
- 						s"memory":            {i: resource.int64Amount{...}, Format: "BinarySI"},
- 					},
- 					Requests: v1.ResourceList{
- 						s"cpu":               {i: resource.int64Amount{...}, s: "100m", Format: "DecimalSI"},
- 						s"ephemeral-storage": {i: resource.int64Amount{...}, s: "1Gi", Format: "BinarySI"},
- 						s"memory":            {i: resource.int64Amount{...}, s: "103Mi", Format: "BinarySI"},
- 					},
- 				},
+ 				Resources: v1.ResourceRequirements{
+ 					Limits: v1.ResourceList{
+ 						s"cpu":    {i: resource.int64Amount{...}, s: "1", Format: "DecimalSI"},
+ 						s"memory": {i: resource.int64Amount{...}, Format: "BinarySI"},
+ 					},
+ 					Requests: v1.ResourceList{
+ 						s"cpu":    {i: resource.int64Amount{...}, s: "100m", Format: "DecimalSI"},
+ 						s"memory": {i: resource.int64Amount{...}, s: "20Mi", Format: "BinarySI"},
+ 					},
+ 				},
...
- 				SecurityContext:          s"&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[NET_RAW],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,"...,
+ 				SecurityContext:          nil,

where those marked with - are from GKE and those with + are prepared by KGO.

Modified fields

• SecurityContext adjustment is the result of Security measures in Autopilot
• Resources adjustment is the result of Resource requests in Autopilot

that leads to getting stuck on

gateway-operator/controller/dataplane/controller.go

Lines 213 to 218 in 52f9b09

	if err != nil {
	return ctrl.Result{}, fmt.Errorf("could not build Deployment for DataPlane %s: %w", dpNn, err)
	}
	if res != op.Noop {
	return ctrl.Result{}, nil
	}

because res is always different than op.Noop and reconciliation doesn't progress, hence code responsible for setting proper status is never reached