[RayCluster][Feature] setup GCS FT annotations and the RAY_REDIS_ADDRESS env by the GcsFaultToleranceOptions #2721
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rueian
commented
Jan 9, 2025
| @@ -96,6 +96,9 @@ func initTemplateAnnotations(instance rayv1.RayCluster, podTemplate *corev1.PodT | |||
| podTemplate.Annotations[utils.RayExternalStorageNSAnnotationKey] = v | |||
| } | |||
| } | |||
| if options := instance.Spec.GcsFaultToleranceOptions; options != nil && options.ExternalStorageNamespace != "" { | |||
| podTemplate.Annotations[utils.RayExternalStorageNSAnnotationKey] = options.ExternalStorageNamespace | |||
There was a problem hiding this comment.
Override the ray.io/external-storage-namespace annotation if the new ExternalStorageNamespace is specified. We rely on this annotation to set the RAY_external_storage_namespace env in the later setContainerEnvVars function.
| @@ -322,7 +325,7 @@ func initLivenessAndReadinessProbe(rayContainer *corev1.Container, rayNodeType r | |||
| } | |||
|
|
|||
| // BuildPod a pod config | |||
| func BuildPod(ctx context.Context, podTemplateSpec corev1.PodTemplateSpec, rayNodeType rayv1.RayNodeType, rayStartParams map[string]string, headPort string, enableRayAutoscaler *bool, creatorCRDType utils.CRDType, fqdnRayIP string) (aPod corev1.Pod) { | |||
| func BuildPod(ctx context.Context, podTemplateSpec corev1.PodTemplateSpec, rayNodeType rayv1.RayNodeType, gcsOptions *rayv1.GcsFaultToleranceOptions, rayStartParams map[string]string, headPort string, enableRayAutoscaler *bool, creatorCRDType utils.CRDType, fqdnRayIP string) (aPod corev1.Pod) { | |||
There was a problem hiding this comment.
Just add a new gcsOptions parameter and pass it to the later setContainerEnvVars function.
Comment on lines
+639
to
+646
| if pod.Annotations[utils.RayFTEnabledAnnotationKey] != "true" { | ||
| t.Fatalf("Ray pod has unexpected %s annotation: %v", utils.RayFTEnabledAnnotationKey, pod.Annotations[utils.RayFTEnabledAnnotationKey]) | ||
| } | ||
| if pod.Annotations[utils.RayExternalStorageNSAnnotationKey] != string(cluster.UID) { | ||
| t.Fatalf("Ray pod has unexpected %s annotation: %v", utils.RayExternalStorageNSAnnotationKey, pod.Annotations[utils.RayExternalStorageNSAnnotationKey]) | ||
| } | ||
| checkContainerEnv(t, rayContainer, utils.RAY_REDIS_ADDRESS, "redis://127.0.0.1:6379") | ||
| checkContainerEnv(t, rayContainer, utils.RAY_EXTERNAL_STORAGE_NS, string(cluster.UID)) |
There was a problem hiding this comment.
Make sure both annotations and envs are set.
|
@MortalHappiness would you mind also reviewing this PR? |
…ESS env by the GcsFaultToleranceOptions Signed-off-by: Rueian <[email protected]>
rueian
force-pushed
the
gcs-ft-crd-annotations
branch
from
1a939dc to
fec9b86
Compare
2 tasks
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
This PR addresses the following selected part:
Related issue number
Resolves #2720
Checks