diff --git a/docs/concepts/ClusterResourcePlacement/README.md b/docs/concepts/ClusterResourcePlacement/README.md index 95716e70b..28f31b432 100644 --- a/docs/concepts/ClusterResourcePlacement/README.md +++ b/docs/concepts/ClusterResourcePlacement/README.md @@ -72,14 +72,22 @@ In addition, each time I update my workloads, the updates take place with zero d ![](placement-concept-overview.jpg) +The placement controller will create `ClusterSchedulingPolicySnapshot` and `ClusterResourceSnapshot` snapshots by watching +the `ClusterResourcePlacement` object. So that it can trigger the scheduling and resource rollout process whenever needed. + +The override controller will create the corresponding snapshots by watching the `ClusterResourceOverride` and `ResourceOverride` +which captures the snapshot of the overrides. + The placement workflow will be divided into several stages: -* Creating `ClusterSchedulingPolicySnapshot` and `ClusterResourceSnapshot` snapshots: clusterResourcePlacement controller -captures the selected resources and placement policy specified in the `ClusterResourcePlacement`. -* Creating `ClusterResourceBinding`: multi-cluster scheduler creates the `clusterResourceBinding` based on the scheduling -decisions for each target cluster and rollout controller binds selected resources according to the rollout strategy. -* Creating `Work` on the member cluster reserved namespace of the hub cluster: work generator creates the work on the -corresponding member cluster namespace. Each work contains the manifest workload to be deployed on the member clusters. -* Applying `Work` on the member clusters: apply work controller applies the manifest workload on the member clusters. +1. Scheduling: multi-cluster scheduler makes the schedule decision by creating the `clusterResourceBinding` for a bundle +of resources based on the latest `ClusterSchedulingPolicySnapshot`generated by the `ClusterResourcePlacement`. +2. Rolling out resources: rollout controller applies the resources to the selected member clusters based on the rollout strategy. +3. Overriding: work generator applies the override rules defined by `ClusterResourceOverride` and `ResourceOverride` to +the selected resources on the target clusters. +4. Creating or updating works: work generator creates the work on the corresponding member cluster namespace. Each work +contains the (overridden) manifest workload to be deployed on the member clusters. +5. Applying resources on target clusters: apply work controller applies the manifest workload on the member clusters. +6. Checking resource availability: apply work controller checks the resource availability on the target clusters ## Resource Selection @@ -247,96 +255,195 @@ Name: crp-1 Namespace: Labels: Annotations: -API Version: placement.kubernetes-fleet.io/v1beta1 +API Version: placement.kubernetes-fleet.io/v1 Kind: ClusterResourcePlacement Metadata: ... Spec: Policy: - Number Of Clusters: 2 - Placement Type: PickN + Placement Type: PickAll Resource Selectors: Group: Kind: Namespace - Name: test + Name: application-1 Version: v1 Revision History Limit: 10 + Strategy: + Rolling Update: + Max Surge: 25% + Max Unavailable: 25% + Unavailable Period Seconds: 2 + Type: RollingUpdate Status: Conditions: - Last Transition Time: 2023-11-10T08:14:52Z - Message: found all the clusters needed as specified by the scheduling policy - Observed Generation: 5 - Reason: SchedulingPolicyFulfilled - Status: True - Type: ClusterResourcePlacementScheduled - Last Transition Time: 2023-11-10T08:23:43Z - Message: All 2 cluster(s) are synchronized to the latest resources on the hub cluster - Observed Generation: 5 - Reason: SynchronizeSucceeded - Status: True - Type: ClusterResourcePlacementSynchronized - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully applied resources to 2 member clusters - Observed Generation: 5 - Reason: ApplySucceeded - Status: True - Type: ClusterResourcePlacementApplied + Last Transition Time: 2024-04-29T09:58:20Z + Message: found all the clusters needed as specified by the scheduling policy + Observed Generation: 1 + Reason: SchedulingPolicyFulfilled + Status: True + Type: ClusterResourcePlacementScheduled + Last Transition Time: 2024-04-29T09:58:20Z + Message: All 3 cluster(s) start rolling out the latest resource + Observed Generation: 1 + Reason: RolloutStarted + Status: True + Type: ClusterResourcePlacementRolloutStarted + Last Transition Time: 2024-04-29T09:58:20Z + Message: No override rules are configured for the selected resources + Observed Generation: 1 + Reason: NoOverrideSpecified + Status: True + Type: ClusterResourcePlacementOverridden + Last Transition Time: 2024-04-29T09:58:20Z + Message: Works(s) are succcesfully created or updated in the 3 target clusters' namespaces + Observed Generation: 1 + Reason: WorkSynchronized + Status: True + Type: ClusterResourcePlacementWorkSynchronized + Last Transition Time: 2024-04-29T09:58:20Z + Message: The selected resources are successfully applied to 3 clusters + Observed Generation: 1 + Reason: ApplySucceeded + Status: True + Type: ClusterResourcePlacementApplied + Last Transition Time: 2024-04-29T09:58:20Z + Message: The selected resources in 3 cluster are available now + Observed Generation: 1 + Reason: ResourceAvailable + Status: True + Type: ClusterResourcePlacementAvailable + Observed Resource Index: 0 Placement Statuses: - Cluster Name: aks-member-1 + Cluster Name: kind-cluster-1 Conditions: - Last Transition Time: 2023-11-10T08:14:52Z - Message: Successfully scheduled resources for placement in aks-member-1 (affinity score: 0, topology spread score: 0): picked by scheduling policy - Observed Generation: 5 - Reason: ScheduleSucceeded + Last Transition Time: 2024-04-29T09:58:20Z + Message: Successfully scheduled resources for placement in kind-cluster-1 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 1 + Reason: Scheduled + Status: True + Type: Scheduled + Last Transition Time: 2024-04-29T09:58:20Z + Message: Detected the new changes on the resources and started the rollout process + Observed Generation: 1 + Reason: RolloutStarted Status: True - Type: ResourceScheduled - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully Synchronized work(s) for placement - Observed Generation: 5 - Reason: WorkSynchronizeSucceeded + Type: RolloutStarted + Last Transition Time: 2024-04-29T09:58:20Z + Message: No override rules are configured for the selected resources + Observed Generation: 1 + Reason: NoOverrideSpecified + Status: True + Type: Overridden + Last Transition Time: 2024-04-29T09:58:20Z + Message: All of the works are synchronized to the latest + Observed Generation: 1 + Reason: AllWorkSynced Status: True Type: WorkSynchronized - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully applied resources - Observed Generation: 5 - Reason: ApplySucceeded + Last Transition Time: 2024-04-29T09:58:20Z + Message: All corresponding work objects are applied + Observed Generation: 1 + Reason: AllWorkHaveBeenApplied + Status: True + Type: Applied + Last Transition Time: 2024-04-29T09:58:20Z + Message: The availability of work object crp-1-work is not trackable + Observed Generation: 1 + Reason: WorkNotTrackable Status: True - Type: ResourceApplied - Cluster Name: aks-member-2 + Type: Available + Cluster Name: kind-cluster-2 Conditions: - Last Transition Time: 2023-11-10T08:14:52Z - Message: Successfully scheduled resources for placement in aks-member-2 (affinity score: 0, topology spread score: 0): picked by scheduling policy - Observed Generation: 5 - Reason: ScheduleSucceeded + Last Transition Time: 2024-04-29T09:58:20Z + Message: Successfully scheduled resources for placement in kind-cluster-2 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 1 + Reason: Scheduled Status: True - Type: ResourceScheduled - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully Synchronized work(s) for placement - Observed Generation: 5 - Reason: WorkSynchronizeSucceeded + Type: Scheduled + Last Transition Time: 2024-04-29T09:58:20Z + Message: Detected the new changes on the resources and started the rollout process + Observed Generation: 1 + Reason: RolloutStarted + Status: True + Type: RolloutStarted + Last Transition Time: 2024-04-29T09:58:20Z + Message: No override rules are configured for the selected resources + Observed Generation: 1 + Reason: NoOverrideSpecified + Status: True + Type: Overridden + Last Transition Time: 2024-04-29T09:58:20Z + Message: All of the works are synchronized to the latest + Observed Generation: 1 + Reason: AllWorkSynced Status: True Type: WorkSynchronized - Last Transition Time: 2023-11-10T08:23:43Z - Message: Successfully applied resources - Observed Generation: 5 - Reason: ApplySucceeded + Last Transition Time: 2024-04-29T09:58:20Z + Message: All corresponding work objects are applied + Observed Generation: 1 + Reason: AllWorkHaveBeenApplied Status: True - Type: ResourceApplied + Type: Applied + Last Transition Time: 2024-04-29T09:58:20Z + Message: The availability of work object crp-1-work is not trackable + Observed Generation: 1 + Reason: WorkNotTrackable + Status: True + Type: Available + Cluster Name: kind-cluster-3 + Conditions: + Last Transition Time: 2024-04-29T09:58:20Z + Message: Successfully scheduled resources for placement in kind-cluster-3 (affinity score: 0, topology spread score: 0): picked by scheduling policy + Observed Generation: 1 + Reason: Scheduled + Status: True + Type: Scheduled + Last Transition Time: 2024-04-29T09:58:20Z + Message: Detected the new changes on the resources and started the rollout process + Observed Generation: 1 + Reason: RolloutStarted + Status: True + Type: RolloutStarted + Last Transition Time: 2024-04-29T09:58:20Z + Message: No override rules are configured for the selected resources + Observed Generation: 1 + Reason: NoOverrideSpecified + Status: True + Type: Overridden + Last Transition Time: 2024-04-29T09:58:20Z + Message: All of the works are synchronized to the latest + Observed Generation: 1 + Reason: AllWorkSynced + Status: True + Type: WorkSynchronized + Last Transition Time: 2024-04-29T09:58:20Z + Message: All corresponding work objects are applied + Observed Generation: 1 + Reason: AllWorkHaveBeenApplied + Status: True + Type: Applied + Last Transition Time: 2024-04-29T09:58:20Z + Message: The availability of work object crp-1-work is not trackable + Observed Generation: 1 + Reason: WorkNotTrackable + Status: True + Type: Available Selected Resources: Kind: Namespace - Name: test + Name: application-1 Version: v1 Kind: ConfigMap - Name: test-1 - Namespace: test + Name: app-config-1 + Namespace: application-1 Version: v1 Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Normal PlacementScheduleSuccess 12m (x5 over 3d22h) cluster-resource-placement-controller Successfully scheduled the placement - Normal PlacementSyncSuccess 3m28s (x7 over 3d22h) cluster-resource-placement-controller Successfully synchronized the placement - Normal PlacementRolloutCompleted 3m28s (x7 over 3d22h) cluster-resource-placement-controller Resources have been applied to the selected clusters -``` + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal PlacementRolloutStarted 3m46s cluster-resource-placement-controller Started rolling out the latest resources + Normal PlacementOverriddenSucceeded 3m46s cluster-resource-placement-controller Placement has been successfully overridden + Normal PlacementWorkSynchronized 3m46s cluster-resource-placement-controller Work(s) have been created or updated successfully for the selected cluster(s) + Normal PlacementApplied 3m46s cluster-resource-placement-controller Resources have been applied to the selected cluster(s) + Normal PlacementRolloutCompleted 3m46s cluster-resource-placement-controller Resources are available in the selected clusters``` ## Tolerations diff --git a/docs/concepts/ClusterResourcePlacement/placement-concept-overview.jpg b/docs/concepts/ClusterResourcePlacement/placement-concept-overview.jpg index 368f914a3..22afc90b6 100644 Binary files a/docs/concepts/ClusterResourcePlacement/placement-concept-overview.jpg and b/docs/concepts/ClusterResourcePlacement/placement-concept-overview.jpg differ