unexpected: first deployment is not for group 0

[erigo]

2024-09-01T03:43:24Z    INFO    controller.milvus       reconcileAll    {"err": "reconcile milvus: run group failed: groups error: reconcile milvus: run group failed: groups error: reconcile milvus deployments: groups error: handle create: get querynode deploys: unexpected: first deployment is not for group 0, admin please fix this by setting a last deployment for group 0"}
2024-09-01T03:43:24Z    ERROR   Reconciler error        {"controller": "milvus", "controllerGroup": "milvus.io", "controllerKind": "Milvus", "Milvus": {"name":"snrobots-milvus","namespace":"default"}, "namespace": "default", "name": "snrobots-milvus", "reconcileID": "9d0efb3e-2c22-4b0c-8b64-c2c4af8ee052", "error": "reconcile milvus: run group failed: groups error: reconcile milvus: run group failed: groups error: reconcile milvus deployments: groups error: handle create: get querynode deploys: unexpected: first deployment is not for group 0, admin please fix this by setting a last deployment for group 0", "errorVerbose": "groups error: reconcile milvus: run group failed: groups error: reconcile milvus deployments: groups error: handle create: get querynode deploys: unexpected: first deployment is not for group 0, admin please fix this by setting a last deployment for group 0\ngithub.com/milvus-io/milvus-operator/pkg/controllers.(*Group).Wait\n\t/workspace/pkg/controllers/groups.go:88\ngithub.com/milvus-io/milvus-operator/pkg/controllers.ParallelGroupRunner.Run\n\t/workspace/pkg/controllers/group_runner.go:59\ngithub.com/milvus-io/milvus-operator/pkg/controllers.(*MilvusReconciler).ReconcileAll\n\t/workspace/pkg/controllers/milvus.go:43\ngithub.com/milvus-io/milvus-operator/pkg/controllers.(*MilvusReconciler).Reconcile\n\t/workspace/pkg/controllers/milvus_controller.go:181\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controlle
[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650\nrun group failed\ngithub.com/milvus-io/milvus-operator/pkg/controllers.ParallelGroupRunner.Run\n\t/workspace/pkg/controllers/group_runner.go:60\ngithub.com/milvus-io/milvus-operator/pkg/controllers.(*MilvusReconciler).ReconcileAll\n\t/workspace/pkg/controllers/milvus.go:43\ngithub.com/milvus-io/milvus-operator/pkg/controllers.(*MilvusReconciler).Reconcile\n\t/workspace/pkg/controllers/milvus_controller.go:181\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650\nreconcile milvus\ngithub.com/milvus-io/milvus-operator/pkg/controllers.(*MilvusReconciler).ReconcileAll\n\t/workspace/pkg/controllers/milvus.go:44\ngithub.com/milvus-io/milvus-operator/pkg/controllers.(*MilvusReconciler).Reconcile\n\t/workspace/pkg/controllers/milvus_controller.go:181\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNe
xtWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:235
2024-09-01T03:43:24Z    INFO    controller.milvus       SetDefault: {"spec":{"components":{"rollingMode":2}}}
2024-09-01T03:43:24Z    DEBUG   controller-runtime.webhook.webhooks     received request        {"webhook": "/mutate-milvus-io-v1beta1-milvus", "UID": "89ac4055-57ed-4a33-bf1c-6d8b757d3463", "kind": "milvus.io/v1beta1, Kind=Milvus", "resource": {"group":"milvus.io","version":"v1beta1","resource":"milvuses"}}
2024-09-01T03:43:24Z    DEBUG   controller-runtime.webhook.webhooks     wrote response  {"webhook": "/mutate-milvus-io-v1beta1-milvus", "code": 200, "reason": "", "UID": "89ac4055-57ed-4a33-bf1c-6d8b757d3463", "allowed": true}
2024-09-01T03:43:24Z    DEBUG   controller-runtime.webhook.webhooks     received request        {"webhook": "/validate-milvus-io-v1beta1-milvus", "UID": "0f544d45-caa1-45a2-a31d-2438c923e731", "kind": "milvus.io/v1beta1, Kind=Milvus", "resource": {"group":"milvus.io","version":"v1beta1","resource":"milvuses"}}
2024-09-01T03:43:24Z    DEBUG   controller-runtime.webhook.webhooks     wrote response  {"webhook": "/validate-milvus-io-v1beta1-milvus", "code": 200, "reason": "", "UID": "0f544d45-caa1-45a2-a31d-2438c923e731", "allowed": true}
2024-09-01T03:43:24Z    INFO    controller.helm getting history for release snrobots-milvus-etcd
2024-09-01T03:43:24Z    INFO    controller.helm getting history for release snrobots-milvus-pulsar
2024-09-01T03:43:24Z    INFO    controller.helm getting history for release snrobots-milvus-minio

For some unknown reason, when upgrading Milvus from version 2.3.x to 2.4.6, the original query-0 pod became 0/0, similar to the standalone deployment.
Today, when upgrading the operator to version v1.0.3, I encountered the above error, but there were no issues when rolling back to version v0.9.14.

admin please fix this by setting a last deployment for group 0 How to do this？

[haorenfsa]

Starting from version 0.9.4, milvus-operator will create two deployments for querynode for high-availability rolling upgrades, querynode-0 and querynode-1. Now this error means that one of the deployments does not exist. If it is deleted by mistake, it needs to be restored.

[erigo]

Starting from version 0.9.4, milvus-operator will create two deployments for querynode for high-availability rolling upgrades, querynode-0 and querynode-1. Now this error means that one of the deployments does not exist. If it is deleted by mistake, it needs to be restored.

Thanks~ 🙏
How to restore querynode-0?

[erigo]

OK，I "copy" querynode-1 deployment, and start a new deployment named "querynode-0".
This issue has been solved.
Thanks again! 🙏

[erigo]

milvus-operator_milvus-operator-6dc67b756c-gzs8t_manager.log

[erigo]

more info:

kubectl get milvus -o yaml
apiVersion: v1
items:
- apiVersion: milvus.io/v1beta1
  kind: Milvus
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"milvus.io/v1beta1","kind":"Milvus","metadata":{"annotations":{},"labels":{"app":"milvus"},"name":"snrobots-milvus","namespace":"default"},"spec":{"components":{"dataCoord":{"replicas":0},"dataNode":{"replicas":3},"enableRollingUpdate":true,"image":"registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6","imageUpdateMode":"rollingUpgrade","indexCoord":{"replicas":0},"indexNode":{"replicas":3},"mixCoord":{"replicas":3},"proxy":{"replicas":3,"resources":{"limits":{"cpu":"2","memory":"4Gi"},"requests":{"cpu":"100m","memory":"128Mi"}},"serviceType":"LoadBalancer"},"queryCoord":{"replicas":0},"queryNode":{"replicas":3},"resources":{"limits":{"cpu":"4","memory":"8Gi"},"requests":{"cpu":"200m","memory":"512Mi"}},"rootCoord":{"replicas":0}},"config":{},"dependencies":{"etcd":{"inCluster":{"deletionPolicy":"Retain","pvcDeletion":false}},"msgStreamType":"pulsar","pulsar":{"inCluster":{"deletionPolicy":"Retain","pvcDeletion":false}},"storage":{"inCluster":{"deletionPolicy":"Retain","pvcDeletion":false}}},"mode":"cluster"}}
      milvus.io/current-querynode-group-id: ""
      milvus.io/dependency-values-merged: "true"
      milvus.io/pod-service-label-added: "true"
      milvus.io/querynode-current-group-id: "1"
    creationTimestamp: "2024-05-16T12:58:44Z"
    finalizers:
    - milvus.milvus.io/finalizer
    generation: 38387
    labels:
      app: milvus
      milvus.io/operator-version: 0.9.14
    name: snrobots-milvus
    namespace: default
    resourceVersion: "52210693"
    uid: f1ac8fce-1959-4dac-8b30-d455730bbefc
  spec:
    components:
      dataCoord:
        paused: false
        replicas: 0
      dataNode:
        paused: false
        replicas: 3
      disableMetric: false
      enableRollingUpdate: true
      image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
      imageUpdateMode: rollingUpgrade
      indexCoord:
        paused: false
        replicas: 0
      indexNode:
        paused: false
        replicas: 3
      metricInterval: ""
      mixCoord:
        paused: false
        replicas: 3
      paused: false
      proxy:
        paused: false
        replicas: 3
        resources:
          limits:
            cpu: "2"
            memory: 4Gi
          requests:
            cpu: 100m
            memory: 128Mi
        serviceType: LoadBalancer
      queryCoord:
        paused: false
        replicas: 0
      queryNode:
        paused: false
        replicas: 3
      resources:
        limits:
          cpu: "4"
          memory: 8Gi
        requests:
          cpu: 200m
          memory: 512Mi
      rootCoord:
        paused: false
        replicas: 0
      standalone:
        paused: false
        replicas: 0
        serviceType: ClusterIP
    config:
      dataCoord:
        enableActiveStandby: true
      indexCoord:
        enableActiveStandby: true
      queryCoord:
        enableActiveStandby: true
      rootCoord:
        enableActiveStandby: true
    dependencies:
      customMsgStream: null
      etcd:
        endpoints:
        - snrobots-milvus-etcd.default:2379
        external: false
        inCluster:
          deletionPolicy: Retain
          values:
            auth:
              rbac:
                enabled: false
            autoCompactionMode: revision
            autoCompactionRetention: "1000"
            enabled: true
            extraEnvVars:
            - name: ETCD_QUOTA_BACKEND_BYTES
              value: "4294967296"
            - name: ETCD_HEARTBEAT_INTERVAL
              value: "500"
            - name: ETCD_ELECTION_TIMEOUT
              value: "2500"
            image:
              pullPolicy: IfNotPresent
              repository: milvusdb/etcd
              tag: 3.5.5-r4
            livenessProbe:
              enabled: true
              timeoutSeconds: 10
            name: etcd
            pdb:
              create: false
            persistence:
              accessMode: ReadWriteOnce
              enabled: true
              size: 10Gi
              storageClass: null
            readinessProbe:
              enabled: true
              periodSeconds: 20
              timeoutSeconds: 10
            replicaCount: 3
            service:
              peerPort: 2380
              port: 2379
              type: ClusterIP
      kafka:
        external: false
      msgStreamType: pulsar
      natsmq:
        persistence:
          persistentVolumeClaim:
            spec: null
      pulsar:
        endpoint: snrobots-milvus-pulsar-proxy.default:6650
        external: false
        inCluster:
          deletionPolicy: Retain
          values:
            affinity:
              anti_affinity: false
            autorecovery:
              resources:
                requests:
                  cpu: 1
                  memory: 512Mi
            bookkeeper:
              configData:
                PULSAR_GC: |
                  -Dio.netty.leakDetectionLevel=disabled -Dio.netty.recycler.linkCapacity=1024 -XX:+UseG1GC -XX:MaxGCPauseMillis=10 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+DoEscapeAnalysis -XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50 -XX:+DisableExplicitGC -XX:-ResizePLAB -XX:+ExitOnOutOfMemoryError -XX:+PerfDisableSharedMem -XX:+PrintGCDetails
                PULSAR_MEM: |
                  -Xms4096m -Xmx4096m -XX:MaxDirectMemorySize=8192m
                nettyMaxFrameSizeBytes: "104867840"
              pdb:
                usePolicy: false
              replicaCount: 3
              resources:
                requests:
                  cpu: 1
                  memory: 2048Mi
              volumes:
                journal:
                  name: journal
                  size: 100Gi
                ledgers:
                  name: ledgers
                  size: 200Gi
            broker:
              component: broker
              configData:
                PULSAR_GC: |
                  -Dio.netty.leakDetectionLevel=disabled -Dio.netty.recycler.linkCapacity=1024 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+DoEscapeAnalysis -XX:ParallelGCThreads=32 -XX:ConcGCThreads=32 -XX:G1NewSizePercent=50 -XX:+DisableExplicitGC -XX:-ResizePLAB -XX:+ExitOnOutOfMemoryError
                PULSAR_MEM: |
                  -Xms4096m -Xmx4096m -XX:MaxDirectMemorySize=8192m
                backlogQuotaDefaultLimitGB: "8"
                backlogQuotaDefaultRetentionPolicy: producer_exception
                defaultRetentionSizeInMB: "-1"
                defaultRetentionTimeInMinutes: "10080"
                maxMessageSize: "104857600"
                subscriptionExpirationTimeMinutes: "3"
                ttlDurationDefaultInSeconds: "259200"
              pdb:
                usePolicy: false
              podMonitor:
                enabled: false
              replicaCount: 1
              resources:
                requests:
                  cpu: 1.5
                  memory: 4096Mi
            components:
              autorecovery: true
              bookkeeper: true
              broker: true
              functions: false
              proxy: true
              pulsar_manager: false
              toolset: false
              zookeeper: true
            enabled: true
            fullnameOverride: ""
            images:
              autorecovery:
                pullPolicy: IfNotPresent
                repository: apachepulsar/pulsar
                tag: 2.8.2
              bookie:
                pullPolicy: IfNotPresent
                repository: apachepulsar/pulsar
                tag: 2.8.2
              broker:
                pullPolicy: IfNotPresent
                repository: apachepulsar/pulsar
                tag: 2.8.2
              proxy:
                pullPolicy: IfNotPresent
                repository: apachepulsar/pulsar
                tag: 2.8.2
              pulsar_manager:
                pullPolicy: IfNotPresent
                repository: apachepulsar/pulsar-manager
                tag: v0.1.0
              zookeeper:
                pullPolicy: IfNotPresent
                repository: apachepulsar/pulsar
                tag: 2.8.2
            maxMessageSize: "5242880"
            monitoring:
              alert_manager: false
              grafana: false
              node_exporter: false
              prometheus: false
            name: pulsar
            persistence: true
            proxy:
              configData:
                PULSAR_GC: |
                  -XX:MaxDirectMemorySize=2048m
                PULSAR_MEM: |
                  -Xms2048m -Xmx2048m
                httpNumThreads: "100"
              pdb:
                usePolicy: false
              podMonitor:
                enabled: false
              ports:
                pulsar: 6650
              replicaCount: 1
              resources:
                requests:
                  cpu: 1
                  memory: 2048Mi
              service:
                type: ClusterIP
            pulsar_manager:
              service:
                type: ClusterIP
            pulsar_metadata:
              component: pulsar-init
              image:
                repository: apachepulsar/pulsar
                tag: 2.8.2
            rbac:
              enabled: false
              limit_to_namespace: true
              psp: false
            zookeeper:
              configData:
                PULSAR_GC: |
                  -Dcom.sun.management.jmxremote -Djute.maxbuffer=10485760 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+DoEscapeAnalysis -XX:+DisableExplicitGC -XX:+PerfDisableSharedMem -Dzookeeper.forceSync=no
                PULSAR_MEM: |
                  -Xms1024m -Xmx1024m
              pdb:
                usePolicy: false
              resources:
                requests:
                  cpu: 0.3
                  memory: 1024Mi
      rocksmq:
        persistence:
          persistentVolumeClaim:
            spec: null
      storage:
        endpoint: snrobots-milvus-minio.default:9000
        external: false
        inCluster:
          deletionPolicy: Retain
          values:
            accessKey: minioadmin
            bucketName: milvus-bucket
            enabled: true
            existingSecret: ""
            iamEndpoint: ""
            image:
              pullPolicy: IfNotPresent
              tag: RELEASE.2023-03-20T20-16-18Z
            livenessProbe:
              enabled: true
              failureThreshold: 5
              initialDelaySeconds: 5
              periodSeconds: 5
              successThreshold: 1
              timeoutSeconds: 5
            mode: distributed
            name: minio
            persistence:
              accessMode: ReadWriteOnce
              enabled: true
              existingClaim: ""
              size: 500Gi
              storageClass: null
            podDisruptionBudget:
              enabled: false
            readinessProbe:
              enabled: true
              failureThreshold: 5
              initialDelaySeconds: 5
              periodSeconds: 5
              successThreshold: 1
              timeoutSeconds: 1
            region: ""
            resources:
              requests:
                memory: 2Gi
            rootPath: file
            secretKey: minioadmin
            service:
              port: 9000
              type: ClusterIP
            startupProbe:
              enabled: true
              failureThreshold: 60
              initialDelaySeconds: 0
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 5
            useIAM: false
            useVirtualHost: false
        secretRef: snrobots-milvus-minio
        type: MinIO
    hookConfig: null
    mode: cluster
  status:
    componentsDeployStatus:
      datacoord:
        generation: 9
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 1
          conditions:
          - lastTransitionTime: "2024-08-30T09:29:32Z"
            lastUpdateTime: "2024-08-30T09:29:32Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          - lastTransitionTime: "2024-05-16T13:02:09Z"
            lastUpdateTime: "2024-09-02T06:07:00Z"
            message: ReplicaSet "snrobots-milvus-milvus-datacoord-686bfc4478" has
              successfully progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          observedGeneration: 9
          readyReplicas: 1
          replicas: 1
          updatedReplicas: 1
      datanode:
        generation: 11
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 3
          conditions:
          - lastTransitionTime: "2024-05-16T13:02:09Z"
            lastUpdateTime: "2024-09-02T06:07:29Z"
            message: ReplicaSet "snrobots-milvus-milvus-datanode-787fd879c" has successfully
              progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          - lastTransitionTime: "2024-09-02T07:18:16Z"
            lastUpdateTime: "2024-09-02T07:18:16Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          observedGeneration: 11
          readyReplicas: 3
          replicas: 3
          updatedReplicas: 3
      indexcoord:
        generation: 7
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 1
          conditions:
          - lastTransitionTime: "2024-08-29T11:02:41Z"
            lastUpdateTime: "2024-08-29T11:02:41Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          - lastTransitionTime: "2024-05-16T13:02:09Z"
            lastUpdateTime: "2024-09-02T06:06:00Z"
            message: ReplicaSet "snrobots-milvus-milvus-indexcoord-5df688b47b" has
              successfully progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          observedGeneration: 7
          readyReplicas: 1
          replicas: 1
          updatedReplicas: 1
      indexnode:
        generation: 9
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 3
          conditions:
          - lastTransitionTime: "2024-05-16T13:02:09Z"
            lastUpdateTime: "2024-09-02T06:06:40Z"
            message: ReplicaSet "snrobots-milvus-milvus-indexnode-7b5b9f75fc" has
              successfully progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          - lastTransitionTime: "2024-09-02T07:18:15Z"
            lastUpdateTime: "2024-09-02T07:18:15Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          observedGeneration: 9
          readyReplicas: 3
          replicas: 3
          updatedReplicas: 3
      mixcoord:
        generation: 4
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 3
          conditions:
          - lastTransitionTime: "2024-09-01T03:59:47Z"
            lastUpdateTime: "2024-09-02T06:37:32Z"
            message: ReplicaSet "snrobots-milvus-milvus-mixcoord-7d5f7df748" has successfully
              progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          - lastTransitionTime: "2024-09-02T07:18:15Z"
            lastUpdateTime: "2024-09-02T07:18:15Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          observedGeneration: 4
          readyReplicas: 3
          replicas: 3
          updatedReplicas: 3
      proxy:
        generation: 10
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 3
          conditions:
          - lastTransitionTime: "2024-05-16T13:02:09Z"
            lastUpdateTime: "2024-09-02T06:05:20Z"
            message: ReplicaSet "snrobots-milvus-milvus-proxy-65f68b5f99" has successfully
              progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          - lastTransitionTime: "2024-09-02T07:18:15Z"
            lastUpdateTime: "2024-09-02T07:18:15Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          observedGeneration: 10
          readyReplicas: 3
          replicas: 3
          updatedReplicas: 3
      querycoord:
        generation: 7
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 1
          conditions:
          - lastTransitionTime: "2024-08-30T09:29:54Z"
            lastUpdateTime: "2024-08-30T09:29:54Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          - lastTransitionTime: "2024-05-16T13:02:09Z"
            lastUpdateTime: "2024-09-02T06:05:40Z"
            message: ReplicaSet "snrobots-milvus-milvus-querycoord-5d9fbfd859" has
              successfully progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          observedGeneration: 7
          readyReplicas: 1
          replicas: 1
          updatedReplicas: 1
      querynode:
        generation: 6
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 3
          conditions:
          - lastTransitionTime: "2024-08-29T10:12:59Z"
            lastUpdateTime: "2024-08-29T15:11:06Z"
            message: ReplicaSet "snrobots-milvus-milvus-querynode-1-799766b89c" has
              successfully progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          - lastTransitionTime: "2024-08-30T09:29:56Z"
            lastUpdateTime: "2024-08-30T09:29:56Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          observedGeneration: 6
          readyReplicas: 3
          replicas: 3
          updatedReplicas: 3
      rootcoord:
        generation: 8
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          availableReplicas: 1
          conditions:
          - lastTransitionTime: "2024-08-30T09:29:33Z"
            lastUpdateTime: "2024-08-30T09:29:33Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          - lastTransitionTime: "2024-05-16T13:02:09Z"
            lastUpdateTime: "2024-09-02T06:08:10Z"
            message: ReplicaSet "snrobots-milvus-milvus-rootcoord-d5dc87789" has successfully
              progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          observedGeneration: 8
          readyReplicas: 1
          replicas: 1
          updatedReplicas: 1
      standalone:
        generation: 2
        image: registry.cn-hangzhou.aliyuncs.com/zilliztech/milvus:v2.4.6
        status:
          conditions:
          - lastTransitionTime: "2024-09-02T07:26:34Z"
            lastUpdateTime: "2024-09-02T07:26:34Z"
            message: Deployment has minimum availability.
            reason: MinimumReplicasAvailable
            status: "True"
            type: Available
          - lastTransitionTime: "2024-09-02T07:26:34Z"
            lastUpdateTime: "2024-09-02T07:26:34Z"
            message: ReplicaSet "snrobots-milvus-milvus-standalone-54999fbf7f" has
              successfully progressed.
            reason: NewReplicaSetAvailable
            status: "True"
            type: Progressing
          observedGeneration: 2
    conditions:
    - lastTransitionTime: "2024-08-30T09:28:44Z"
      message: Etcd endpoints is healthy
      reason: EtcdReady
      status: "True"
      type: EtcdReady
    - lastTransitionTime: "2024-09-02T05:58:59Z"
      reason: StorageReady
      status: "True"
      type: StorageReady
    - lastTransitionTime: "2024-09-02T05:59:37Z"
      message: MsgStream is ready
      reason: MsgStreamReady
      status: "True"
      type: MsgStreamReady
    - lastTransitionTime: "2024-09-02T07:26:34Z"
      message: '[querynode] not ready, detail: component[querynode]: deployment not
        created'
      reason: MilvusComponentNotHealthy
      status: "False"
      type: MilvusReady
    - lastTransitionTime: "2024-09-02T07:26:34Z"
      message: Milvus components are all updated
      reason: MilvusComponentsUpdated
      status: "True"
      type: MilvusUpdated
    ingress:
      loadBalancer: {}
    observedGeneration: 38386
    replicas:
      dataCoord: 1
      dataNode: 3
      indexCoord: 1
      indexNode: 3
      mixCoord: 3
      proxy: 3
      queryCoord: 1
      queryNode: 3
      rootCoord: 1
    status: Unhealthy
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

[haorenfsa]

OK，I "copy" querynode-1 deployment, and start a new deployment named "querynode-0". This issue has been solved. Thanks again! 🙏

Here are the fields to change when restoring querynode-0 deployment:

[erigo]

Thanks~