Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADD: KubeEdge Image PrePull Feature Guide Document #552

Open
wants to merge 56 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
c96e24a
kubeedge部署文档内容优化:边缘节点资源紧张无法正常部署的问题处理
hyp4293 May 22, 2024
571e55c
Optimization of kubeedge deployment documentation: Handling issues wi…
hyp4293 May 23, 2024
a294732
镜像预加载功能指导文档
hyp4293 May 24, 2024
b2f2d09
Merge remote-tracking branch 'upstream/master'
hyp4293 May 25, 2024
2b3a6ef
KubeEdge Image PrePull Feature Guide Document
hyp4293 May 25, 2024
9440c27
KubeEdge Image PrePull Feature Guide Document
hyp4293 Jun 12, 2024
fbe1744
6/7/2024
hyp4293 Jul 6, 2024
6af55cd
Merge branch 'master' into IamgePrePullDoc
hyp4293 Jul 6, 2024
fef325e
update version for release 1.17
fisherxu May 20, 2024
d75689d
New Blog for Release KubeEdge v1.11
1Shubham7 May 15, 2024
214001a
fixed the release date
1Shubham7 May 16, 2024
b512169
New Blog for Release KubeEdge v1.11
1Shubham7 May 16, 2024
cd9db44
New Blog for Release KubeEdge v1.14
1Shubham7 May 16, 2024
755bcee
added the release date
1Shubham7 May 16, 2024
11c7d68
update version for release 1.17
fisherxu May 20, 2024
18d7615
kubeedge部署文档内容优化:边缘节点资源紧张无法正常部署的问题处理
hyp4293 May 22, 2024
345a540
Optimization of kubeedge deployment documentation: Handling issues wi…
hyp4293 May 23, 2024
4af8dd1
镜像预加载功能指导文档
hyp4293 May 24, 2024
1712f6a
Replacing Twitter with X
1Shubham7 May 17, 2024
603928a
Merge branch 'master' of https://github.com/hyp4293/website
hyp4293 Jul 8, 2024
7b143df
kubeedge部署文档内容优化:边缘节点资源紧张无法正常部署的问题处理
hyp4293 May 22, 2024
65c99f1
Optimization of kubeedge deployment documentation: Handling issues wi…
hyp4293 May 23, 2024
36f7033
镜像预加载功能指导文档
hyp4293 May 24, 2024
c71c1f7
Replacing Twitter with X
1Shubham7 May 17, 2024
2c6a16f
KubeEdge Image PrePull Feature Guide Document
hyp4293 May 25, 2024
2b2cc16
KubeEdge Image PrePull Feature Guide Document
hyp4293 Jun 12, 2024
942bcb9
6/7/2024
hyp4293 Jul 6, 2024
d5fba0a
DCO
hyp4293 Jul 6, 2024
137dcb3
fix merge
hyp4293 Jul 10, 2024
d59f6bc
KubeEdge Image PrePull Feature Guide Document
hyp4293 May 25, 2024
52ac1e3
KubeEdge Image PrePull Feature Guide Document
hyp4293 Jun 12, 2024
bf2a75e
6/7/2024
hyp4293 Jul 6, 2024
3bb5a84
add case study about raisecom tech
wbc6080 May 27, 2024
0c2e057
add case study about XingHai IoT
wbc6080 May 27, 2024
a5f98d9
Part 1: improving the install with keadm docs
1Shubham7 May 17, 2024
628e9d7
fix merge
hyp4293 Jul 10, 2024
9230122
fixed suggestions from reviewers
1Shubham7 May 22, 2024
c9f46a3
fixed suggestions from reviewers (removing extra files)
1Shubham7 May 22, 2024
02038ea
edgecode -> EdgeCore
1Shubham7 May 22, 2024
6d5812b
small fix
1Shubham7 May 22, 2024
35ebb0c
adding suggestions from Shelley-BaoYue
1Shubham7 May 25, 2024
e9a3ba1
Add LookCan to community
CadenTango Jun 3, 2024
36273a2
add case study about JingYing
wbc6080 May 28, 2024
488a7eb
fixing links to docs and base
1Shubham7 Jun 3, 2024
110963d
modify blog time
Shelley-BaoYue Jun 6, 2024
4ab6e12
Odd text color fix
iArchitSharma Jun 20, 2024
5d8d032
add: added blog for release v1.15
1Shubham7 Jun 18, 2024
d9fd771
fixing dates
1Shubham7 Jun 20, 2024
8f426f0
PR template update
iArchitSharma Jun 21, 2024
9859594
Update PULL_REQUEST_TEMPLATE.md
iArchitSharma Jun 24, 2024
7f0c4c8
add inclusterconfig related doc
Shelley-BaoYue Jun 27, 2024
5a541e8
add blog to explain how to enable Cilium with KubeEdge.
fujitatomoya Jun 5, 2024
93861f0
modify authors.yml in zh locale.
fujitatomoya Jun 6, 2024
8804e73
some cosmetic fixes.
fujitatomoya Jun 6, 2024
074cc97
some commands are not appropriate with procedure.
fujitatomoya Jun 18, 2024
e1eacf6
merge fix
hyp4293 Jul 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions docs/advanced/image-prepull.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
---
title: KubeEdge Image PrePull Feature Guide Document
sidebar_position: 6
---


# KubeEdge Image PrePull Feature Guide Document

KubeEdge version 1.16 introduces a new feature called Image Pre-Pull, which allows users to load images ahead of time on edge nodes through the Kubernetes API of ImagePrePullJob. This feature supports pre-pull multiple images in batches across multiple edge nodes or node groups, helping to reduce the failure rates and inefficiencies associated with loading images during application deployment or updates, especially in large-scale scenarios.

API example for ImagePrePullJob:

```
apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
name: imageprepull-example
labels:
description: ImagePrePullLabel
spec:
imagePrePullTemplate:
images:
- image1
- image2
nodes:
- edgenode1
- edgenode2
checkItems:
- "disk"
failureTolerate: "0.3"
concurrency: 2
timeoutSeconds: 180
retryTimes: 1

```


## 1. Preparation

**Example:Nginx Demo**

Nginx is a lightweight image that allows users to demonstrate it without any prerequisite environment. Nginx image will be uploaded to a private image repository in advance. Users can call the pre-pull function API from the cloud to pre-pull the Nginx image to edge nodes from the private image repository.

**1)This example requires KubeEdge version to be v1.16.0 or above, and Kubernetes version to be v1.27.0 or above. The selected version is KubeEdge v1.16.0 and Kubernetes version is v1.27.3.**

```
[root@ke-cloud ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
cloud.kubeedge Ready control-plane,master 3d v1.27.3
edge.kubeedge Ready agent,edge 2d v1.27.7-kubeedge-v1.16.0

Note: The following operations will use the edge node edge.kubeedge. If you refer to this document for related operations, the configuration of the edge node name in subsequent steps needs to be changed according to your actual situation.
```

**2)Ensure that the CloudCore has the following configuration enabled**


```
taskManager:
enable: true // Change from false to true
```
changes can be made by editing the file kubectl edit configmap cloudcore -n kubeedge with commands, and restarting the cloudcore component of the K8s apiserver.




## 2. Prepare the Secret for the privare image (optional)
Here is a private image repository prepared for demonstration purposes using Alibaba Cloud's registry URL: registry.cn-hangzhou.aliyuncs.com. The demo space used is jilimoxing, and modifications may be necessary based on actual circumstances during the actual operation.

**1)Pushing nginx into the private image repository**

```
[root@cloud ~]# docker tag nginx registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
[root@cloud crds~]# docker push registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
```

**2)Create a Secret on the cloud**
Secret is not a required field in ImagePrePullJob. If you need to prepull private image, you can generate a secret for it.
You can also use kubectl to create a Secret for accessing a container registry,such as when you don`t have a Docker configuration file:

```
[root@cloud ~]# kubectl create secret docker-registry my-secret \
[email protected] \
--docker-username=tiger \
--docker-password=pass1234 \
--docker-email=my-registry.example:5000

[root@cloud ~]# kubectl get secret -A
NAMESPACE NAME TYPE DATA AGE
default my-secret kubernetes.io/dockerconfigjson 1 31s

```

## 3. Create Yaml File

**1)Modify Code**

To create a yaml file on a cloud node, you need to modify the corresponding images information and imageSecrets information to keep them consistent with the pre-pull image repository secret. The information should be as follows:
```

[root@ke-cloud ~]# vim imageprepull.yaml

apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
name: imageprepull-example
spec:
imagePrePullTemplate:
concurrency: 1
failureTolerate: '0.1'
images:
- registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
nodeNames:
- edge.kubeedge
imageSecrets: default/my-secret
retryTimes: 1
timeoutSeconds: 120

```

**2)executable file**


```
[root@ke-cloud ~]# kubectl apply -f imageprepull.yaml
```


**3) Get ImagePrepulljob Status**

use:kubectl get imageprepulljobs.operations.kubeedge.io imageprepull-example -o jsonpath='{.status}'

```
[root@ke-cloud ~]# kubectl get imageprepulljobs.operations.kubeedge.io imageprepull-example -o jsonpath='{.status}'
[root@ke-cloud ~]# {"action":"Success","event":"Pull","state":"Successful","status":[{"imageStatus":[{"image":"registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx","state":"Successful"}],"nodeStatus":{"action":"Success","event":"Pull","nodeName":"edge.kubeedge","state":"Successful","time":"2024-04-26T18:51:41Z"}}],"time":"2024-04-26T18:51:41Z"}
```

## 4. Check if the edge node image has been pre-pull successfully

Enter the edge terminal and use the command ctr -n k8s.io i ls to view.
```
[root@edge ~]# ctr -n k8s.io i ls
```
The corresponding image has been successfully pre-pull.
```
REF TYPE DIGEST SIZE PLATFORMS LABELS
registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx application/vnd.docker.distribution.manifest.v2+json sha256:73e957703f1266530db0aeac1fd6a3f87c1e59943f4c13eb340bb8521c6041d7 67.3 MiB linux/amd64
```

12 changes: 11 additions & 1 deletion docs/setup/install-with-keadm.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,16 @@ keadm init --set server.advertiseAddress="THE-EXPOSED-IP" --set server.nodeName=
If you are familiar with the Helm chart installation, please refer to [KubeEdge Helm Charts](https://github.com/kubeedge/kubeedge/tree/master/manifests/charts).


**SPECIAL SCENARIO:**
In the case of insufficient qualifications for edge nodes, we need to label them to prevent some applications from extending to edge nodes. `Kube-proxy` and some others is not required at the edge.We can handle it accordingly.

```
kubectl get daemonset -n kube-system |grep -v NAME |awk '{print $1}' | xargs -n 1 kubectl patch daemonset -n kube-system --type='json' -p='[{"op": "replace","path": "/spec/template/spec/affinity","value":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-role.kubernetes.io/edge","operator":"DoesNotExist"}]}]}}}}]'

```

To handle kube-proxy, you can refer to the [two methods](#anchor-name) mentioned in the " Enable `kubectl logs` Feature " section of this document.

### keadm manifest generate

You can generate the manifests using `keadm manifest generate`.
Expand Down Expand Up @@ -373,7 +383,7 @@ Before deploying the metrics-server, the `kubectl logs` feature must be activate
systemctl restart edgecore.service
```

If restarting EdgeCore fails, check if that is due to `kube-proxy` and kill it. **kubeedge** rejects it by default, we use a succedaneum called [edgemesh](https://github.com/kubeedge/kubeedge/blob/master/docs/proposals/edgemesh-design.md)
**Note:** the importance is to avoid `kube-proxy` being deployed on edgenode.<a name="anchor-name"> There are two methods to solve it: </a >

**Note:** It is important to avoid `kube-proxy` being deployed on edgenode and there are two methods to achieve this:

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
---
title: KubeEdge 镜像预加载功能指导文档
sidebar_position: 6
---

# KubeEdge 镜像预加载功能指导文档

KubeEdge 1.16版本引入了镜像预下载新特性,用户可以通过ImagePrePullJob的Kubernetes API提前在边缘节点上加载镜像,该特性支持在批量边缘节点或节点组中预下载多个镜像,帮助减少加载镜像在应用部署或更新过程,尤其是大规模场景中,带来的失败率高、效率低下等问题。

镜像预下载API示例:

```
apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
name: imageprepull-example
labels:
description:ImagePrePullLabel
spec:
imagePrePullTemplate:
images:
- image1
- image2
nodes:
- edgenode1
- edgenode2
checkItems:
- "disk"
failureTolerate: "0.3"
concurrency: 2
timeoutSeconds: 180
retryTimes: 1

```


## 1. 准备工作

**选用示例:Nginx Demo**

nginx是一个轻量级镜像,用户无需任何环境即可进行此演示。nginx镜像将会提前上传到一个私有镜像仓库中。用户可以从云端调用预加载功能API,将私有镜像仓库中的nginx镜像,提前下发到边缘节点中。


**1)本示例要求KubeEdge版本必须是v1.16.0+,kubernetes版本是v1.27.0+,此次选择的版本是KubeEdge v1.16.0,Kubernetes版本是v1.27.3**

```
[root@ke-cloud ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
cloud.kubeedge Ready control-plane,master 3d v1.27.3
edge.kubeedge Ready agent,edge 2d v1.27.7-kubeedge-v1.16.0

说明:本文接下来的操作将使用边缘节点edge.kubeedge进行,如果你参考本文进行相关操作,后续边缘节点名称的配置需要根据你的实际情况进行更改。
```

**2)确保CloudCore开启了以下配置:**


```
taskManager:
enable: true // 由false修改为true
```
可以通过命令修改kubectl edit configmap cloudcore -n kubeedge文件,并重启k8s-apiserver组件的cloudcore来进行更改。






## 2. 为私有镜像准备密钥 (可选)
在这里准备了一个阿里云的私有镜像仓用作演示:registry.cn-hangzhou.aliyuncs.com/,使用的演示空间为jilimoxing。实际操作过程中可以依据真实情况进行修改

**1)推送nginx进入私有镜像仓**
```
[root@cloud ~]# docker tag nginx registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
[root@cloud crds~]# docker push registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
```

**2)在云端创建Secret**
Secret 不是 ImagePrePullJob 中的必填字段。如果你需要预拉私有镜像,你可以为它生成一个密钥。
您还可以使用kubectl创建一个用于访问docker registry的Secret,例如在没有Docker配置文件的情况下:

```
[root@cloud ~]# kubectl create secret docker-registry my-secret \
[email protected] \
--docker-username=tiger \
--docker-password=pass1234 \
--docker-email=my-registry.example:5000

[root@cloud ~]# kubectl get secret -A
NAMESPACE NAME TYPE DATA AGE
default my-secret kubernetes.io/dockerconfigjson 1 31s

```

## 3. 创建Yaml文件

**1)修改代码**

在云端节点上创建yaml文件,需要修改对应的images信息以及imageSecrets信息,保持和所需要预加载的镜像仓库secret一致,如下所示:
```

[root@ke-cloud ~]# vim imageprepull.yaml

apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
name: imageprepull-example
spec:
imagePrePullTemplate:
concurrency: 1
failureTolerate: '0.1'
images:
- registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx
nodeNames:
- edge.kubeedge
imageSecrets: default/my-secret
retryTimes: 1
timeoutSeconds: 120

```

**2)执行文件**


```
[root@ke-cloud ~]# kubectl apply -f imageprepull.yaml
```

**3) 获取 ImagePrepulljob 的状态**

使用命令:kubectl get imageprepulljobs.operations.kubeedge.io imageprepull-example -o jsonpath='{.status}'进行查看

```
[root@ke-cloud ~]# kubectl get imageprepulljobs.operations.kubeedge.io imageprepull-example -o jsonpath='{.status}'
[root@ke-cloud ~]# {"action":"Success","event":"Pull","state":"Successful","status":[{"imageStatus":[{"image":"registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx","state":"Successful"}],"nodeStatus":{"action":"Success","event":"Pull","nodeName":"edge.kubeedge","state":"Successful","time":"2024-04-26T18:51:41Z"}}],"time":"2024-04-26T18:51:41Z"}
```


## 4. 检查边缘节点镜像是否预加载成功

进入边缘端,使用命令ctr -n k8s.io i ls进行查看
```
[root@edge ~]# ctr -n k8s.io i ls
```
找到对应的镜像已被预加载成功
```
REF TYPE DIGEST SIZE PLATFORMS LABELS
registry.cn-hangzhou.aliyuncs.com/jilimoxing/test:nginx application/vnd.docker.distribution.manifest.v2+json sha256:73e957703f1266530db0aeac1fd6a3f87c1e59943f4c13eb340bb8521c6041d7 67.3 MiB linux/amd64
```
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,17 @@ KubeEdge cloudcore is running, For logs visit: /var/log/kubeedge/cloudcore.log

当您看到以上信息,说明 KubeEdge 的云端组件 cloudcore 已经成功运行。

**特殊场景:**
边缘计算的硬件条件不好的情况,这里我们需要打上标签,让一些应用不扩展到edge节点上去。 kube-proxy和其他的一些应用不是必须部署在边缘端,所以我们可以对他们进行处理。


```
kubectl get daemonset -n kube-system |grep -v NAME |awk '{print $1}' | xargs -n 1 kubectl patch daemonset -n kube-system --type='json' -p='[{"op": "replace","path": "/spec/template/spec/affinity","value":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"node-role.kubernetes.io/edge","operator":"DoesNotExist"}]}]}}}}]'

```

如何处理kube-proxy,可以参考本文中在'启用 kubectl logs 功能'部分提到的 [2种方法](#anchor-name)

### keadm beta init

如果您想要使用容器化方式部署云端组件 cloudcore ,您可以使用 `keadm beta init` 进行云端组件安装。
Expand Down Expand Up @@ -287,7 +298,7 @@ KubeEdge edgecore is running, For logs visit: /var/log/kubeedge/edgecore.log
如果您无法重启 edgecore,请检查是否是由于 `kube-proxy` 的缘故,同时杀死这个进程。 **kubeedge**
默认不纳入该进程,我们使用 [edgemesh](https://github.com/kubeedge/kubeedge/blob/master/docs/proposals/edgemesh-design.md) 来进行替代

**注意:** 可以考虑避免 `kube-proxy` 部署在 edgenode 上。有两种解决方法:
**注意:** 可以考虑避免 `kube-proxy` 部署在 edgenode 上。<a name="anchor-name">有两种解决方法:</a >

1. 通过调用 `kubectl edit daemonsets.apps -n kube-system kube-proxy` 添加以下设置:

Expand Down
2 changes: 1 addition & 1 deletion versionsArchived.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"Next": "/docs/",
"Next": "https://kubeedge.io/docs/",
"v1.17": "https://release-1-17.docs.kubeedge.io/docs/",
"v1.16": "https://release-1-16.docs.kubeedge.io/docs/",
"v1.15": "https://release-1-15.docs.kubeedge.io/docs/",
Expand Down