diff --git a/_posts/aews/2025-02-22-aews03.md b/_posts/aews/2025-02-22-aews03.md new file mode 100644 index 0000000..0d87fcb --- /dev/null +++ b/_posts/aews/2025-02-22-aews03.md @@ -0,0 +1,7691 @@ +--- +title: AEWS 3์ฃผ์ฐจ ์ •๋ฆฌ +date: 2025-02-22 23:20:00 +0900 +categories: [EKS] +tags: [AEWS] +--- + + +## **๐Ÿš€ ์‹ค์Šต ํ™˜๊ฒฝ ๋ฐฐํฌ** + +2๊ฐœ์˜ VPC(EKS ๋ฐฐํฌ, ์šด์˜์šฉ ๊ตฌ๋ถ„), myeks-vpc์˜ public ์— EFS ์ถ”๊ฐ€ + + +![Image](https://github.com/user-attachments/assets/c6dc5ae2-e376-4c68-acf2-59d53d18b904) + +## **๐Ÿ—๏ธ AWS CloudFormation์„ ํ†ตํ•ด ๊ธฐ๋ณธ ์‹ค์Šต ํ™˜๊ฒฝ ๋ฐฐํฌ** + +### **1. yaml ํŒŒ์ผ ๋‹ค์šด๋กœ๋“œ** + +```bash +curl -O https://s3.ap-northeast-2.amazonaws.com/cloudformation.cloudneta.net/K8S/myeks-3week.yaml +# ๊ฒฐ๊ณผ + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed + 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--100 14933 100 14933 0 0 141k 0 --:--:-- --:--:-- --:--:-- 140k +``` + +### **2. ๋ฐฐํฌ** + +```bash +aws cloudformation deploy --template-file ~/Downloads/myeks-3week.yaml \ + --stack-name myeks --parameter-overrides KeyName=kp-aews SgIngressSshCidr=$(curl -s ipinfo.io/ip)/32 --region ap-northeast-2 + +# ๊ฒฐ๊ณผ +Waiting for changeset to be created.. +Waiting for stack create/update to complete +Successfully created/updated stack - myeks +``` + +### **3. ์šด์˜์„œ๋ฒ„ EC2 ์— SSH ์ ‘์†** + +```bash +ssh -i kp-aews.pem ec2-user@$(aws cloudformation describe-stacks --stack-name myeks --query 'Stacks[*].Outputs[0].OutputValue' --output text) + , #_ + ~\_ ####_ Amazon Linux 2 + ~~ \_#####\ + ~~ \###| AL2 End of Life is 2026-06-30. + ~~ \#/ ___ + ~~ V~' '-> + ~~~ / A newer version of Amazon Linux is available! + ~~._. _/ + _/ _/ Amazon Linux 2023, GA and supported until 2028-03-15. + _/m/' https://aws.amazon.com/linux/amazon-linux-2023/ + +[root@operator-host ~]# +``` + +EFS์˜ ๋„คํŠธ์›Œํฌ ์ธํ„ฐํŽ˜์ด์Šค(ENI)๋ฅผ ํ†ตํ•ด ์„œ๋ธŒ๋„ท ID ๋ฐ IPv4 ์ฃผ์†Œ๊ฐ€ ํ• ๋‹น๋จ + +EFS > Network ๋ฉ”๋‰ด์—์„œ **์„œ๋ธŒ๋„ท ID, IPv4 Address ํ™•์ธ ๊ฐ€๋Šฅ** โœ… + +![Image](https://github.com/user-attachments/assets/dc27ce13-2cd2-48fe-af9f-fe94831cf368) + + +## **๐Ÿ› ๏ธ eksctl ์„ ํ†ตํ•ด EKS ๋ฐฐํฌ** + +### **1. ๋ณ€์ˆ˜ ์„ค์ •** + +- **`Cluster Name`, `VPC ID`, `Public Subnet`, `SSH Key`** ๋ณ€์ˆ˜๋กœ ์ €์žฅ + +```bash +export CLUSTER_NAME=myeks + +# VPC ID ๊ฐ€์ ธ์˜ค๊ธฐ +export VPCID=$(aws ec2 describe-vpcs --filters "Name=tag:Name,Values=$CLUSTER_NAME-VPC" --query 'Vpcs[*].VpcId' --output text) +echo $VPCID +vpc-0e32b5a6653acdcd9 + +# Public Subnet ID ๊ฐ€์ ธ์˜ค๊ธฐ +export PubSubnet1=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet1" --query "Subnets[0].[SubnetId]" --output text) +export PubSubnet2=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet2" --query "Subnets[0].[SubnetId]" --output text) +export PubSubnet3=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet3" --query "Subnets[0].[SubnetId]" --output text) +echo $PubSubnet1 $PubSubnet2 $PubSubnet3 +subnet-0fed28a1b3e108719 subnet-0e4fb63cb543698fe subnet-0861bd68771150000 + +# ์ž์‹ ์˜ SSH Keypair ์ด๋ฆ„ +SSHKEYNAME=kp-aews +``` + +### **2. EKS ํด๋Ÿฌ์Šคํ„ฐ ์„ค์ • ํŒŒ์ผ ์ƒ์„ฑ** + +- **`myeks.yaml` ํŒŒ์ผ ์ƒ์„ฑ** + +```bash +cat << EOF > myeks.yaml +apiVersion: eksctl.io/v1alpha5 +kind: ClusterConfig +metadata: + name: myeks + region: ap-northeast-2 + version: "1.31" + +iam: + withOIDC: true # enables the IAM OIDC provider as well as IRSA for the Amazon CNI plugin + + serviceAccounts: # service accounts to create in the cluster. See IAM Service Accounts + - metadata: + name: aws-load-balancer-controller + namespace: kube-system + wellKnownPolicies: + awsLoadBalancerController: true + +vpc: + cidr: 192.168.0.0/16 + clusterEndpoints: + privateAccess: true # if you only want to allow private access to the cluster + publicAccess: true # if you want to allow public access to the cluster + id: $VPCID + subnets: + public: + ap-northeast-2a: + az: ap-northeast-2a + cidr: 192.168.1.0/24 + id: $PubSubnet1 + ap-northeast-2b: + az: ap-northeast-2b + cidr: 192.168.2.0/24 + id: $PubSubnet2 + ap-northeast-2c: + az: ap-northeast-2c + cidr: 192.168.3.0/24 + id: $PubSubnet3 + +addons: + - name: vpc-cni # no version is specified so it deploys the default version + version: latest # auto discovers the latest available + attachPolicyARNs: # attach IAM policies to the add-on's service account + - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy + configurationValues: |- + enableNetworkPolicy: "true" + + - name: kube-proxy + version: latest + + - name: coredns + version: latest + + - name: metrics-server + version: latest + +managedNodeGroups: +- amiFamily: AmazonLinux2023 + desiredCapacity: 3 + iam: + withAddonPolicies: + certManager: true # Enable cert-manager + externalDNS: true # Enable ExternalDNS + instanceType: t3.medium + preBootstrapCommands: + # install additional packages + - "dnf install nvme-cli links tree tcpdump sysstat ipvsadm ipset bind-utils htop -y" + labels: + alpha.eksctl.io/cluster-name: myeks + alpha.eksctl.io/nodegroup-name: ng1 + maxPodsPerNode: 100 + maxSize: 3 + minSize: 3 + name: ng1 + ssh: + allow: true + publicKeyName: $SSHKEYNAME + tags: + alpha.eksctl.io/nodegroup-name: ng1 + alpha.eksctl.io/nodegroup-type: managed + volumeIOPS: 3000 + volumeSize: 120 + volumeThroughput: 125 + volumeType: gp3 +EOF +``` + +- **์‹ค์ œ ์ž‘์„ฑ๋œ `myeks.yaml` ํŒŒ์ผ** + +```bash +apiVersion: eksctl.io/v1alpha5 +kind: ClusterConfig +metadata: + name: myeks + region: ap-northeast-2 + version: "1.31" + +iam: + withOIDC: true # enables the IAM OIDC provider as well as IRSA for the Amazon CNI plugin + + serviceAccounts: # service accounts to create in the cluster. See IAM Service Accounts + - metadata: + name: aws-load-balancer-controller + namespace: kube-system + wellKnownPolicies: + awsLoadBalancerController: true + +vpc: + cidr: 192.168.0.0/16 + clusterEndpoints: + privateAccess: true # if you only want to allow private access to the cluster + publicAccess: true # if you want to allow public access to the cluster + id: vpc-0e32b5a6653acdcd9 + subnets: + public: + ap-northeast-2a: + az: ap-northeast-2a + cidr: 192.168.1.0/24 + id: subnet-0fed28a1b3e108719 + ap-northeast-2b: + az: ap-northeast-2b + cidr: 192.168.2.0/24 + id: subnet-0e4fb63cb543698fe + ap-northeast-2c: + az: ap-northeast-2c + cidr: 192.168.3.0/24 + id: subnet-0861bd68771150000 + +addons: + - name: vpc-cni # no version is specified so it deploys the default version + version: latest # auto discovers the latest available + attachPolicyARNs: # attach IAM policies to the add-on's service account + - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy + configurationValues: |- + enableNetworkPolicy: "true" + + - name: kube-proxy + version: latest + + - name: coredns + version: latest + + - name: metrics-server + version: latest + +managedNodeGroups: +- amiFamily: AmazonLinux2023 + desiredCapacity: 3 + iam: + withAddonPolicies: + certManager: true # Enable cert-manager + externalDNS: true # Enable ExternalDNS + instanceType: t3.medium + preBootstrapCommands: + # install additional packages + - "dnf install nvme-cli links tree tcpdump sysstat ipvsadm ipset bind-utils htop -y" + labels: + alpha.eksctl.io/cluster-name: myeks + alpha.eksctl.io/nodegroup-name: ng1 + maxPodsPerNode: 100 + maxSize: 3 + minSize: 3 + name: ng1 + ssh: + allow: true + publicKeyName: kp-aews + tags: + alpha.eksctl.io/nodegroup-name: ng1 + alpha.eksctl.io/nodegroup-type: managed + volumeIOPS: 3000 + volumeSize: 120 + volumeThroughput: 125 + volumeType: gp3 +``` + +### **3. eks ๋ฐฐํฌ** + +```bash +eksctl create cluster -f myeks.yaml --verbose 4 + +# ๊ฒฐ๊ณผ +2025-02-18 11:31:47 [โ–ถ] Setting credentials expiry window to 30 minutes +2025-02-18 11:31:48 [โ–ถ] role ARN for the current session is "arn:aws:iam::378102432899:user/eks-user" +2025-02-18 11:31:48 [โ„น] eksctl version 0.203.0 +2025-02-18 11:31:48 [โ„น] using region ap-northeast-2 +2025-02-18 11:31:48 [โœ”] using existing VPC (vpc-0e32b5a6653acdcd9) and subnets (private:map[] public:map[ap-northeast-2a:{subnet-0fed28a1b3e108719 ap-northeast-2a 192.168.1.0/24 0 } ap-northeast-2b:{subnet-0e4fb63cb543698fe ap-northeast-2b 192.168.2.0/24 0 } ap-northeast-2c:{subnet-0861bd68771150000 ap-northeast-2c 192.168.3.0/24 0 }]) +2025-02-18 11:31:48 [!] custom VPC/subnets will be used; if resulting cluster doesn't function as expected, make sure to review the configuration of VPC/subnets +2025-02-18 11:31:48 [โ„น] nodegroup "ng1" will use "" [AmazonLinux2023/1.31] +2025-02-18 11:31:48 [โ„น] using EC2 key pair "kp-aews" +2025-02-18 11:31:48 [โ„น] using Kubernetes version 1.31 +2025-02-18 11:31:48 [โ„น] creating EKS cluster "myeks" in "ap-northeast-2" region with managed nodes +2025-02-18 11:31:48 [โ–ถ] cfg.json = \ +....... +``` + +![Image](https://github.com/user-attachments/assets/046d473e-9084-430b-9bda-519c0b24ea69) + +![Image](https://github.com/user-attachments/assets/1b5f8b1d-861c-40c7-be8e-2efdf538e324) + + +### **4. EKS ๋„ค์ž„์ŠคํŽ˜์ด์Šค ๋ฐ ์ปจํ…์ŠคํŠธ ์„ค์ • ๋ณ€๊ฒฝ** + +- **๋„ค์ž„์ŠคํŽ˜์ด์Šค default ๋ณ€๊ฒฝ ์ ์šฉ** + +```bash +kubens default +# ๊ฒฐ๊ณผ +โœ” Active namespace is "default" +``` + +- **EKS ์ปจํ…์ŠคํŠธ๋ช… ๋ณ€๊ฒฝ (eks-user โ†’ eksworkshop)** + +```bash +kubectl ctx +# ๊ฒฐ๊ณผ +Switched to context "eks-user@myeks.ap-northeast-2.eksctl.io". + +kubectl config rename-context "eks-user@myeks.ap-northeast-2.eksctl.io" "eksworkshop" +# ๊ฒฐ๊ณผ +Context "eks-user@myeks.ap-northeast-2.eksctl.io" renamed to "eksworkshop". +``` + +### **5. EKS ํด๋Ÿฌ์Šคํ„ฐ ์ •๋ณด ํ™•์ธ** + +```bash +kubectl cluster-info +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Kubernetes control plane is running at https://791BC5A9BB3716EA88C45304E0696F83.gr7.ap-northeast-2.eks.amazonaws.com +CoreDNS is running at https://791BC5A9BB3716EA88C45304E0696F83.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy + +To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. +``` + +### **6. EKS ๋…ธ๋“œ ์ •๋ณด ์กฐํšŒ** + +- ํด๋Ÿฌ์Šคํ„ฐ์˜ ๋…ธ๋“œ ์ƒํƒœ, ์ธ์Šคํ„ด์Šค ์œ ํ˜•, ์šฉ๋Ÿ‰ ์œ ํ˜•(์˜จ๋””๋งจ๋“œ), ๊ฐ€์šฉ ์˜์—ญ ํ™•์ธ + +```bash +kubectl get node --label-columns=node.kubernetes.io/instance-type,eks.amazonaws.com/capacityType,topology.kubernetes.io/zone +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION INSTANCE-TYPE CAPACITYTYPE ZONE +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 26m v1.31.5-eks-5d632ec t3.medium ON_DEMAND ap-northeast-2a +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 26m v1.31.5-eks-5d632ec t3.medium ON_DEMAND ap-northeast-2b +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 26m v1.31.5-eks-5d632ec t3.medium ON_DEMAND ap-northeast-2c +``` + +### **7. ๊ด€๋ฆฌํ˜• ๋…ธ๋“œ ๊ทธ๋ฃน ํ™•์ธ** + +```bash +eksctl get nodegroup --cluster $CLUSTER_NAME +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +CLUSTER NODEGROUP STATUS CREATED MIN SIZE MAX SIZE DESIRED CAPACITY INSTANCE TYPE IMAGE ID ASG NAME TYPE +myeks ng1 ACTIVE 2025-02-18T02:45:19Z 3 3 3 t3.medium AL2023_x86_64_STANDARD eks-ng1-c8ca8b79-064a-1771-a08d-6ace8e163bd4 managed +``` + +```bash +aws eks describe-nodegroup --cluster-name $CLUSTER_NAME --nodegroup-name ng1 | jq +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +{ + "nodegroup": { + "nodegroupName": "ng1", + "nodegroupArn": "arn:aws:eks:ap-northeast-2:378102432899:nodegroup/myeks/ng1/c8ca8b79-064a-1771-a08d-6ace8e163bd4", + "clusterName": "myeks", + "version": "1.31", + "releaseVersion": "1.31.5-20250212", + "createdAt": "2025-02-18T11:45:19.813000+09:00", + "modifiedAt": "2025-02-18T12:06:04.747000+09:00", + "status": "ACTIVE", + "capacityType": "ON_DEMAND", + "scalingConfig": { + "minSize": 3, + "maxSize": 3, + "desiredSize": 3 + }, + "instanceTypes": [ + "t3.medium" + ], + "subnets": [ + "subnet-0fed28a1b3e108719", + "subnet-0e4fb63cb543698fe", + "subnet-0861bd68771150000" + ], + "amiType": "AL2023_x86_64_STANDARD", + "nodeRole": "arn:aws:iam::378102432899:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-rGyQG9rZlOwl", + "labels": { + "alpha.eksctl.io/cluster-name": "myeks", + "alpha.eksctl.io/nodegroup-name": "ng1" + }, + "resources": { + "autoScalingGroups": [ + { + "name": "eks-ng1-c8ca8b79-064a-1771-a08d-6ace8e163bd4" + } + ] + }, + "health": { + "issues": [] + }, + "updateConfig": { + "maxUnavailable": 1 + }, + "launchTemplate": { + "name": "eksctl-myeks-nodegroup-ng1", + "version": "1", + "id": "lt-070d49ace29ca00a6" + }, + "tags": { + "aws:cloudformation:stack-name": "eksctl-myeks-nodegroup-ng1", + "alpha.eksctl.io/cluster-name": "myeks", + "alpha.eksctl.io/nodegroup-name": "ng1", + "aws:cloudformation:stack-id": "arn:aws:cloudformation:ap-northeast-2:378102432899:stack/eksctl-myeks-nodegroup-ng1/55132e60-eda2-11ef-876d-06d9644ecb71", + "eksctl.cluster.k8s.io/v1alpha1/cluster-name": "myeks", + "aws:cloudformation:logical-id": "ManagedNodeGroup", + "alpha.eksctl.io/nodegroup-type": "managed", + "alpha.eksctl.io/eksctl-version": "0.203.0" + } + } +} +``` + +### **8. EKS addon ์ƒํƒœ ํ™•์ธ** + +```bash +eksctl get addon --cluster $CLUSTER_NAME +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +2025-02-18 12:16:16 [โ„น] Kubernetes version "1.31" in use by cluster "myeks" +2025-02-18 12:16:16 [โ„น] getting all addons +2025-02-18 12:16:18 [โ„น] to see issues for an addon run `eksctl get addon --name --cluster ` +NAME VERSION STATUS ISSUES IAMROLE UPDATE AVAILABLE CONFIGURATION VALUES POD IDENTITY ASSOCIATION ROLES +coredns v1.11.4-eksbuild.2 ACTIVE 0 +kube-proxy v1.31.3-eksbuild.2 ACTIVE 0 +metrics-server v0.7.2-eksbuild.2 ACTIVE 0 +vpc-cni v1.19.2-eksbuild.5 ACTIVE 0 arn:aws:iam::378102432899:role/eksctl-myeks-addon-vpc-cni-Role1-ZTYxtOMDwfFu enableNetworkPolicy: "true" +``` + +### **9. AWS Load Balancer Controller์šฉ IAM ์„œ๋น„์Šค ๊ณ„์ • ํ™•์ธ** + +**(1) IAM ์„œ๋น„์Šค ๊ณ„์ • ํ™•์ธ** + +- `aws-load-balancer-controller` ์„œ๋น„์Šค ๊ณ„์ •์ด **IAM Role**๊ณผ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ๋ฐ”์ธ๋”ฉ๋˜์—ˆ๋Š”์ง€ ํ™•์ธ + +```bash +eksctl get iamserviceaccount --cluster $CLUSTER_NAME +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAMESPACE NAME ROLE ARN +kube-system aws-load-balancer-controller arn:aws:iam::378102432899:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-O6YEYsN7iVeQ +``` + +**(2) myeks.yaml ํŒŒ์ผ ํ™•์ธ** + +- `certManager` ๋ฐ `externalDNS`๋ฅผ ํ™œ์„ฑํ™”ํ•˜์—ฌ **SSL ์ธ์ฆ์„œ ์ž๋™ํ™” ๋ฐ ์™ธ๋ถ€ DNS ๋ ˆ์ฝ”๋“œ ๊ด€๋ฆฌ ๊ธฐ๋Šฅ ์ถ”๊ฐ€ํ–ˆ์Œ** + +```bash +managedNodeGroups: +- amiFamily: AmazonLinux2023 + desiredCapacity: 3 + iam: + withAddonPolicies: + certManager: true # Enable cert-manager + externalDNS: true # Enable ExternalDNS +``` + +**(3) EC2 ์ธ์Šคํ„ด์Šค์˜ IAM Role ํ™•์ธ** + +- ํด๋Ÿฌ์Šคํ„ฐ์˜ ๊ด€๋ฆฌํ˜• ๋…ธ๋“œ ๊ทธ๋ฃน(`myeks-ng1-Node`)์— ์—ฐ๊ฒฐ๋œ **IAM Role** ํ™•์ธ +- `eksctl-myeks-nodegroup-ng1-NodeInstanceRole-rGyQG9rZlOwl` ์—ญํ• ์ด ๋ถ€์—ฌ๋จ + +![Image](https://github.com/user-attachments/assets/d186768c-2d88-4c5c-bf63-117579cea517) + + +- ์ด ์—ญํ• ์„ ํ†ตํ•ด `externalDNS` ๋ฐ `certManager`์˜ IAM ์ •์ฑ…์ด ์ ์šฉ๋จ + +![Image](https://github.com/user-attachments/assets/b34a83c7-1596-4bca-be99-b0ed84524bdd) + +--- + +## **๐Ÿ–ฅ๏ธ ๊ด€๋ฆฌํ˜• ๋…ธ๋“œ ๊ทธ๋ฃน(EC2) ์ ‘์†** + +### **1. ์ธ์Šคํ„ด์Šค ์ •๋ณด ํ™•์ธ** + +```bash +aws ec2 describe-instances --query "Reservations[*].Instances[*].{InstanceID:InstanceId, PublicIPAdd:PublicIpAddress, PrivateIPAdd:PrivateIpAddress, InstanceName:Tags[?Key=='Name']|[0].Value, Status:State.Name}" --filters Name=instance-state-name,Values=running --output table +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +---------------------------------------------------------------------------------------- +| DescribeInstances | ++----------------------+-----------------+----------------+----------------+-----------+ +| InstanceID | InstanceName | PrivateIPAdd | PublicIPAdd | Status | ++----------------------+-----------------+----------------+----------------+-----------+ +| i-0484d2b724be33973 | myeks-ng1-Node | 192.168.3.80 | 15.164.49.232 | running | +| i-0b995a92db06f58d8 | operator-host | 172.20.1.100 | 3.35.230.35 | running | +| i-093ad32d5ff5a8770 | myeks-ng1-Node | 192.168.1.207 | 3.35.47.226 | running | +| i-0a80fdc36a856f394 | myeks-ng1-Node | 192.168.2.84 | 43.203.131.45 | running | ++----------------------+-----------------+----------------+----------------+-----------+ +``` + +### **2. ๊ฐ€์šฉ ์˜์—ญ(AZ)๋ณ„ EC2 ๊ณต์ธ IP ์กฐํšŒ** + +- **AZ1 ๋ฐฐ์น˜๋œ EC2 ๊ณต์ธ IP** + +```bash +aws ec2 describe-instances \ + --filters "Name=tag:Name,Values=myeks-ng1-Node" "Name=availability-zone,Values=ap-northeast-2a" \ + --query 'Reservations[*].Instances[*].PublicIpAddress' \ + --output text + +# ๊ฒฐ๊ณผ +3.35.47.226 +``` + +- **AZ2 ๋ฐฐ์น˜๋œ EC2 ๊ณต์ธ IP** + +```bash +aws ec2 describe-instances \ + --filters "Name=tag:Name,Values=myeks-ng1-Node" "Name=availability-zone,Values=ap-northeast-2b" \ + --query 'Reservations[*].Instances[*].PublicIpAddress' \ + --output text + +# ๊ฒฐ๊ณผ +43.203.131.45 +``` + +- **AZ3 ๋ฐฐ์น˜๋œ EC2 ๊ณต์ธ IP** + +```bash +aws ec2 describe-instances \ + --filters "Name=tag:Name,Values=myeks-ng1-Node" "Name=availability-zone,Values=ap-northeast-2c" \ + --query 'Reservations[*].Instances[*].PublicIpAddress' \ + --output text + +# ๊ฒฐ๊ณผ +15.164.49.232 +``` + +### **3. AZ๋ณ„ EC2 ๊ณต์ธ IP๋ฅผ ํ™˜๊ฒฝ ๋ณ€์ˆ˜์— ์ €์žฅ** + +- ๊ฐ ๊ฐ€์šฉ ์˜์—ญ(AZ)์— ๋ฐฐ์น˜๋œ EC2 ๊ณต์ธ IP๋ฅผ ๋ณ€์ˆ˜(`N1`, `N2`, `N3`)์— ์ €์žฅ + +```bash +export N1=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=myeks-ng1-Node" "Name=availability-zone,Values=ap-northeast-2a" --query 'Reservations[*].Instances[*].PublicIpAddress' --output text) +export N2=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=myeks-ng1-Node" "Name=availability-zone,Values=ap-northeast-2b" --query 'Reservations[*].Instances[*].PublicIpAddress' --output text) +export N3=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=myeks-ng1-Node" "Name=availability-zone,Values=ap-northeast-2c" --query 'Reservations[*].Instances[*].PublicIpAddress' --output text) + +echo $N1, $N2, $N3 +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +3.35.47.226, 43.203.131.45, 15.164.49.232 +``` + +### **4. EKS ์›Œ์ปค๋…ธ๋“œ์˜ SSH ๋ณด์•ˆ ๊ทธ๋ฃน ์ž๋™ ์ถ”๊ฐ€ ํ™•์ธ** + +- `myeks.yaml`์—์„œ `ssh.allow: true`๋กœ ์„ค์ •ํ•˜๋ฉด **EKS ์›Œ์ปค๋…ธ๋“œ ๋ณด์•ˆ ๊ทธ๋ฃน**์ด ์ž๋™ ์ƒ์„ฑ๋จ + +```bash + ssh: + allow: true + publicKeyName: kp-aews +``` + +- ์ƒ์„ฑ๋œ ๋ณด์•ˆ ๊ทธ๋ฃน: **`sg-0edd80591095505a9 (eksctl-myeks-nodegroup-ng1-remoteAccess)`** + +![Image](https://github.com/user-attachments/assets/787514f6-98b4-4013-a802-c1df0c973bd4) + +- ๊ธฐ๋ณธ์„ค์ •: **ํฌํŠธ 22๋ฒˆ(SSH)์ด Any(0.0.0.0/0)๋กœ ์˜คํ”ˆ**๋จ +- **๋ณด์•ˆ ์œ„ํ—˜:** ๋ถˆํŠน์ • ๋‹ค์ˆ˜๊ฐ€ SSH ์ ‘๊ทผ ๊ฐ€๋Šฅํ•˜์—ฌ ๋ณด์•ˆ ์ทจ์•ฝ์  ๋ฐœ์ƒ ๊ฐ€๋Šฅ +- **๊ฐœ์„  ๋ฐฉ์•ˆ:** `ssh.allow: false`๋กœ ์„ค์ •ํ•˜๊ณ , **์ง์ ‘ ๋ณด์•ˆ ๊ทธ๋ฃน์—์„œ IP๋ฅผ ํ†ต์ œํ•˜๋Š” ๋ฐฉ์‹์ด ์•ˆ์ „** + +![Image](https://github.com/user-attachments/assets/b6840204-d4df-49c7-8257-b8da0b9a1af2) + + +### **5. ํŠน์ • IP์—์„œ๋งŒ ์ ‘๊ทผ ํ—ˆ์šฉ (๋ณด์•ˆ ๊ฐ•ํ™”)** + +- ๋ณด์•ˆ ๊ทธ๋ฃน(`remoteAccess`)์—์„œ **์ง‘ IP์™€ ์šด์˜ ์„œ๋ฒ„ EC2๋งŒ ์ ‘๊ทผ ๊ฐ€๋Šฅํ•˜๋„๋ก ์ˆ˜์ •** +- **๋ชจ๋“  ํŠธ๋ž˜ํ”ฝ์„ ํ—ˆ์šฉํ•˜์ง€ ์•Š๊ณ , ํŠน์ • IP์—์„œ๋งŒ ์ ‘๊ทผ ๊ฐ€๋Šฅํ•˜๋„๋ก ์ œํ•œ** + +**(1) `remoteAccess` ํฌํ•จ๋œ ๋ณด์•ˆ ๊ทธ๋ฃน ID ํ™•์ธ** + +```bash +aws ec2 describe-security-groups --filters "Name=group-name,Values=*remoteAccess*" | jq +export MNSGID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=*remoteAccess*" --query 'SecurityGroups[*].GroupId' --output text) +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +{ + "SecurityGroups": [ + { + "GroupId": "sg-0edd80591095505a9", + "IpPermissionsEgress": [ + { + "IpProtocol": "-1", + "UserIdGroupPairs": [], + "IpRanges": [ + { + "CidrIp": "0.0.0.0/0" + } + ], + "Ipv6Ranges": [], + "PrefixListIds": [] + } + ], + "Tags": [ + { + "Key": "alpha.eksctl.io/nodegroup-type", + "Value": "managed" + }, + { + "Key": "aws:cloudformation:logical-id", + "Value": "SSH" + }, + { + "Key": "alpha.eksctl.io/nodegroup-name", + "Value": "ng1" + }, + { + "Key": "alpha.eksctl.io/cluster-name", + "Value": "myeks" + }, + { + "Key": "Name", + "Value": "eksctl-myeks-nodegroup-ng1/SSH" + }, + { + "Key": "aws:cloudformation:stack-name", + "Value": "eksctl-myeks-nodegroup-ng1" + }, + { + "Key": "aws:cloudformation:stack-id", + "Value": "arn:aws:cloudformation:ap-northeast-2:378102432899:stack/eksctl-myeks-nodegroup-ng1/55132e60-eda2-11ef-876d-06d9644ecb71" + }, + { + "Key": "alpha.eksctl.io/eksctl-version", + "Value": "0.203.0" + }, + { + "Key": "eksctl.cluster.k8s.io/v1alpha1/cluster-name", + "Value": "myeks" + } + ], + "VpcId": "vpc-0e32b5a6653acdcd9", + "SecurityGroupArn": "arn:aws:ec2:ap-northeast-2:378102432899:security-group/sg-0edd80591095505a9", + "OwnerId": "378102432899", + "GroupName": "eksctl-myeks-nodegroup-ng1-remoteAccess", + "Description": "Allow SSH access", + "IpPermissions": [ + { + "IpProtocol": "tcp", + "FromPort": 22, + "ToPort": 22, + "UserIdGroupPairs": [], + "IpRanges": [ + { + "Description": "Allow SSH access to managed worker nodes in group ng1", + "CidrIp": "0.0.0.0/0" + } + ], + "Ipv6Ranges": [ + { + "Description": "Allow SSH access to managed worker nodes in group ng1", + "CidrIpv6": "::/0" + } + ], + "PrefixListIds": [] + } + ] + } + ] +} +``` + +**(2) ์ง‘ ๊ณต์ธ IP๋ฅผ ๋ณด์•ˆ ๊ทธ๋ฃน Inbound์— ์ถ”๊ฐ€** + +- ํ˜„์žฌ ์ ‘์† ์ค‘์ธ **์ง‘ ๊ณต์ธ IP**๋ฅผ ๋ณด์•ˆ ๊ทธ๋ฃน(`remoteAccess`)์˜ Inbound ๋ฃฐ์— ์ถ”๊ฐ€ + +```bash +aws ec2 authorize-security-group-ingress --group-id $MNSGID --protocol '-1' --cidr $(curl -s ipinfo.io/ip)/32 +{ + "Return": true, + "SecurityGroupRules": [ + { + "SecurityGroupRuleId": "sgr-08fceda8b8811d00b", + "GroupId": "sg-0edd80591095505a9", + "GroupOwnerId": "378102432899", + "IsEgress": false, + "IpProtocol": "-1", + "FromPort": -1, + "ToPort": -1, + "CidrIpv4": "182.230.60.93/32", + "SecurityGroupRuleArn": "arn:aws:ec2:ap-northeast-2:378102432899:security-group-rule/sgr-08fceda8b8811d00b" + } + ] +} +``` + +**(3) ์šด์˜ ์„œ๋ฒ„ ๋‚ด๋ถ€ IP๋ฅผ ๋ณด์•ˆ ๊ทธ๋ฃน Inbound์— ์ถ”๊ฐ€** + +- ์šด์˜ ์„œ๋ฒ„ ๋‚ด๋ถ€ IP(`172.20.1.100/32`)๋„ ๋ณด์•ˆ ๊ทธ๋ฃน์˜ Inbound ๋ฃฐ์— ์ถ”๊ฐ€ + +```bash +aws ec2 authorize-security-group-ingress --group-id $MNSGID --protocol '-1' --cidr 172.20.1.100/32 +{ + "Return": true, + "SecurityGroupRules": [ + { + "SecurityGroupRuleId": "sgr-02799adf73d669efa", + "GroupId": "sg-0edd80591095505a9", + "GroupOwnerId": "378102432899", + "IsEgress": false, + "IpProtocol": "-1", + "FromPort": -1, + "ToPort": -1, + "CidrIpv4": "172.20.1.100/32", + "SecurityGroupRuleArn": "arn:aws:ec2:ap-northeast-2:378102432899:security-group-rule/sgr-02799adf73d669efa" + } + ] +} +``` + +**(4) ์›Œ์ปค๋…ธ๋“œ ping ํ…Œ์ŠคํŠธ** + +```bash +ping -c 2 $N1 +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +PING 3.35.47.226 (3.35.47.226) 56(84) bytes of data. +64 bytes from 3.35.47.226: icmp_seq=1 ttl=115 time=13.7 ms +64 bytes from 3.35.47.226: icmp_seq=2 ttl=115 time=13.8 ms + +--- 3.35.47.226 ping statistics --- +2 packets transmitted, 2 received, 0% packet loss, time 1002ms +rtt min/avg/max/mdev = 13.744/13.779/13.815/0.035 ms +``` + +**(5) ์›Œ์ปค๋…ธ๋“œ SSH ์ ‘์† ํ…Œ์ŠคํŠธ** + +```bash +ssh -i kp-aews.pem -o StrictHostKeyChecking=no ec2-user@$N1 hostname +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +ip-192-168-1-207.ap-northeast-2.compute.internal +``` + +```bash +for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh -o StrictHostKeyChecking=no ec2-user@$i hostname; echo; done +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +>> node 3.35.47.226 << +ip-192-168-1-207.ap-northeast-2.compute.internal + +>> node 43.203.131.45 << +ip-192-168-2-84.ap-northeast-2.compute.internal + +>> node 15.164.49.232 << +ip-192-168-3-80.ap-northeast-2.compute.internal +``` + +```bash +ssh ec2-user@$N1 + +# ๊ฒฐ๊ณผ +A newer release of "Amazon Linux" is available. + Version 2023.6.20250211: +Run "/usr/bin/dnf check-release-update" for full release and version update info + , #_ + ~\_ ####_ Amazon Linux 2023 + ~~ \_#####\ + ~~ \###| + ~~ \#/ ___ https://aws.amazon.com/linux/amazon-linux-2023 + ~~ V~' '-> + ~~~ / + ~~._. _/ + _/ _/ + _/m/' +Last login: Wed Feb 12 05:52:48 2025 from 52.94.123.236 +[ec2-user@ip-192-168-1-207 ~]$ +``` + +```bash +ssh ec2-user@$N2 + +# ๊ฒฐ๊ณผ +A newer release of "Amazon Linux" is available. + Version 2023.6.20250211: +Run "/usr/bin/dnf check-release-update" for full release and version update info + , #_ + ~\_ ####_ Amazon Linux 2023 + ~~ \_#####\ + ~~ \###| + ~~ \#/ ___ https://aws.amazon.com/linux/amazon-linux-2023 + ~~ V~' '-> + ~~~ / + ~~._. _/ + _/ _/ + _/m/' +Last login: Wed Feb 12 05:52:48 2025 from 52.94.123.236 +[ec2-user@ip-192-168-2-84 ~]$ +``` + +```bash +ssh ec2-user@$N3 + +# ๊ฒฐ๊ณผ +A newer release of "Amazon Linux" is available. + Version 2023.6.20250211: +Run "/usr/bin/dnf check-release-update" for full release and version update info + , #_ + ~\_ ####_ Amazon Linux 2023 + ~~ \_#####\ + ~~ \###| + ~~ \#/ ___ https://aws.amazon.com/linux/amazon-linux-2023 + ~~ V~' '-> + ~~~ / + ~~._. _/ + _/ _/ + _/m/' +Last login: Wed Feb 12 05:52:48 2025 from 52.94.123.236 +[ec2-user@ip-192-168-3-80 ~]$ +``` + +--- + +## ๐Ÿ” **๋…ธ๋“œ ์ •๋ณด ํ™•์ธ** + +### **1. EKS ์›Œ์ปค๋…ธ๋“œ ๊ธฐ๋ณธ ์ •๋ณด ํ™•์ธ** + +```bash +for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i hostnamectl; echo; done +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +>> node 3.35.47.226 << + Static hostname: ip-192-168-1-207.ap-northeast-2.compute.internal + Icon name: computer-vm + Chassis: vm ๐Ÿ–ด + Machine ID: ec26a612d3fd1934a5b6b39b72fa9d18 + Boot ID: 98d99077d96a424daa3ab81b196f38a0 + Virtualization: amazon +Operating System: Amazon Linux 2023.6.20250203 + CPE OS Name: cpe:2.3:o:amazon:amazon_linux:2023 + Kernel: Linux 6.1.127-135.201.amzn2023.x86_64 + Architecture: x86-64 + Hardware Vendor: Amazon EC2 + Hardware Model: t3.medium +Firmware Version: 1.0 + +>> node 43.203.131.45 << + Static hostname: ip-192-168-2-84.ap-northeast-2.compute.internal + Icon name: computer-vm + Chassis: vm ๐Ÿ–ด + Machine ID: ec2f16bc9e1f90914542402b6bd7e2db + Boot ID: b921993e9cd54993bed4e93693d89aab + Virtualization: amazon +Operating System: Amazon Linux 2023.6.20250203 + CPE OS Name: cpe:2.3:o:amazon:amazon_linux:2023 + Kernel: Linux 6.1.127-135.201.amzn2023.x86_64 + Architecture: x86-64 + Hardware Vendor: Amazon EC2 + Hardware Model: t3.medium +Firmware Version: 1.0 + +>> node 15.164.49.232 << + Static hostname: ip-192-168-3-80.ap-northeast-2.compute.internal + Icon name: computer-vm + Chassis: vm ๐Ÿ–ด + Machine ID: ec2e2414fefb5afd3b0c3ec76d9735ce + Boot ID: 909e95d8c70f4b00a9cd6545fad4d33b + Virtualization: amazon +Operating System: Amazon Linux 2023.6.20250203 + CPE OS Name: cpe:2.3:o:amazon:amazon_linux:2023 + Kernel: Linux 6.1.127-135.201.amzn2023.x86_64 + Architecture: x86-64 + Hardware Vendor: Amazon EC2 + Hardware Model: t3.medium +Firmware Version: 1.0 +``` + +### **2. EKS ์›Œ์ปค๋…ธ๋“œ ๋„คํŠธ์›Œํฌ ์ธํ„ฐํŽ˜์ด์Šค ํ™•์ธ** + +```bash +for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i sudo ip -c addr; echo; done +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +>> node 3.35.47.226 << +1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host noprefixroute + valid_lft forever preferred_lft forever +2: ens5: mtu 9001 qdisc mq state UP group default qlen 1000 + link/ether 02:e7:74:d0:4e:63 brd ff:ff:ff:ff:ff:ff + altname enp0s5 + inet 192.168.1.207/24 metric 1024 brd 192.168.1.255 scope global dynamic ens5 + valid_lft 1811sec preferred_lft 1811sec + inet6 fe80::e7:74ff:fed0:4e63/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever + +>> node 43.203.131.45 << +1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host noprefixroute + valid_lft forever preferred_lft forever +2: ens5: mtu 9001 qdisc mq state UP group default qlen 1000 + link/ether 06:3b:f6:53:20:41 brd ff:ff:ff:ff:ff:ff + altname enp0s5 + inet 192.168.2.84/24 metric 1024 brd 192.168.2.255 scope global dynamic ens5 + valid_lft 1812sec preferred_lft 1812sec + inet6 fe80::43b:f6ff:fe53:2041/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever + +>> node 15.164.49.232 << +1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 + link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 + inet 127.0.0.1/8 scope host lo + valid_lft forever preferred_lft forever + inet6 ::1/128 scope host noprefixroute + valid_lft forever preferred_lft forever +2: ens5: mtu 9001 qdisc mq state UP group default qlen 1000 + link/ether 0a:2f:05:2e:be:c1 brd ff:ff:ff:ff:ff:ff + altname enp0s5 + inet 192.168.3.80/24 metric 1024 brd 192.168.3.255 scope global dynamic ens5 + valid_lft 1805sec preferred_lft 1805sec + inet6 fe80::82f:5ff:fe2e:bec1/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever +3: eni81d769258b0@if3: mtu 9001 qdisc noqueue state UP group default + link/ether ae:50:23:e1:4f:0f brd ff:ff:ff:ff:ff:ff link-netns cni-fa2faa2f-0179-8831-ef6d-458723488300 + inet6 fe80::ac50:23ff:fee1:4f0f/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever +4: enia025c0419e6@if3: mtu 9001 qdisc noqueue state UP group default + link/ether 4e:d3:3a:8d:e9:53 brd ff:ff:ff:ff:ff:ff link-netns cni-994c349e-66b0-685d-52c0-6358456a1ee1 + inet6 fe80::4cd3:3aff:fe8d:e953/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever +5: eni181f90d8a40@if3: mtu 9001 qdisc noqueue state UP group default + link/ether be:89:71:d7:fc:1c brd ff:ff:ff:ff:ff:ff link-netns cni-dca67549-f5c2-daf8-b13c-c641ff9b4191 + inet6 fe80::bc89:71ff:fed7:fc1c/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever +6: enifa068c4d7bd@if3: mtu 9001 qdisc noqueue state UP group default + link/ether 92:11:37:91:87:7f brd ff:ff:ff:ff:ff:ff link-netns cni-99f480e7-d3cc-12d0-c204-65a883192bdb + inet6 fe80::9011:37ff:fe91:877f/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever +7: ens6: mtu 9001 qdisc mq state UP group default qlen 1000 + link/ether 0a:75:29:ce:c9:4d brd ff:ff:ff:ff:ff:ff + altname enp0s6 + inet 192.168.3.26/24 brd 192.168.3.255 scope global ens6 + valid_lft forever preferred_lft forever + inet6 fe80::875:29ff:fece:c94d/64 scope link proto kernel_ll + valid_lft forever preferred_lft forever +``` + +### **3. EKS ์›Œ์ปค๋…ธ๋“œ ์Šคํ† ๋ฆฌ์ง€ ์ •๋ณด ํ™•์ธ** + +- ์Šคํ† ๋ฆฌ์ง€ ํ™•์ธ ๋ฐ ๋ฃจํŠธ ๋ณผ๋ฅจ ํฌ๊ธฐ ์ฒดํฌ +- ๋ฃจํŠธ ๋ณผ๋ฅจ **120GB** (`nvme0n1`) +- `/boot/efi` ํฌํ•จ๋œ ํŒŒํ‹ฐ์…˜ ๊ตฌ์กฐ ํ™•์ธ + +```bash +for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i lsblk; echo; done +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +>> node 3.35.47.226 << +NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS +nvme0n1 259:0 0 120G 0 disk +โ”œโ”€nvme0n1p1 259:1 0 120G 0 part / +โ”œโ”€nvme0n1p127 259:2 0 1M 0 part +โ””โ”€nvme0n1p128 259:3 0 10M 0 part /boot/efi + +>> node 43.203.131.45 << +NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS +nvme0n1 259:0 0 120G 0 disk +โ”œโ”€nvme0n1p1 259:1 0 120G 0 part / +โ”œโ”€nvme0n1p127 259:2 0 1M 0 part +โ””โ”€nvme0n1p128 259:3 0 10M 0 part /boot/efi + +>> node 15.164.49.232 << +NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS +nvme0n1 259:0 0 120G 0 disk +โ”œโ”€nvme0n1p1 259:1 0 120G 0 part / +โ”œโ”€nvme0n1p127 259:2 0 1M 0 part +โ””โ”€nvme0n1p128 259:3 0 10M 0 part /boot/efi +``` + +### **4. EKS ์›Œ์ปค๋…ธ๋“œ ๋””์Šคํฌ ์‚ฌ์šฉ๋Ÿ‰ ํ™•์ธ** + +- ๋ฃจํŠธ ํŒŒ์ผ ์‹œ์Šคํ…œ์˜ ๋””์Šคํฌ ์‚ฌ์šฉ๋Ÿ‰ ํ™•์ธ + +```bash +for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i df -hT /; echo; done +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +>> node 3.35.47.226 << +Filesystem Type Size Used Avail Use% Mounted on +/dev/nvme0n1p1 xfs 120G 3.4G 117G 3% / + +>> node 43.203.131.45 << +Filesystem Type Size Used Avail Use% Mounted on +/dev/nvme0n1p1 xfs 120G 3.4G 117G 3% / + +>> node 15.164.49.232 << +Filesystem Type Size Used Avail Use% Mounted on +/dev/nvme0n1p1 xfs 120G 3.5G 117G 3% / +``` + +### **5. ๊ธฐ๋ณธ ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ํ™•์ธ** + +- ๊ธฐ๋ณธ์ ์œผ๋กœ `gp2`(AWS EBS) ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ์‚ฌ์šฉ +- `gp3`๊ฐ€ **์„ฑ๋Šฅ ๋ฐ ๋น„์šฉ ์ธก๋ฉด์—์„œ ๋” ์œ ๋ฆฌ**ํ•˜๋ฏ€๋กœ ์ดํ›„ `gp3` ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค๋ฅผ ์ถ”๊ฐ€ํ•  ์˜ˆ์ • + +```bash +kubectl get sc +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 102m +``` + +- ๊ธฐ๋ณธ ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค(`gp2`) ์ƒ์„ธ ์ •๋ณด ์กฐํšŒ + +```bash +kubectl describe sc gp2 +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Name: gp2 +IsDefaultClass: No +Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"gp2"},"parameters":{"fsType":"ext4","type":"gp2"},"provisioner":"kubernetes.io/aws-ebs","volumeBindingMode":"WaitForFirstConsumer"} + +Provisioner: kubernetes.io/aws-ebs +Parameters: fsType=ext4,type=gp2 +AllowVolumeExpansion: +MountOptions: +ReclaimPolicy: Delete +VolumeBindingMode: WaitForFirstConsumer +Events: +``` + +### **6. ํ˜„์žฌ ์„ค์น˜๋œ CRD ํ™•์ธ** + +- ๊ธฐ๋ณธ CRD ๋ชฉ๋ก ์กฐํšŒ +- **์ปจํŠธ๋กค๋Ÿฌ ์„ค์น˜ ํ›„ CRD๊ฐ€ ์ถ”๊ฐ€๋  ์˜ˆ์ •** + +```bash +kubectl get crd +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAME CREATED AT +cninodes.vpcresources.k8s.aws 2025-02-18T02:37:25Z +eniconfigs.crd.k8s.amazonaws.com 2025-02-18T02:40:58Z +policyendpoints.networking.k8s.aws 2025-02-18T02:37:25Z +securitygrouppolicies.vpcresources.k8s.aws 2025-02-18T02:37:25Z +``` + +### **7. CSI ๋…ธ๋“œ ํ™•์ธ** + +- ํ˜„์žฌ **CSI(Storage Interface) ๋“œ๋ผ์ด๋ฒ„ ์—†์Œ** +- ์ดํ›„ **์Šคํ† ๋ฆฌ์ง€ ๊ด€๋ จ ์ปจํŠธ๋กค๋Ÿฌ ์„ค์น˜ ์‹œ ์ถ”๊ฐ€๋  ์˜ˆ์ •** + +```bash +kubectl get csinodes +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAME DRIVERS AGE +ip-192-168-1-207.ap-northeast-2.compute.internal 0 97m +ip-192-168-2-84.ap-northeast-2.compute.internal 0 97m +ip-192-168-3-80.ap-northeast-2.compute.internal 0 97m +``` + +### **8. EKS ๋…ธ๋“œ๋ณ„ ์ตœ๋Œ€ Pod ๊ฐœ์ˆ˜ ํ™•์ธ** + +```bash +kubectl describe node | grep Capacity: -A13 +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Capacity: + cpu: 2 + ephemeral-storage: 125751276Ki + hugepages-1Gi: 0 + hugepages-2Mi: 0 + memory: 3919536Ki + pods: 100 +Allocatable: + cpu: 1930m + ephemeral-storage: 114818633946 + hugepages-1Gi: 0 + hugepages-2Mi: 0 + memory: 3364528Ki + pods: 100 +-- +Capacity: + cpu: 2 + ephemeral-storage: 125751276Ki + hugepages-1Gi: 0 + hugepages-2Mi: 0 + memory: 3919536Ki + pods: 100 +Allocatable: + cpu: 1930m + ephemeral-storage: 114818633946 + hugepages-1Gi: 0 + hugepages-2Mi: 0 + memory: 3364528Ki + pods: 100 +-- +Capacity: + cpu: 2 + ephemeral-storage: 125751276Ki + hugepages-1Gi: 0 + hugepages-2Mi: 0 + memory: 3919544Ki + pods: 100 +Allocatable: + cpu: 1930m + ephemeral-storage: 114818633946 + hugepages-1Gi: 0 + hugepages-2Mi: 0 + memory: 3364536Ki + pods: 100 +``` + +```bash +kubectl get nodes -o custom-columns="NAME:.metadata.name,MAXPODS:.status.capacity.pods" +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAME MAXPODS +ip-192-168-1-207.ap-northeast-2.compute.internal 100 +ip-192-168-2-84.ap-northeast-2.compute.internal 100 +ip-192-168-3-80.ap-northeast-2.compute.internal 100 +``` + +### **9. Kubelet `maxPods` ๊ธฐ๋ณธ๊ฐ’ ํ™•์ธ** + +- `/etc/kubernetes/kubelet/config.json` + +```bash +for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i sudo cat /etc/kubernetes/kubelet/config.json | grep maxPods; echo; done +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +>> node 3.35.47.226 << + "maxPods": 17, + +>> node 43.203.131.45 << + "maxPods": 17, + +>> node 15.164.49.232 << + "maxPods": 17, +``` + +### **10. ์‚ฌ์šฉ์ž ์ •์˜ Kubelet ์„ค์ • ํ™•์ธ** + +- `/etc/kubernetes/kubelet/config.json.d/00-nodeadm.conf` +- `maxPods: 100`์ด ์„ค์ •๋œ ํŒŒ์ผ ํ™•์ธ +- ๊ธฐ๋ณธ ๊ฐ’(17)์ด **์˜ค๋ฒ„๋ผ์ด๋“œ๋˜์–ด 100์œผ๋กœ ์ ์šฉ๋จ** + +```bash +for i in $N1 $N2 $N3; do echo ">> node $i <<"; ssh ec2-user@$i sudo cat /etc/kubernetes/kubelet/config.json.d/00-nodeadm.conf | grep maxPods; echo; done +>> node 3.35.47.226 << + "maxPods": 100 + +>> node 43.203.131.45 << + "maxPods": 100 + +>> node 15.164.49.232 << + "maxPods": 100 +``` + +--- + +## **์šด์˜์„œ๋ฒ„ EC2 : eks kubeconfig ์„ค์ •, EFS ๋งˆ์šดํŠธ ํ…Œ์ŠคํŠธ** + +### **1. ์šด์˜ ์„œ๋ฒ„ SSH ์ ‘์†** + +```bash +ssh -i kp-aews.pem ec2-user@$(aws cloudformation describe-stacks --stack-name myeks --query 'Stacks[*].Outputs[0].OutputValue' --output text) +Last login: Tue Feb 18 11:20:41 2025 from 182.230.60.93 + , #_ + ~\_ ####_ Amazon Linux 2 + ~~ \_#####\ + ~~ \###| AL2 End of Life is 2026-06-30. + ~~ \#/ ___ + ~~ V~' '-> + ~~~ / A newer version of Amazon Linux is available! + ~~._. _/ + _/ _/ Amazon Linux 2023, GA and supported until 2028-03-15. + _/m/' https://aws.amazon.com/linux/amazon-linux-2023/ + +Last login: Tue Feb 18 11:20:41 KST 2025 on pts/0 +[root@operator-host ~]# +``` + +### **2. AWS CLI ์ž๊ฒฉ์ฆ๋ช… ์„ค์ •** + +```bash +[root@operator-host ~]# aws configure +AWS Access Key ID [None]: XXXXXXXXXXXXXXXXXX +AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXX +Default region name [None]: ap-northeast-2 +Default output format [None]: json +``` +- **AWS Access Key ID**: (๋ฐœ๊ธ‰๋ฐ›์€ Access Key ID ์ž…๋ ฅ) +- **AWS Secret Access Key**: (Secret Access Key ์ž…๋ ฅ) +- **Default region name**:ย `ap-northeast-2`ย (์„œ์šธ ๋ฆฌ์ „, ์›ํ•˜๋Š” ๋ฆฌ์ „ ์„ ํƒ ๊ฐ€๋Šฅ) +- **Default output format**:ย `json`ย ๋˜๋Š”ย `yaml`ย (๊ธฐ๋ณธ๊ฐ’:ย `json`) + + +### **3. ํ˜„์žฌ IAM ์‚ฌ์šฉ์ž ํ™•์ธ** + +```bash +[root@operator-host ~]# aws sts get-caller-identity --query Arn + +# ๊ฒฐ๊ณผ +"arn:aws:iam::378102432899:user/eks-user" +``` + +### **4. `kubeconfig` ์ƒ์„ฑ ๋ฐ EKS ์—ฐ๊ฒฐ ์„ค์ •** + +- ์šด์˜ ์„œ๋ฒ„์—์„œ EKS ํด๋Ÿฌ์Šคํ„ฐ์™€ ์—ฐ๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด `kubeconfig` ํŒŒ์ผ์„ ์ƒ์„ฑ + +```bash +[root@operator-host ~]# aws eks update-kubeconfig --name myeks --user-alias eks-user +Added new context eks-user to /root/.kube/config +(eks-user:N/A) [root@operator-host ~]# +``` + +- `kubectl`์„ ์‚ฌ์šฉํ•˜์—ฌ ํด๋Ÿฌ์Šคํ„ฐ ์ •๋ณด ํ™•์ธ + +```bash +(eks-user:N/A) [root@operator-host ~]# kubectl cluster-info +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Kubernetes control plane is running at https://791BC5A9BB3716EA88C45304E0696F83.gr7.ap-northeast-2.eks.amazonaws.com +CoreDNS is running at https://791BC5A9BB3716EA88C45304E0696F83.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy + +To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. +``` + +- ๋„ค์ž„์ŠคํŽ˜์ด์Šค ๋ณ€๊ฒฝ +- ๊ธฐ๋ณธ ๋„ค์ž„์ŠคํŽ˜์ด์Šค๋ฅผ `default`๋กœ ์„ค์ • + +```bash +(eks-user:N/A) [root@operator-host ~]# kubectl ns default +Context "eks-user" modified. +Active namespace is "default". +(eks-user:default) [root@operator-host ~]# +``` + +### **5. EKS ํด๋Ÿฌ์Šคํ„ฐ์˜ ๋…ธ๋“œ ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# kubectl get node -v6 +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +I0218 13:47:55.441335 2910 loader.go:395] Config loaded from file: /root/.kube/config +I0218 13:47:56.203890 2910 round_trippers.go:553] GET https://791BC5A9BB3716EA88C45304E0696F83.gr7.ap-northeast-2.eks.amazonaws.com/api/v1/nodes?limit=500 200 OK in 755 milliseconds +NAME STATUS ROLES AGE VERSION +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 121m v1.31.5-eks-5d632ec +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 121m v1.31.5-eks-5d632ec +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 121m v1.31.5-eks-5d632ec +``` + +### **6. EFS ๋งˆ์šดํŠธ ํ…Œ์ŠคํŠธ ์ค€๋น„** + +- ์šด์˜ ์„œ๋ฒ„(172.20.1.100/24)๊ฐ€ **VPC Peering**์„ ํ†ตํ•ด EFS๋ฅผ ์›๊ฒฉ ๋งˆ์šดํŠธํ•˜์—ฌ ์Šคํ† ๋ฆฌ์ง€ ์‚ฌ์šฉ ์˜ˆ์ • +- ํ˜„์žฌ ์šด์˜ ์„œ๋ฒ„์™€ EKS ํด๋Ÿฌ์Šคํ„ฐ๊ฐ€ **๋‹ค๋ฅธ VPC์— ์žˆ์œผ๋ฏ€๋กœ**, VPC Peering์„ ํ†ตํ•ด ์šด์˜ ์„œ๋ฒ„์—์„œ EFS์— ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ๋„๋ก ์„ค์ •ํ•  ๊ฒƒ + +![Image](https://github.com/user-attachments/assets/a7dccf44-06a6-4eab-9913-60d5aebb77ab) + + +### 7. ํŒŒ์ผ ์‹œ์Šคํ…œ ID ํ™•์ธ + +- ํ˜„์žฌ ์‚ฌ์šฉ ์ค‘์ธ **EFS ํŒŒ์ผ ์‹œ์Šคํ…œ ID** ํ™•์ธ + +```bash +(eks-user:default) [root@operator-host ~]# aws efs describe-file-systems --query "FileSystems[*].FileSystemId" --output text +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +fs-0aeb6f8c0c228b9d2 +``` + +![Image](https://github.com/user-attachments/assets/4a04c5ae-b981-463c-8c25-33bddadc701c) + + +### **8. EFS ๋งˆ์šดํŠธ ๋Œ€์ƒ ์ •๋ณด ํ™•์ธ** + +- EFS์— ์—ฐ๊ฒฐ ๊ฐ€๋Šฅํ•œ **๋งˆ์šดํŠธ ํƒ€๊ฒŸ(Subnet ๋ฐ IP) ์กฐํšŒ** + +```bash +(eks-user:default) [root@operator-host ~]# aws efs describe-mount-targets --file-system-id $(aws efs describe-file-systems --query "FileSystems[*].FileSystemId" --output text) | jq +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +{ + "MountTargets": [ + { + "OwnerId": "378102432899", + "MountTargetId": "fsmt-0604d5d000fcd5bac", + "FileSystemId": "fs-0aeb6f8c0c228b9d2", + "SubnetId": "subnet-0e4fb63cb543698fe", + "LifeCycleState": "available", + "IpAddress": "192.168.2.121", + "NetworkInterfaceId": "eni-01a6638b93a0f3c69", + "AvailabilityZoneId": "apne2-az2", + "AvailabilityZoneName": "ap-northeast-2b", + "VpcId": "vpc-0e32b5a6653acdcd9" + }, + { + "OwnerId": "378102432899", + "MountTargetId": "fsmt-06d0ec46b5d5eb2e7", + "FileSystemId": "fs-0aeb6f8c0c228b9d2", + "SubnetId": "subnet-0fed28a1b3e108719", + "LifeCycleState": "available", + "IpAddress": "192.168.1.145", + "NetworkInterfaceId": "eni-081306b696d34563b", + "AvailabilityZoneId": "apne2-az1", + "AvailabilityZoneName": "ap-northeast-2a", + "VpcId": "vpc-0e32b5a6653acdcd9" + }, + { + "OwnerId": "378102432899", + "MountTargetId": "fsmt-079d4fd81a4d39374", + "FileSystemId": "fs-0aeb6f8c0c228b9d2", + "SubnetId": "subnet-0861bd68771150000", + "LifeCycleState": "available", + "IpAddress": "192.168.3.250", + "NetworkInterfaceId": "eni-05a1c8ad9d2d52f44", + "AvailabilityZoneId": "apne2-az3", + "AvailabilityZoneName": "ap-northeast-2c", + "VpcId": "vpc-0e32b5a6653acdcd9" + } + ] +} +``` + +### **9. ๋งˆ์šดํŠธ ๋Œ€์ƒ IP๋งŒ ์ถœ๋ ฅ** + +- ๋งˆ์šดํŠธ ๊ฐ€๋Šฅํ•œ **EFS ๋„คํŠธ์›Œํฌ ์ธํ„ฐํŽ˜์ด์Šค IP ๋ชฉ๋ก ์กฐํšŒ** + +```bash +(eks-user:default) [root@operator-host ~]# aws efs describe-mount-targets --file-system-id $(aws efs describe-file-systems --query "FileSystems[*].FileSystemId" --output text) --query "MountTargets[*].IpAddress" --output text +192.168.2.121 192.168.1.145 192.168.3.250 +``` + +![Image](https://github.com/user-attachments/assets/a4619d15-cc10-465a-a186-61c8e7e810e0) + + +### **10. VPC Peering ํ™˜๊ฒฝ์—์„œ ๋งˆ์šดํŠธ ๋ฐฉ์‹ ๊ฒฐ์ •** + +- **๊ฐ™์€ VPC ๋‚ด๋ถ€๋ผ๋ฉด** `fs-xxxx.efs.ap-northeast-2.amazonaws.com` ํ˜•์‹์˜ **DNS ๊ธฐ๋ฐ˜ ๋งˆ์šดํŠธ ๊ฐ€๋Šฅ** +- **๋‹ค๋ฅธ VPC์—์„œ VPC Peering์„ ํ†ตํ•ด ์ ‘๊ทผํ•˜๋Š” ๊ฒฝ์šฐ** โ†’ **IP ๊ธฐ๋ฐ˜ ๋งˆ์šดํŠธ ํ•„์š”** + +```bash +# DNS ์งˆ์˜ ํ…Œ์ŠคํŠธ: ๊ฐ™์€ VPC๊ฐ€ ์•„๋‹ˆ๋ฏ€๋กœ DNS ์งˆ์˜ ์‹คํŒจ +(eks-user:default) [root@operator-host ~]# dig +short $(aws efs describe-file-systems --query "FileSystems[*].FileSystemId" --output text).efs.ap-northeast-2.amazonaws.com +``` + +### **11. EFS ๋งˆ์šดํŠธ IP ์„ ํƒ ๋ฐ ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •** + +- ๋งˆ์šดํŠธํ•  **IP ์ฃผ์†Œ(์˜ˆ: 192.168.1.145)** ์„ ํƒ ํ›„ ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์ €์žฅ + +```bash +(eks-user:default) [root@operator-host ~]# EFSIP1=192.168.1.145 +``` + +### **12. EFS ๋งˆ์šดํŠธ ์ˆ˜ํ–‰** + +- ์šด์˜ ์„œ๋ฒ„์—์„œ **EFS๋ฅผ ๋งˆ์šดํŠธํ•  ๋””๋ ‰ํ„ฐ๋ฆฌ ์ƒ์„ฑ** +- `192.168.1.145:/mnt/myefs` ๊ฒฝ๋กœ๋กœ ๋งˆ์šดํŠธ๋จ + +```bash +(eks-user:default) [root@operator-host ~]# mkdir /mnt/myefs +(eks-user:default) [root@operator-host ~]# mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport $EFSIP1:/ /mnt/myefs +``` + +- **AWS ์ฝ˜์†” โ†’ EFS โ†’ Attach ์˜ต์…˜**์—์„œ **๋งˆ์šดํŠธ ๋ฐฉ๋ฒ• ๋ฐ ๋ช…๋ น์–ด ์ œ๊ณต** + +![Image](https://github.com/user-attachments/assets/f3be2fab-0a0c-4fc0-8b67-91e6b642393b) + +- `sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 192.168.1.145:/ efs` + +![Image](https://github.com/user-attachments/assets/ef2f670d-33d3-4905-9fc0-bee26fe489e1) + + +### **13. ๋งˆ์šดํŠธ ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# findmnt -t nfs4 +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +TARGET SOURCE FSTYPE OPTIONS +/mnt/myefs 192.168.1.145:/ nfs4 rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,noresvport,proto=tcp,timeo=600,retrans=2,sec=sy +``` + +**NFS4 ํƒ€์ž… ํŒŒ์ผ ์‹œ์Šคํ…œ๋งŒ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# df -hT --type nfs4 +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +192.168.1.145:/ nfs4 8.0E 0 8.0E 0% /mnt/myefs +``` + +### **14. EFS์— ํŒŒ์ผ ์ €์žฅ ๋ฐ NFS ํ†ต๊ณ„ ํ™•์ธ** + +**(1) ํŒŒ์ผ ์ €์žฅ ์ „ NFS ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# nfsstat +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Client rpc stats: +calls retrans authrefrsh +22 0 22 + +Client nfs v4: +null read write commit open open_conf +1 4% 0 0% 0 0% 0 0% 0 0% 0 0% +open_noat open_dgrd close setattr fsinfo renew +0 0% 0 0% 0 0% 0 0% 2 9% 0 0% +setclntid confirm lock lockt locku access +0 0% 0 0% 0 0% 0 0% 0 0% 0 0% +getattr lookup lookup_root remove rename link +2 9% 0 0% 1 4% 0 0% 0 0% 0 0% +symlink create pathconf statfs readlink readdir +0 0% 0 0% 1 4% 1 4% 0 0% 0 0% +server_caps delegreturn getacl setacl fs_locations rel_lkowner +3 13% 0 0% 0 0% 0 0% 0 0% 0 0% +secinfo exchange_id create_ses destroy_ses sequence get_lease_t +0 0% 0 0% 2 9% 1 4% 0 0% 6 27% +reclaim_comp layoutget getdevinfo layoutcommit layoutreturn getdevlist +0 0% 1 4% 0 0% 0 0% 0 0% 0 0% +(null) +1 4% +``` + +- ์ด ์š”์ฒญ(Call) ์ˆ˜: `22` +- `write` ์š”์ฒญ ์—†์Œ + +**(2) ํŒŒ์ผ ์ €์žฅ ์‹คํ–‰** + +- EFS ๋งˆ์šดํŠธ ๋””๋ ‰ํ† ๋ฆฌ(`/mnt/myefs`)์— ํŒŒ์ผ ์ƒ์„ฑ +- ์›๊ฒฉ NFS ์ €์žฅ์†Œ์— ํŒŒ์ผ(`memo.txt`) ์ƒ์„ฑ๋จ + +```bash +(eks-user:default) [root@operator-host ~]# echo "EKS Workshop" > /mnt/myefs/memo.txt +``` + +**(3) ํŒŒ์ผ ์ €์žฅ ํ›„ NFS ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# nfsstat +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Client rpc stats: +calls retrans authrefrsh +27 0 27 + +Client nfs v4: +null read write commit open open_conf +1 3% 0 0% 1 3% 0 0% 1 3% 0 0% +open_noat open_dgrd close setattr fsinfo renew +0 0% 0 0% 1 3% 0 0% 2 7% 0 0% +setclntid confirm lock lockt locku access +0 0% 0 0% 0 0% 0 0% 0 0% 1 3% +getattr lookup lookup_root remove rename link +2 7% 0 0% 1 3% 0 0% 0 0% 0 0% +symlink create pathconf statfs readlink readdir +0 0% 0 0% 1 3% 1 3% 0 0% 0 0% +server_caps delegreturn getacl setacl fs_locations rel_lkowner +3 11% 0 0% 0 0% 0 0% 0 0% 0 0% +secinfo exchange_id create_ses destroy_ses sequence get_lease_t +0 0% 0 0% 2 7% 1 3% 0 0% 7 25% +reclaim_comp layoutget getdevinfo layoutcommit layoutreturn getdevlist +0 0% 1 3% 0 0% 0 0% 0 0% 0 0% +(null) +1 3% +``` + +- ์ด ์š”์ฒญ(Call) ์ˆ˜: `27` +- `write` ์š”์ฒญ 1ํšŒ ๊ธฐ๋ก๋จ + +### **15. ์ €์žฅ๋œ ํŒŒ์ผ ํ™•์ธ** + +**(1) ์ €์žฅ๋œ ํŒŒ์ผ ๋ชฉ๋ก ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# ls -l /mnt/myefs +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +total 4 +-rw-r--r-- 1 root root 13 Feb 18 14:09 memo.txt +``` + +**(2) ํŒŒ์ผ ๋‚ด์šฉ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# cat /mnt/myefs/memo.txt +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +EKS Workshop +``` + +### **16. EFS ์ž๋™ ๋งˆ์šดํŠธ ์„ค์ • (EC2 ์žฌ๋ถ€ํŒ… ํ›„ ์œ ์ง€)** + +**(1) ํ˜„์žฌ `/etc/fstab` ์ƒํƒœ ํ™•์ธ** + +- ๊ธฐ์กด ๋งˆ์šดํŠธ ์„ค์ • ํ™•์ธ + +```bash +(eks-user:default) [root@operator-host ~]# cat /etc/fstab +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +UUID=43b4f483-987f-429f-ad61-9e2993518248 / xfs defaults,noatime 1 1 +``` + +**(2) `/etc/fstab` ํŒŒ์ผ ์ˆ˜์ • (EFS ์ž๋™ ๋งˆ์šดํŠธ ์ถ”๊ฐ€)** + +- **`fstab` ํŒŒ์ผ ์—ด๊ธฐ** + +```bash +sudo vim /etc/fstab +``` + +- **ํŒŒ์ผ ๋งจ ์•„๋ž˜์— EFS ๋งˆ์šดํŠธ ์ •๋ณด ์ถ”๊ฐ€** + +```bash +192.168.1.145:/ /mnt/myefs nfs4 defaults,_netdev,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0 +``` + +![Image](https://github.com/user-attachments/assets/344434e5-3431-4241-a8ac-c527cc875b03) + + +**(3) ๋ณ€๊ฒฝ ์‚ฌํ•ญ ์ ์šฉ** + +- ์ˆ˜์ •ํ•œ `/etc/fstab` ๋‚ด์šฉ์„ ์ฆ‰์‹œ ๋ฐ˜์˜ + +```bash +(eks-user:default) [root@operator-host ~]# sudo mount -a +``` + +**(4) ๋งˆ์šดํŠธ ์ƒํƒœ ํ™•์ธ** + +- ๋งˆ์šดํŠธ๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ์ ์šฉ๋˜์—ˆ๋Š”์ง€ ํ™•์ธ + +```bash +(eks-user:default) [root@operator-host ~]# df -hT +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +devtmpfs devtmpfs 981M 0 981M 0% /dev +tmpfs tmpfs 990M 0 990M 0% /dev/shm +tmpfs tmpfs 990M 432K 989M 1% /run +tmpfs tmpfs 990M 0 990M 0% /sys/fs/cgroup +/dev/xvda1 xfs 30G 3.0G 28G 10% / +tmpfs tmpfs 198M 0 198M 0% /run/user/1000 +192.168.1.145:/ nfs4 8.0E 0 8.0E 0% /mnt/myefs +``` + +--- + +## ๐Ÿ—๏ธ **EKS ๋ฐฐํฌ ํ›„ ์‹ค์Šต์„ ์œ„ํ•œ ํŽธ์˜ ์„ค์ •** + +### **1. ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์ž๋™ ์„ค์ • ๋ฐฐ๊ฒฝ** + +- **์ด์ „ ์‹ค์Šต์—์„œ ํ™˜๊ฒฝ ๋ณ€์ˆ˜๋ฅผ ๋งค๋ฒˆ ์ˆ˜๋™์œผ๋กœ ์„ค์ •ํ•ด์•ผ ํ•˜๋Š” ๋ถˆํŽธํ•จ**์ด ์žˆ์—ˆ์Œ +- ์‹ ๊ทœ ํ„ฐ๋ฏธ๋„ ์ฐฝ์„ ์—ด ๋•Œ๋งˆ๋‹ค **ํ™˜๊ฒฝ ๋ณ€์ˆ˜๊ฐ€ ์œ ์ง€๋˜๋„๋ก ์ž๋™ ์„ค์ •** + +### **2. ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ • (`~/.bashrc`์— ์ถ”๊ฐ€)** + +- **Route 53 ๋„๋ฉ”์ธ ๋ฐ Hosted Zone ID ๊ฐ€์ ธ์˜ค๊ธฐ** + +```bash +MyDomain=gagajin.com +MyDnzHostedZoneId=$(aws route53 list-hosted-zones-by-name --dns-name "$MyDomain." --query "HostedZones[0].Id" --output text) +``` + +- **ํ™˜๊ฒฝ ๋ณ€์ˆ˜๋ฅผ `~/.bashrc`์— ์ถ”๊ฐ€ํ•˜์—ฌ ์ƒˆ ํ„ฐ๋ฏธ๋„์—์„œ๋„ ์œ ์ง€** + +```bash +cat << EOF >> ~/.bashrc + +# eksworkshop +export CLUSTER_NAME=myeks +export VPCID=$(aws ec2 describe-vpcs --filters "Name=tag:Name,Values=$CLUSTER_NAME-VPC" --query 'Vpcs[*].VpcId' --output text) +export PubSubnet1=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet1" --query "Subnets[0].[SubnetId]" --output text) +export PubSubnet2=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet2" --query "Subnets[0].[SubnetId]" --output text) +export PubSubnet3=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet3" --query "Subnets[0].[SubnetId]" --output text) +export N1=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=$CLUSTER_NAME-ng1-Node" "Name=availability-zone,Values=ap-northeast-2a" --query 'Reservations[*].Instances[*].PublicIpAddress' --output text) +export N2=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=$CLUSTER_NAME-ng1-Node" "Name=availability-zone,Values=ap-northeast-2b" --query 'Reservations[*].Instances[*].PublicIpAddress' --output text) +export N3=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=$CLUSTER_NAME-ng1-Node" "Name=availability-zone,Values=ap-northeast-2c" --query 'Reservations[*].Instances[*].PublicIpAddress' --output text) +MyDomain=gagajin.com # ๊ฐ์ž ์ž์‹ ์˜ ๋„๋ฉ”์ธ ์ด๋ฆ„ ์ž…๋ ฅ +MyDnzHostedZoneId=$(aws route53 list-hosted-zones-by-name --dns-name "$MyDomain." --query "HostedZones[0].Id" --output text) +EOF +``` + +### **3. ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์ ์šฉ ํ™•์ธ** + +- **์‹ ๊ทœ ํ„ฐ๋ฏธ๋„์—์„œ ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ํ™•์ธ** + +```bash +echo $CLUSTER_NAME $VPCID $PubSubnet1 $PubSubnet2 $PubSubnet3 +echo $N1 $N2 $N3 $MyDomain $MyDnzHostedZoneId +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +myeks vpc-0e32b5a6653acdcd9 subnet-0fed28a1b3e108719 subnet-0e4fb63cb543698fe subnet-0861bd68771150000 +3.35.47.226 43.203.131.45 15.164.49.232 gagajin.com /hostedzone/Z099663315X74TRCYB7J5 +``` + +- ํ™˜๊ฒฝ ๋ณ€์ˆ˜๊ฐ€ `~/.bashrc`์— ์ •์ƒ์ ์œผ๋กœ ์ถ”๊ฐ€๋˜์—ˆ๋Š”์ง€ ํ™•์ธ + +```bash +tail -n 12 ~/.bashrc +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +# eksworkshop +export CLUSTER_NAME=myeks +export VPCID=vpc-0e32b5a6653acdcd9 +export PubSubnet1=subnet-0fed28a1b3e108719 +export PubSubnet2=subnet-0e4fb63cb543698fe +export PubSubnet3=subnet-0861bd68771150000 +export N1=3.35.47.226 +export N2=43.203.131.45 +export N3=15.164.49.232 +MyDomain=gagajin.com # ๊ฐ์ž ์ž์‹ ์˜ ๋„๋ฉ”์ธ ์ด๋ฆ„ ์ž…๋ ฅ +MyDnzHostedZoneId=/hostedzone/Z099663315X74TRCYB7J5 +``` + +### **4. ์‹ค์Šต ํ›„ ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์‚ญ์ œ ํ•„์š”** + +- **์ƒˆ๋กœ์šด ํ„ฐ๋ฏธ๋„ ์ฐฝ์—์„œ๋„ ํ™˜๊ฒฝ ๋ณ€์ˆ˜๊ฐ€ ์ž๋™ ์„ค์ •๋˜์–ด ํŽธ๋ฆฌํ•˜๊ฒŒ ์‚ฌ์šฉ ๊ฐ€๋Šฅ** +- **์‹ค์Šต์ด ๋๋‚˜๋ฉด `~/.bashrc`์—์„œ ํ•ด๋‹น ๋‚ด์šฉ์„ ์‚ญ์ œํ•  ๊ฒƒ** + +--- + +## **๐ŸŒ AWS LoadBalancerController, ExternalDNS, kube-ops-view ์„ค์น˜** + +### **1. Helm ์ €์žฅ์†Œ ์ถ”๊ฐ€ ๋ฐ ์—…๋ฐ์ดํŠธ** + +```bash +helm repo add geek-cookbook https://geek-cookbook.github.io/charts/ +helm repo add eks https://aws.github.io/eks-charts +helm repo update +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +"geek-cookbook" already exists with the same configuration, skipping +"eks" already exists with the same configuration, skipping +Hang tight while we grab the latest from your chart repositories... +...Successfully got an update from the "eks" chart repository +...Successfully got an update from the "prometheus-community" chart repository +...Successfully got an update from the "geek-cookbook" chart repository +Update Complete. โŽˆHappy Helming!โŽˆ +``` + +### **2. kube-ops-view ์„ค์น˜** + +```bash +helm install kube-ops-view geek-cookbook/kube-ops-view --version 1.2.2 --set service.main.type=ClusterIP --set env.TZ="Asia/Seoul" --namespace kube-system +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAME: kube-ops-view +LAST DEPLOYED: Tue Feb 18 14:43:54 2025 +NAMESPACE: kube-system +STATUS: deployed +REVISION: 1 +TEST SUITE: None +NOTES: +1. Get the application URL by running these commands: + export POD_NAME=$(kubectl get pods --namespace kube-system -l "app.kubernetes.io/name=kube-ops-view,app.kubernetes.io/instance=kube-ops-view" -o jsonpath="{.items[0].metadata.name}") + echo "Visit http://127.0.0.1:8080 to use your application" + kubectl port-forward $POD_NAME 8080:8080 +``` + +### **3. AWS LoadBalancerController ๋ฐฐํฌ** + +```bash +helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system --set clusterName=$CLUSTER_NAME \ + --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +NAME: aws-load-balancer-controller +LAST DEPLOYED: Tue Feb 18 14:46:39 2025 +NAMESPACE: kube-system +STATUS: deployed +REVISION: 1 +TEST SUITE: None +NOTES: +AWS Load Balancer controller installed! +``` + +### **4. ExternalDNS ์ปจํŠธ๋กค๋Ÿฌ ์„ค์น˜** + +๋„๋ฉ”์ธ๊ณผ Route 53 Hosted Zone์„ ์—ฐ๋™ํ•˜์—ฌ ์ž๋™์œผ๋กœ DNS ๋ ˆ์ฝ”๋“œ ๊ด€๋ฆฌ + +```bash +curl -s https://raw.githubusercontent.com/gasida/PKOS/main/aews/externaldns.yaml | MyDomain=$MyDomain MyDnzHostedZoneId=$MyDnzHostedZoneId envsubst | kubectl apply -f - +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +serviceaccount/external-dns created +clusterrole.rbac.authorization.k8s.io/external-dns created +clusterrolebinding.rbac.authorization.k8s.io/external-dns-viewer created +deployment.apps/external-dns created +``` + +### **5. HTTPS ์„ค์ •์„ ์œ„ํ•œ ์ธ์ฆ์„œ ๋ฐœ๊ธ‰ (AWS Certificate Manager)** + +**(1) ๋„๋ฉ”์ธ ์ธ์ฆ์„œ ๋ฐœ๊ธ‰ ์š”์ฒญ** + +**์š”์ฒญ ํ›„, Create records in Route 53 ํด๋ฆญํ•˜์—ฌ CNAME ๋ ˆ์ฝ”๋“œ ์ถ”๊ฐ€** + + +![Image](https://github.com/user-attachments/assets/38c0c2c7-9f90-4e17-a4af-094f1d731555) + +**Route 53์— CNAME ๋ ˆ์ฝ”๋“œ๊ฐ€ ์ž๋™์œผ๋กœ ์ถ”๊ฐ€๋จ** + + +![Image](https://github.com/user-attachments/assets/d9d4a2a1-7a96-41a7-8e02-0d9e56bb90a6) + +**์ธ์ฆ์„œ ์ƒํƒœ๊ฐ€ "Issued"๋กœ ๋ณ€๊ฒฝ๋˜๋ฉด ์™„๋ฃŒ** + +![Image](https://github.com/user-attachments/assets/285a05b2-ede3-4314-9120-77571ea1ca23) + + +**(2) ์ธ์ฆ์„œ ARN ํ™•์ธ** + +```bash +CERT_ARN=$(aws acm list-certificates --query 'CertificateSummaryList[].CertificateArn[]' --output text) +echo $CERT_ARN +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +arn:aws:acm:ap-northeast-2:378102432899:certificate/f967e8ca-f0b5-471d-bbe4-bee231aeb32b +``` + +### **6. ALB ๊ธฐ๋ฐ˜ Ingress ์„ค์ • ๋ฐ kube-ops-view ๋ฐฐํฌ** + +**(1) Ingress ๋ฐฐํฌ** + +- **ALB ๊ทธ๋ฃน ์ด๋ฆ„์„ ์ถ”๊ฐ€ํ•˜์—ฌ ์—ฌ๋Ÿฌ Ingress์—์„œ ๋™์ผํ•œ ALB ์‚ฌ์šฉ ๊ฐ€๋Šฅ** +- Ingress๋ฅผ ๋ฐฐํฌํ•˜์—ฌ HTTPS ๊ธฐ๋ฐ˜์œผ๋กœ ๋„๋ฉ”์ธ(`kubeopsview.$MyDomain`) ์—ฐ๊ฒฐ + +```bash +cat < 443/TCP 75m +service/eks-extension-metrics-api ClusterIP 10.100.124.121 443/TCP 4h24m +service/kube-dns ClusterIP 10.100.0.10 53/UDP,53/TCP,9153/TCP 4h20m +service/kube-ops-view ClusterIP 10.100.136.90 8080/TCP 77m +service/metrics-server ClusterIP 10.100.110.47 443/TCP 4h20m + +NAME ENDPOINTS AGE +endpoints/aws-load-balancer-webhook-service 192.168.1.218:9443,192.168.2.198:9443 75m +endpoints/eks-extension-metrics-api 172.0.32.0:10443 4h24m +endpoints/kube-dns 192.168.3.104:53,192.168.3.131:53,192.168.3.104:53 + 3 more... 4h20m +endpoints/kube-ops-view 192.168.1.204:8080 77m +endpoints/metrics-server 192.168.3.152:10251,192.168.3.77:10251 4h20m +``` + +**(4) kube-ops-view ์ ‘์† URL ํ™•์ธ** + +```bash +echo -e "Kube Ops View URL = https://kubeopsview.$MyDomain/#scale=1.5" +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Kube Ops View URL = https://kubeopsview.gagajin.com/#scale=1.5 +``` + +**(5) ์ ‘์† ๊ฒฐ๊ณผ** + +![Image](https://github.com/user-attachments/assets/f791a085-fef5-444a-b064-82274f71e8a9) + +--- + +## **๐Ÿ“ฆ Pod ๊ธฐ๋ณธ ์ €์žฅ์†Œ๋ฅผ ํ™œ์šฉํ•œ ๋ฐ์ดํ„ฐ ์œ ์ง€ ํ…Œ์ŠคํŠธ** + +### **1. EBS ์ปจํŠธ๋กค๋Ÿฌ ๋ฐฐํฌ ์ „ ์ƒํƒœ ํ™•์ธ** + +- **EBS ์ปจํŠธ๋กค๋Ÿฌ ๋ฐฐํฌ ์ „์—๋Š” `kubectl describe csinodes` ๋ช…๋ น์–ด ์‹คํ–‰ ์‹œ Spec ์ •๋ณด๊ฐ€ ํ‘œ์‹œ๋˜์ง€ ์•Š์Œ** +- **EBS ์ปจํŠธ๋กค๋Ÿฌ ๋ฐฐํฌ ํ›„์—๋Š” ๊ฐ ๋…ธ๋“œ์—์„œ ๊ด€๋ จ ์ •๋ณด๋ฅผ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Œ** + +```bash +kubectl describe csinodes +``` + +โœ…ย **์ถœ๋ ฅ** + +```bash +Name: ip-192-168-1-207.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:46 +0900 +Spec: +Events: + +Name: ip-192-168-2-84.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:49 +0900 +Spec: +Events: + +Name: ip-192-168-3-80.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:42 +0900 +Spec: +Events: +``` + +### **2. ํ„ฐ๋ฏธ๋„ ํ™˜๊ฒฝ ์ค€๋น„** + +- **์šด์˜ ์„œ๋ฒ„์™€ ๋กœ์ปฌ PC์—์„œ ๊ฐ๊ฐ ํ„ฐ๋ฏธ๋„ ์ฐฝ์„ ์—ด์–ด ๋ชจ๋‹ˆํ„ฐ๋ง** + +```bash +ssh -i kp-aews.pem ec2-user@$(aws cloudformation describe-stacks --stack-name myeks --query 'Stacks[*].Outputs[0].OutputValue' --output text) +# ๊ฒฐ๊ณผ +Last login: Tue Feb 18 16:22:43 2025 from 182.230.60.93 + , #_ + ~\_ ####_ Amazon Linux 2 + ~~ \_#####\ + ~~ \###| AL2 End of Life is 2026-06-30. + ~~ \#/ ___ + ~~ V~' '-> + ~~~ / A newer version of Amazon Linux is available! + ~~._. _/ + _/ _/ Amazon Linux 2023, GA and supported until 2028-03-15. + _/m/' https://aws.amazon.com/linux/amazon-linux-2023/ + +Last login: Tue Feb 18 16:22:43 KST 2025 on pts/0 +(eks-user:default) [root@operator-host ~]# +``` + +- **Pod ์ƒํƒœ ์‹ค์‹œ๊ฐ„ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# kubectl get pod -w +``` + +### **3. Redis ํŒŒ๋“œ ๋ฐฐํฌ** + +```bash +cat < /data/hello.txt" +``` + +**(3) ํŒŒ์ผ ๋‚ด์šฉ ํ™•์ธ** + +```bash +kubectl exec -it redis -- cat /data/hello.txt +# ๊ฒฐ๊ณผ +hello +``` + +### **5. Pod ์žฌ์‹œ์ž‘ ๋ฐ ๋ฐ์ดํ„ฐ ์œ ์‹ค ํ…Œ์ŠคํŠธ** + +**(1) ps ์„ค์น˜** + +```bash +kubectl exec -it redis -- sh -c "apt update && apt install procps -y" + +Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB] +Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB] +Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB] +Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8792 kB] +Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.5 kB] +Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [246 kB] +Fetched 9306 kB in 1s (6437 kB/s) +Reading package lists... Done +Building dependency tree... Done +.... +``` + +**(2) Redis ์ปจํ…Œ์ด๋„ˆ ํ”„๋กœ์„ธ์Šค ์ข…๋ฃŒ (kill 1)** + +- Pod ๋‚ด์—์„œ ์‹คํ–‰ ์ค‘์ธ ํ”„๋กœ์„ธ์Šค ํ™•์ธ (`redis-server`๊ฐ€ PID 1๋กœ ์‹คํ–‰๋จ) + +```bash +kubectl exec -it redis -- ps aux +``` + +โœ… **์ถœ๋ ฅ** + +```bash +USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND +redis 1 0.2 0.2 143876 10436 ? Ssl 07:27 0:01 redis-server +root 227 0.0 0.1 8088 4100 pts/0 Rs+ 07:34 0:00 ps aux +``` + +- PID 1(๋ฉ”์ธ ํ”„๋กœ์„ธ์Šค) ์ข…๋ฃŒ + +```bash +kubectl exec -it redis -- kill 1 +``` + +**(3) Pod ์ƒํƒœ ๋ณ€ํ™” ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# kubectl get pod -w +NAME READY STATUS RESTARTS AGE +redis 0/1 Pending 0 0s +redis 0/1 Pending 0 0s +redis 0/1 ContainerCreating 0 0s +redis 1/1 Running 0 9s +redis 0/1 Completed 0 9m2s +redis 1/1 Running 1 (3s ago) 9m4s +``` + +![Image](https://github.com/user-attachments/assets/9b7ab91f-6861-43ac-be84-011fefa5d8b8) + +![Image](https://github.com/user-attachments/assets/015f05ab-a750-4b95-98af-63751a3434cf) + + +- **Pod๊ฐ€ ์‚ญ์ œ๋˜์ง€ ์•Š๊ณ  ์ปจํ…Œ์ด๋„ˆ๋งŒ ์žฌ์‹œ์ž‘๋จ** +- **์ปจํ…Œ์ด๋„ˆ์˜ Restart Count ๊ฐ’์ด ์ฆ๊ฐ€ํ•จ (`RESTARTS: 1`)** + +**(4) Redis Pod์˜ ์žฌ์‹œ์ž‘ ์›์ธ ํ™•์ธ** + +```bash +k describe pod +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: redis +Namespace: default +Priority: 0 +Service Account: default +Node: ip-192-168-1-207.ap-northeast-2.compute.internal/192.168.1.207 +Start Time: Tue, 18 Feb 2025 16:27:11 +0900 +Labels: +Annotations: +Status: Running +IP: 192.168.1.234 +IPs: + IP: 192.168.1.234 +Containers: + redis: + Container ID: containerd://c76cf2a72506effed078046cd25d30bcfed857b594b228071541967ade2058c7 + Image: redis + Image ID: docker.io/library/redis@sha256:93a8d83b707d0d6a1b9186edecca2e37f83722ae0e398aee4eea0ff17c2fad0e + Port: + Host Port: + State: Running + Started: Tue, 18 Feb 2025 16:36:14 +0900 + Last State: Terminated + Reason: Completed + Exit Code: 0 + Started: Tue, 18 Feb 2025 16:27:19 +0900 + Finished: Tue, 18 Feb 2025 16:36:12 +0900 + Ready: True + Restart Count: 1 + Environment: + Mounts: + /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-dr52g (ro) +... +``` + +- **Redis ์ปจํ…Œ์ด๋„ˆ๊ฐ€ "Completed" ์ƒํƒœ๋กœ ์ข…๋ฃŒ๋จ โ†’ Kubernetes๊ฐ€ ์ž๋™์œผ๋กœ ์žฌ์‹œ์ž‘** +- **Pod ์ž์ฒด๋Š” ์œ ์ง€๋˜์ง€๋งŒ ์ปจํ…Œ์ด๋„ˆ ๋‚ด๋ถ€ ํ”„๋กœ์„ธ์Šค๊ฐ€ ์žฌ์‹œ์ž‘๋จ** + +**(5) Redis Pod ์žฌ์‹œ์ž‘ ํ›„ ๋ฐ์ดํ„ฐ ์œ ์‹ค ํ™•์ธ** + +- ๊ธฐ์กด์— ์ €์žฅํ•œ ํŒŒ์ผ ํ™•์ธ + +```bash +kubectl exec -it redis -- cat /data/redis/hello.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +cat: /data/redis/hello.txt: No such file or directory +command terminated with exit code 1 +``` + +- **Pod๊ฐ€ ์‚ญ์ œ๋˜์ง€ ์•Š์•˜์ง€๋งŒ, ์ปจํ…Œ์ด๋„ˆ ์žฌ์‹œ์ž‘์œผ๋กœ ์ธํ•ด ๋‚ด๋ถ€ ์ €์žฅ ๋ฐ์ดํ„ฐ๊ฐ€ ์œ ์‹ค๋จ** +- **๊ธฐ๋ณธ์ ์œผ๋กœ Pod ๋‚ด๋ถ€์˜ ํŒŒ์ผ ์‹œ์Šคํ…œ์€ ํœ˜๋ฐœ์„ฑ์ด๋ฉฐ, ์žฌ์‹œ์ž‘ ํ›„์—๋„ ๋ฐ์ดํ„ฐ๋ฅผ ์œ ์ง€ํ•˜๋ ค๋ฉด ๋ณผ๋ฅจ ์„ค์ •์ด ํ•„์š”** + +### **6. Redis Pod ์‚ญ์ œ** + +```bash +kubectl delete pod redis + +# ๊ฒฐ๊ณผ +pod "redis" deleted +``` + +--- + +## ๐Ÿ—‚๏ธ **emptyDir๋ฅผ ํ™œ์šฉํ•œ Pod ๋ฐ์ดํ„ฐ ์œ ์ง€ ํ…Œ์ŠคํŠธ** + +### **1. emptyDir ๋ณผ๋ฅจ์„ ํ™œ์šฉํ•œ Redis Pod ๋ฐฐํฌ** + +- Pod ๋‚ด์—์„œ `emptyDir` ๋ณผ๋ฅจ์„ ์ƒ์„ฑ ํ›„ `/data/redis` ๊ฒฝ๋กœ์— ๋งˆ์šดํŠธ +- Pod๊ฐ€ ์žฌ์‹œ์ž‘๋˜๋”๋ผ๋„ `emptyDir` ๋ณผ๋ฅจ์€ ์œ ์ง€๋จ + +```bash +cat < /data/redis/hello.txt" +``` + +**(2) ํŒŒ์ผ ๋‚ด์šฉ ํ™•์ธ** + +```bash +kubectl exec -it redis -- cat /data/redis/hello.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +hello +``` + +### **3. Pod ์žฌ์‹œ์ž‘ ๋ฐ ๋ฐ์ดํ„ฐ ์œ ์ง€ ํ™•์ธ** + +**(1) ps ์„ค์น˜** + +```bash +kubectl exec -it redis -- sh -c "apt update && apt install procps -y" + +Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB] +Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB] +Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB] +Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8792 kB] +Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.5 kB] +Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [246 kB] +Fetched 9306 kB in 2s (6071 kB/s) +Reading package lists... Done +Building dependency tree... Done +... +``` + +**(2) Redis ์ปจํ…Œ์ด๋„ˆ ํ”„๋กœ์„ธ์Šค ์ข…๋ฃŒ (kill 1)** + +- **Pod ๋‚ด์—์„œ ์‹คํ–‰ ์ค‘์ธ ํ”„๋กœ์„ธ์Šค ํ™•์ธ** + +```bash +kubectl exec -it redis -- ps aux + +USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND +redis 1 0.2 0.2 143876 10416 ? Ssl 07:50 0:00 redis +root 228 33.3 0.1 8088 4000 pts/0 Rs+ 07:53 0:00 ps au +``` + +- **Redis ํ”„๋กœ์„ธ์Šค ์ข…๋ฃŒ (Pod ์žฌ์‹œ์ž‘ ์œ ๋„)** + +```bash +kubectl exec -it redis -- kill 1 +``` + +**(3) Pod ์ƒํƒœ ํ™•์ธ** + +```bash +kubectl get pod +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME READY STATUS RESTARTS AGE +redis 1/1 Running 1 (17s ago) 4m52s +``` + +**(4) ํŒŒ์ผ ๋ฐ์ดํ„ฐ ์œ ์ง€ ์—ฌ๋ถ€ ํ™•์ธ** + +```bash +kubectl exec -it redis -- ls -l +``` + +โœ… **์ถœ๋ ฅ** + +```bash +total 0 +drwxrwxrwx. 2 redis root 23 Feb 18 07:51 redis +``` + +```bash +kubectl exec -it redis -- cat /data/redis/hello.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +hello +``` + +- **Pod๊ฐ€ ์žฌ์‹œ์ž‘๋˜๋”๋ผ๋„ `emptyDir` ๋ณผ๋ฅจ์€ ์œ ์ง€๋จ** +- **์ปจํ…Œ์ด๋„ˆ๊ฐ€ ์žฌ์‹œ์ž‘๋˜๋”๋ผ๋„ ๋ฐ์ดํ„ฐ๋Š” ๋ณด์กด๋จ** + +### **4. Pod ์‚ญ์ œ ๋ฐ ๋ฐ์ดํ„ฐ ์œ ์‹ค ํ…Œ์ŠคํŠธ** + +**(1) Redis Pod ์‚ญ์ œ** + +```bash +kubectl delete pod redis +# ๊ฒฐ๊ณผ +pod "redis" deleted +``` + +```bash +k get pod +``` + +โœ… **์ถœ๋ ฅ** + +```bash +No resources found in default namespace. +``` + +**(2) ์ƒˆ๋กœ์šด Redis Pod ์žฌ๋ฐฐํฌ** + +```bash +cat < +``` + +### **4. ConfigMap ์„ค์ • ํ™•์ธ** + +PV๊ฐ€ ๋™์ ์œผ๋กœ ์ƒ์„ฑ๋  ๊ฒฝ๋กœ๋ฅผ ์ง€์ •ํ•˜๋Š” ConfigMap ํ™•์ธ + +```bash +kubectl describe cm -n local-path-storage local-path-config +``` + +โœ… **์ถœ๋ ฅ (config.json)** + +```bash +config.json: +---- +{ + "nodePathMap":[ + { + "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES", + "paths":["/opt/local-path-provisioner"] + } + ] +} +``` + +- `/opt/local-path-provisioner` ํ•˜์œ„์— ๋™์  ๋””๋ ‰ํ† ๋ฆฌ ์ƒ์„ฑ ๋ฐ ๊ด€๋ฆฌ +- PVC ์š”์ฒญ ์‹œ ํ•ด๋‹น ๊ฒฝ๋กœ์— PV๊ฐ€ ์ž๋™ ์ƒ์„ฑ๋จ + +โœ… **์ถœ๋ ฅ (helperPod.yaml)** + +๋””๋ ‰ํ† ๋ฆฌ ์ƒ์„ฑ ๋ฐ ์‚ญ์ œ๋ฅผ ๊ด€๋ฆฌํ•˜๋Š” Pod ์„ค์ • + +```bash +helperPod.yaml: +---- +apiVersion: v1 +kind: Pod +metadata: + name: helper-pod +spec: + priorityClassName: system-node-critical + tolerations: + - key: node.kubernetes.io/disk-pressure + operator: Exists + effect: NoSchedule + containers: + - name: helper-pod + image: busybox + imagePullPolicy: IfNotPresent + +setup: +---- +#!/bin/sh +set -eu +mkdir -m 0777 -p "$VOL_DIR" + +teardown: +---- +#!/bin/sh +set -eu +rm -rf "$VOL_DIR" + +BinaryData +==== +``` + +- helperPod๋ฅผ ํ†ตํ•ด ๋™์  ๋””๋ ‰ํ† ๋ฆฌ ์ƒ์„ฑ (`setup` ์Šคํฌ๋ฆฝํŠธ) +- PV ์‚ญ์ œ ์‹œ ํ•ด๋‹น ๋””๋ ‰ํ† ๋ฆฌ๋„ ์ œ๊ฑฐ (`teardown` ์Šคํฌ๋ฆฝํŠธ) + +### **5. StorageClass ํ™•์ธ** + +```bash +kubectl get sc local-path +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +local-path rancher.io/local-path Delete WaitForFirstConsumer false 5m28s +``` + +### **6. PVC(Persistent Volume Claim) ์ƒ์„ฑ ๋ฐ ์ƒํƒœ ํ™•์ธ** + +**(1) PVC ์ƒํƒœ ๋ชจ๋‹ˆํ„ฐ๋ง** + +```bash +(eks-user:default) [root@operator-host ~]# watch -d kubectl get pv,pvc,pod +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Every 2.0s: kubectl get pv,pvc,pod Tue Feb 18 17:18:32 2025 + +No resources found +``` + +**(2) PVC ์ƒ์„ฑ์„ ์œ„ํ•œ ํ™˜๊ฒฝ ํ™•์ธ** + +```bash +k get pod -A +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAMESPACE NAME READY STATUS RESTARTS AGE +kube-system aws-load-balancer-controller-554fbd9d-vk2p8 1/1 Running 0 158m +kube-system aws-load-balancer-controller-554fbd9d-xnx6r 1/1 Running 0 158m +kube-system aws-node-rf9bf 2/2 Running 0 5h38m +kube-system aws-node-tbbhl 2/2 Running 0 5h38m +kube-system aws-node-xb7dt 2/2 Running 0 5h38m +kube-system coredns-86f5954566-mskq6 1/1 Running 0 5h44m +kube-system coredns-86f5954566-wxwqw 1/1 Running 0 5h44m +kube-system external-dns-dc4878f5f-mvnt9 1/1 Running 0 156m +kube-system kube-ops-view-657dbc6cd8-fgbqc 1/1 Running 0 161m +kube-system kube-proxy-6bc4m 1/1 Running 0 5h38m +kube-system kube-proxy-qsd8t 1/1 Running 0 5h38m +kube-system kube-proxy-rvw86 1/1 Running 0 5h38m +kube-system metrics-server-6bf5998d9c-nt4ks 1/1 Running 0 5h44m +kube-system metrics-server-6bf5998d9c-prz6f 1/1 Running 0 5h44m +local-path-storage local-path-provisioner-84967477f-g6xvh 1/1 Running 0 16m +``` + +- `local-path-provisioner` Pod๊ฐ€ ๋ฐฐํฌ๋˜์–ด ์žˆ์–ด์•ผ PVC๋ฅผ ๋™์ ์œผ๋กœ ๊ด€๋ฆฌํ•  ์ˆ˜ ์žˆ์Œ + +**(3) PVC(Persistent Volume Claim) ์ƒ์„ฑ** + +```bash +cat < 31s +``` + +![Image](https://github.com/user-attachments/assets/579de882-1e09-4991-871b-1a7aa118fc44) + + +**(5) StorageClass ํ™•์ธ** + +- **StorageClass์˜ `VOLUMEBINDINGMODE`๊ฐ€ `WaitForFirstConsumer` ์ƒํƒœ** +- **PVC๊ฐ€ Pod์— ์˜ํ•ด ์š”์ฒญ๋  ๋•Œ๊นŒ์ง€ PV๊ฐ€ ์ž๋™ ์ƒ์„ฑ๋˜์ง€ ์•Š์Œ** + +```bash +k get sc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 5h52m +local-path rancher.io/local-path Delete WaitForFirstConsumer false 20m +``` + +**(6) PVC ์ƒ์„ธ ์ •๋ณด ํ™•์ธ** + +- **PVC๊ฐ€ ๋ฐ”์ธ๋”ฉ๋˜์ง€ ์•Š์€ ์ด์œ  ๋ฐ ์ด๋ฒคํŠธ ํ™•์ธ** +- **Pod ์ƒ์„ฑ ์ „๊นŒ์ง€ PVC๋Š” `WaitForFirstConsumer` ์ƒํƒœ** + +```bash +kubectl describe pvc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: localpath-claim +Namespace: default +StorageClass: local-path +Status: Pending +Volume: +Labels: +Annotations: +Finalizers: [kubernetes.io/pvc-protection] +Capacity: +Access Modes: +VolumeMode: Filesystem +Used By: +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal WaitForFirstConsumer 6s (x15 over 3m25s) persistentvolume-controller waiting for first consumer to be created before binding +``` + +**(7) Pod ์ƒ์„ฑ ๋ฐ PVC ๋ฐ”์ธ๋”ฉ** + +- PVC๋ฅผ ์‚ฌ์šฉํ•˜๋Š” Pod ์ƒ์„ฑ +- Pod์—์„œ `localpath-claim` PVC๋ฅผ `/data` ๊ฒฝ๋กœ์— ๋งˆ์šดํŠธ + +```bash +cat <> /data/out.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: localpath-claim +EOF + +# ๊ฒฐ๊ณผ +pod/app created +``` + +- **Pod๊ฐ€ ์ƒ์„ฑ๋˜๋ฉด PVC๊ฐ€ ์ž๋™์œผ๋กœ PV์— ๋ฐ”์ธ๋”ฉ๋จ** + + +![Image](https://github.com/user-attachments/assets/836fb881-6286-4d61-b08d-5ce574ece21c) + +**(8) PVC ๋ฐ PV ๋ฐ”์ธ๋”ฉ ํ™•์ธ** + +- **PVC ์ƒํƒœ๊ฐ€ `Bound`๋กœ ๋ณ€๊ฒฝ๋จ** +- **PV๊ฐ€ ์ž๋™ ์ƒ์„ฑ๋˜์–ด PVC์™€ ์—ฐ๊ฒฐ๋จ** + +```bash +k get pod,pv,pvc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME READY STATUS RESTARTS AGE +pod/app 1/1 Running 0 109s + +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE +persistentvolume/pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018 1Gi RWO Delete Bound default/localpath-claim local-path 101s + +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +persistentvolumeclaim/localpath-claim Bound pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018 1Gi RWO local-path 11m +``` + +**(9) PV ์ƒ์„ธ ์ •๋ณด ํ™•์ธ** + +- PV๋Š” ํŠน์ • ๋…ธ๋“œ(`192.168.1.207`)**์— ๋ฐ”์ธ๋”ฉ๋จ (**`Node Affinity` ์„ค์ •) +- ์ €์žฅ์†Œ๋Š” `HostPath`๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ **๋…ธ๋“œ์˜ ํŠน์ • ๊ฒฝ๋กœ(`/opt/local-path-provisioner/...`)์— ์ €์žฅ๋จ** +- Pod๊ฐ€ ๋‹ค๋ฅธ ๋…ธ๋“œ๋กœ ์ด๋™ํ•˜๋ฉด ๊ธฐ์กด PV๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์—†์Œ + +```bash +kubectl describe pv +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018 +Labels: +Annotations: local.path.provisioner/selected-node: ip-192-168-1-207.ap-northeast-2.compute.internal + pv.kubernetes.io/provisioned-by: rancher.io/local-path +Finalizers: [kubernetes.io/pv-protection] +StorageClass: local-path +Status: Bound +Claim: default/localpath-claim +Reclaim Policy: Delete +Access Modes: RWO +VolumeMode: Filesystem +Capacity: 1Gi +Node Affinity: + Required Terms: + Term 0: kubernetes.io/hostname in [ip-192-168-1-207.ap-northeast-2.compute.internal] +Message: +Source: + Type: HostPath (bare host directory volume) + Path: /opt/local-path-provisioner/pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018_default_localpath-claim + HostPathType: DirectoryOrCreate +Events: +``` + +### **7. Pod ๋‚ด ๋ฐ์ดํ„ฐ ์ €์žฅ ๋ฐ ์˜์†์„ฑ ํ…Œ์ŠคํŠธ** + +**(1) ๋ฐ์ดํ„ฐ ์ €์žฅ ํ™•์ธ** + +- **Pod๋Š” 5์ดˆ๋งˆ๋‹ค `/data/out.txt`์— ํƒ€์ž„์Šคํƒฌํ”„ ๊ธฐ๋ก** +- **๋ฐ์ดํ„ฐ๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ์ €์žฅ๋˜๋Š”์ง€ ํ™•์ธ** + +```bash +kubectl exec -it app -- tail -f /data/out.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 08:48:00 UTC 2025 +Tue Feb 18 08:48:05 UTC 2025 +Tue Feb 18 08:48:10 UTC 2025 +Tue Feb 18 08:48:15 UTC 2025 +Tue Feb 18 08:48:20 UTC 2025 +Tue Feb 18 08:48:25 UTC 2025 +Tue Feb 18 08:48:30 UTC 2025 +Tue Feb 18 08:48:35 UTC 2025 +Tue Feb 18 08:48:40 UTC 2025 +Tue Feb 18 08:48:45 UTC 2025 +... +``` + +**(2) ๋ฐ์ดํ„ฐ ์ €์žฅ ์œ„์น˜ ํ™•์ธ** + +- **ํ˜„์žฌ ํด๋Ÿฌ์Šคํ„ฐ์— 3๊ฐœ์˜ ์›Œ์ปค ๋…ธ๋“œ ์กด์žฌ** +- **๊ฐ ๋…ธ๋“œ์˜ `/opt/local-path-provisioner` ๊ฒฝ๋กœ๋ฅผ ํ™•์ธํ•˜์—ฌ ๋ฐ์ดํ„ฐ ์ €์žฅ ์—ฌ๋ถ€ ํ™•์ธ** + +```bash +for node in $N1 $N2 $N3; do ssh ec2-user@$node tree /opt/local-path-provisioner; done +``` + +โœ… **์ถœ๋ ฅ** + +```bash +/opt/local-path-provisioner +โ””โ”€โ”€ pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018_default_localpath-claim + โ””โ”€โ”€ out.txt + +1 directory, 1 file +/opt/local-path-provisioner [error opening dir] + +0 directories, 0 files +/opt/local-path-provisioner [error opening dir] + +0 directories, 0 files +``` + +- Pod๊ฐ€ ๋ฐฐํฌ๋œ ๋…ธ๋“œ์—์„œ๋งŒ ๋ฐ์ดํ„ฐ๊ฐ€ ์ €์žฅ๋จ +- ๋‹ค๋ฅธ ๋…ธ๋“œ์—์„œ๋Š” ๋ฐ์ดํ„ฐ๊ฐ€ ์กด์žฌํ•˜์ง€ ์•Š์Œ + +**(3) ๋…ธ๋“œ ์ง์ ‘ ์ ‘๊ทผํ•˜์—ฌ ๋ฐ์ดํ„ฐ ํ™•์ธ** + +- **์„œ๋ฒ„์— ์ง์ ‘ ์ ‘๊ทผํ•˜์—ฌ `out.txt` ํŒŒ์ผ ๋‚ด์šฉ ํ™•์ธ** +- **Pod ๋‚ด์—์„œ ํ™•์ธํ•˜๋Š” ๊ฒƒ๊ณผ ๋™์ผํ•œ ๊ฒฐ๊ณผ ํ™•์ธ ๊ฐ€๋Šฅ** + +```bash +ssh ec2-user@$N1 tail -f /opt/local-path-provisioner/pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018_default_localpath-claim/out.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 08:53:46 UTC 2025 +Tue Feb 18 08:53:51 UTC 2025 +Tue Feb 18 08:53:56 UTC 2025 +Tue Feb 18 08:54:01 UTC 2025 +Tue Feb 18 08:54:06 UTC 2025 +Tue Feb 18 08:54:11 UTC 2025 +Tue Feb 18 08:54:16 UTC 2025 +Tue Feb 18 08:54:21 UTC 2025 +Tue Feb 18 08:54:26 UTC 2025 +... +``` + +### **8. Pod ์‚ญ์ œ ํ›„ ๋ฐ์ดํ„ฐ ์œ ์ง€ ํ™•์ธ** + +**(1) pod ์‚ญ์ œ** + +- Pod๋ฅผ ์‚ญ์ œํ•˜์—ฌ ๋ฐ์ดํ„ฐ๊ฐ€ ์œ ์ง€๋˜๋Š”์ง€ ํ™•์ธ + +```bash +kubectl delete pod app +# ๊ฒฐ๊ณผ +pod "app" deleted +``` + +**(2) pod ์‚ญ์ œ ํ›„ PVC ์ƒํƒœ ํ™•์ธ** + +```bash +kubectl get pod,pv,pvc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE +persistentvolume/pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018 1Gi RWO Delete Bound default/localpath-claim local-path 21m + +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +persistentvolumeclaim/localpath-claim Bound pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018 1Gi RWO local-path 31m +``` + +- **PVC์™€ PV๋Š” ์‚ญ์ œ๋˜์ง€ ์•Š๊ณ  ์œ ์ง€๋จ** +- **Pod๊ฐ€ ์—†์–ด๋„ ๋ฐ์ดํ„ฐ๋Š” ๋ณด์กด๋จ** + +**(3) ์„œ๋ฒ„์—์„œ ๋ฐ์ดํ„ฐ ์œ ์ง€ ์—ฌ๋ถ€ ํ™•์ธ** + +- **Pod๊ฐ€ ์‚ญ์ œ๋œ ํ›„์—๋„ ๋ฐ์ดํ„ฐ๊ฐ€ ์œ ์ง€๋˜๋Š”์ง€ ์ง์ ‘ ํ™•์ธ** +- **๊ฐ ๋…ธ๋“œ์˜ `/opt/local-path-provisioner` ๊ฒฝ๋กœ๋ฅผ ํ™•์ธํ•˜์—ฌ ํŒŒ์ผ ์กด์žฌ ์—ฌ๋ถ€ ํ™•์ธ** + +```bash +for node in $N1 $N2 $N3; do ssh ec2-user@$node tree /opt/local-path-provisioner; done +``` + +โœ… **์ถœ๋ ฅ** + +```bash +/opt/local-path-provisioner +โ””โ”€โ”€ pvc-fc043ac6-1fdc-4ef1-b03b-cabf03df8018_default_localpath-claim + โ””โ”€โ”€ out.txt + +1 directory, 1 file +/opt/local-path-provisioner [error opening dir] + +0 directories, 0 files +/opt/local-path-provisioner [error opening dir] + +0 directories, 0 files +``` + +- **Pod๊ฐ€ ๋ฐฐํฌ๋œ ๋…ธ๋“œ(1๋ฒˆ ์„œ๋ฒ„)์— ๋ฐ์ดํ„ฐ๊ฐ€ ์œ ์ง€๋จ** +- **Pod๊ฐ€ ์‚ญ์ œ๋˜์—ˆ์–ด๋„ ํŒŒ์ผ(`out.txt`)์ด ๋ณด์กด๋จ** + +**(4) Pod ์žฌ๋ฐฐํฌ** + +- Pod๋ฅผ ๋‹ค์‹œ ๋ฐฐํฌํ•˜์—ฌ ๊ธฐ์กด PVC์— ์—ฐ๊ฒฐ + +```bash +cat <> /data/out.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: localpath-claim +EOF + +# ๊ฒฐ๊ณผ +pod/app created +``` + +**(5) ๊ธฐ์กด ๋ฐ์ดํ„ฐ ์œ ์ง€ ์—ฌ๋ถ€ ํ™•์ธ** + +```bash +kubectl exec -it app -- tail -f /data/out.txt +Tue Feb 18 08:38:35 UTC 2025 +Tue Feb 18 08:38:40 UTC 2025 +Tue Feb 18 09:02:27 UTC 2025 +Tue Feb 18 09:02:32 UTC 2025 +Tue Feb 18 09:02:37 UTC 2025 +Tue Feb 18 09:02:42 UTC 2025 +Tue Feb 18 09:02:47 UTC 2025 +Tue Feb 18 09:02:52 UTC 2025 +Tue Feb 18 09:02:57 UTC 2025 +Tue Feb 18 09:03:02 UTC 2025 +Tue Feb 18 09:03:07 UTC 2025 +Tue Feb 18 09:03:12 UTC 2025 +... +``` + +- **Pod ์‚ญ์ œ ํ›„์—๋„ ๊ธฐ์กด ๋ฐ์ดํ„ฐ(`out.txt`)๊ฐ€ ์œ ์ง€๋จ** +- **emptyDir์™€ ๋‹ฌ๋ฆฌ PVC๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด Pod๊ฐ€ ์‚ญ์ œ๋˜์–ด๋„ ๋ฐ์ดํ„ฐ๊ฐ€ ๋ณด์กด๋จ** + +### **9. PVC ์‚ญ์ œ ํ›„ ๋ฐ์ดํ„ฐ ์™„์ „ ์‚ญ์ œ ํ™•์ธ** + +**(1) PVC ์‚ญ์ œ** + +```bash +kubectl delete pvc localpath-claim +``` + +โœ… **์ถœ๋ ฅ** + +```bash +persistentvolumeclaim "localpath-claim" deleted +``` + +**(2) PV ์ƒํƒœ ํ™•์ธ** + +```bash +kubectl get pv +``` + +โœ… **์ถœ๋ ฅ** + +```bash +No resources found +``` + +- **PVC๋ฅผ ์‚ญ์ œํ•˜๋ฉด PV๋„ ์ž๋™์œผ๋กœ ์‚ญ์ œ๋จ** +- **๋ฐ์ดํ„ฐ๋„ ํ•จ๊ป˜ ์‚ญ์ œ๋จ** + +**(3) ๋…ธ๋“œ์—์„œ ๋ฐ์ดํ„ฐ ์‚ญ์ œ ํ™•์ธ** + +```bash +for node in $N1 $N2 $N3; do ssh ec2-user@$node tree /opt/local-path-provisioner; done +``` + +โœ… **์ถœ๋ ฅ** + +```bash +/opt/local-path-provisioner + +0 directories, 0 files +/opt/local-path-provisioner [error opening dir] + +0 directories, 0 files +/opt/local-path-provisioner [error opening dir] + +0 directories, 0 files +``` + +- **PVC ์‚ญ์ œ ์‹œ PV๋„ ์‚ญ์ œ๋˜๋ฉฐ, ์ €์žฅ๋œ ๋ฐ์ดํ„ฐ๋„ ์™„์ „ํžˆ ์ œ๊ฑฐ๋จ** +- **PVC๋ฅผ ์œ ์ง€ํ•˜๋ฉด ๋ฐ์ดํ„ฐ ๋ณด์กด, ์‚ญ์ œํ•˜๋ฉด ๋ฐ์ดํ„ฐ ์™„์ „ ์‚ญ์ œ๋จ** + +--- + +## **๐Ÿ“Š ๋””์Šคํฌ ์„ฑ๋Šฅ ์ธก์ • ๋ฐ Kubestr ํ™œ์šฉ** + +### **1. ๋””์Šคํฌ ์„ฑ๋Šฅ ์ธก์ • ๊ฐœ์š”** + +**(1) ๋””์Šคํฌ ์„ฑ๋Šฅ ์ธก์ • ์‹œ ์ฃผ์š” ์ง€ํ‘œ** + +- **IOPS (Input/Output Operations Per Second)**: ์ดˆ๋‹น ์ฒ˜๋ฆฌํ•  ์ˆ˜ ์žˆ๋Š” I/O ์ž‘์—… ์ˆ˜ +- **Bandwidth (Throughput)**: ์ดˆ๋‹น ๋ฐ์ดํ„ฐ ์ „์†ก๋Ÿ‰ + +**(2) ํ˜„์žฌ ์›Œ์ปค ๋…ธ๋“œ์˜ ๋””์Šคํฌ ์ •๋ณด** + +- ์‚ฌ์šฉ ์ค‘์ธ ๋””์Šคํฌ: **AWS EBS gp3 ํƒ€์ž…** +- IOPS: **3000** +- Throughput: **125 MB/s** + + +![Image](https://github.com/user-attachments/assets/d5b970ac-dd72-449f-a466-22f6365106e0) + +### **2. Kubestr ์„ค์น˜** + +- **`Kubestr`: Kubernetes์—์„œ ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค๋ณ„ ์„ฑ๋Šฅ ์ธก์ •์„ ์œ„ํ•œ ๋„๊ตฌ** +- **PV๋ฅผ ์ƒ์„ฑํ•˜์—ฌ ํŠน์ •ํ•œ ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค์˜ ์†๋„๋ฅผ ์ธก์ • ํ›„ ์ž๋™ ์‚ญ์ œ** + +**(1) Kubestr ๋‹ค์šด๋กœ๋“œ ๋ฐ ์„ค์น˜** + +```bash +(eks-user:default) [root@operator-host ~]# wget https://github.com/kastenhq/kubestr/releases/download/v0.4.48/kubestr_0.4.48_Linux_amd64.tar.gz +``` + +โœ… **์ถœ๋ ฅ** + +```bash +--2025-02-18 18:36:53-- https://github.com/kastenhq/kubestr/releases/download/v0.4.48/kubestr_0.4.48_Linux_amd64.tar.gz +Resolving github.com (github.com)... 20.200.245.247 +Connecting to github.com (github.com)|20.200.245.247|:443... connected. +HTTP request sent, awaiting response... 302 Found +Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/291834502/210e8359-9fb9-4740-afef-17a5b458ab0e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250218%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250218T093653Z&X-Amz-Expires=300&X-Amz-Signature=8fb0bacd16f3bcb0eec0ff2fc8939dd6fe2549fcea74b7edcb30e345e4bf026b&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dkubestr_0.4.48_Linux_amd64.tar.gz&response-content-type=application%2Foctet-stream [following] +--2025-02-18 18:36:53-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/291834502/210e8359-9fb9-4740-afef-17a5b458ab0e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250218%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250218T093653Z&X-Amz-Expires=300&X-Amz-Signature=8fb0bacd16f3bcb0eec0ff2fc8939dd6fe2549fcea74b7edcb30e345e4bf026b&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dkubestr_0.4.48_Linux_amd64.tar.gz&response-content-type=application%2Foctet-stream +Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... +Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected. +HTTP request sent, awaiting response... 200 OK +Length: 14703952 (14M) [application/octet-stream] +Saving to: โ€˜kubestr_0.4.48_Linux_amd64.tar.gzโ€™ + +100%[========================================================================================================>] 14,703,952 38.6MB/s in 0.4s + +2025-02-18 18:36:55 (38.6 MB/s) - โ€˜kubestr_0.4.48_Linux_amd64.tar.gzโ€™ saved [14703952/14703952] +``` + +```bash +(eks-user:default) [root@operator-host ~]# tar xvfz kubestr_0.4.48_Linux_amd64.tar.gz && mv kubestr /usr/local/bin/ && chmod +x /usr/local/bin/kubestr +``` + +โœ… **์ถœ๋ ฅ** + +```bash +LICENSE +README.md +kubestr +``` + +**(2) ์„ค์น˜ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# kubestr -h +``` + +โœ… **์ถœ๋ ฅ** + +```bash +kubestr is a tool that will scan your k8s cluster + and validate that the storage systems in place as well as run + performance tests. + +Usage: + kubestr [flags] + kubestr [command] + +Available Commands: + blockmount Checks if a storage class supports block volumes + completion Generate the autocompletion script for the specified shell + csicheck Runs the CSI snapshot restore check + file-restore Restore file(s) from a Snapshot or PVC to it's source PVC + fio Runs an fio test + help Help about any command + +Flags: + -h, --help help for kubestr + -e, --outfile string The file where test results will be written + -o, --output string Options(json) + +Use "kubestr [command] --help" for more information about a command. +``` + +### **3. ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# kubestr +``` + +โœ… **์ถœ๋ ฅ** + +```bash +************************************** + _ ___ _ ___ ___ ___ _____ ___ + | |/ / | | | _ ) __/ __|_ _| _ \ + | ' <| |_| | _ \ _|\__ \ | | | / + |_|\_\\___/|___/___|___/ |_| |_|_\ + +Explore your Kubernetes storage options +************************************** +Kubernetes Version Check: + Valid kubernetes version (v1.31.5-eks-8cce635) - OK + +RBAC Check: + Kubernetes RBAC is enabled - OK + +Aggregated Layer Check: + The Kubernetes Aggregated Layer is enabled - OK + +Available Storage Provisioners: + + kubernetes.io/aws-ebs: + This is an in tree provisioner. + + Storage Classes: + * gp2 + + To perform a FIO test, run- + ./kubestr fio -s + + To perform a check for block device support, run- + ./kubestr blockmount -s + + rancher.io/local-path: + Unknown driver type. + + Storage Classes: + * local-path + + To perform a FIO test, run- + ./kubestr fio -s + + To perform a check for block device support, run- + ./kubestr blockmount -s +``` + +- **์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ๋ชฉ๋ก ํ™•์ธ (`gp2`, `local-path`)** +- **ํ•ด๋‹น ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค๋ฅผ ๋Œ€์ƒ์œผ๋กœ ์„ฑ๋Šฅ ํ…Œ์ŠคํŠธ ๊ฐ€๋Šฅ** + +### **4. ๋ชจ๋‹ˆํ„ฐ๋ง ์„ค์ •** + +**(1) ์Šคํ† ๋ฆฌ์ง€ ์„ฑ๋Šฅ ๋ชจ๋‹ˆํ„ฐ๋ง** + +```bash +(eks-user:default) [root@operator-host ~]# watch 'kubectl get pod -owide;echo;kubectl get pv,pvc' +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Every 2.0s: kubectl get pod -owide;echo;kubectl get pv,pvc Tue Feb 18 18:39:29 2025 + +No resources found in default namespace. + +No resources found +``` + +**(2) ๋…ธ๋“œ๋ณ„ ๋””์Šคํฌ ์„ฑ๋Šฅ ๋ชจ๋‹ˆํ„ฐ๋ง (`iostat`)** + +```bash +ssh ec2-user@$N1 + +A newer release of "Amazon Linux" is available. + Version 2023.6.20250211: +Run "/usr/bin/dnf check-release-update" for full release and version update info + , #_ + ~\_ ####_ Amazon Linux 2023 + ~~ \_#####\ + ~~ \###| + ~~ \#/ ___ https://aws.amazon.com/linux/amazon-linux-2023 + ~~ V~' '-> + ~~~ / + ~~._. _/ + _/ _/ + _/m/' +Last login: Tue Feb 18 10:13:47 2025 from 182.230.60.93 +[ec2-user@ip-192-168-1-207 ~]$ iostat -xmdz 1 +``` + +```bash +ssh ec2-user@$N2 + +A newer release of "Amazon Linux" is available. + Version 2023.6.20250211: +Run "/usr/bin/dnf check-release-update" for full release and version update info + , #_ + ~\_ ####_ Amazon Linux 2023 + ~~ \_#####\ + ~~ \###| + ~~ \#/ ___ https://aws.amazon.com/linux/amazon-linux-2023 + ~~ V~' '-> + ~~~ / + ~~._. _/ + _/ _/ + _/m/' +Last login: Tue Feb 18 10:13:58 2025 from 182.230.60.93 +[ec2-user@ip-192-168-2-84 ~]$ iostat -xmdz 1 +``` + +```bash +ssh ec2-user@$N3 + +A newer release of "Amazon Linux" is available. + Version 2023.6.20250211: +Run "/usr/bin/dnf check-release-update" for full release and version update info + , #_ + ~\_ ####_ Amazon Linux 2023 + ~~ \_#####\ + ~~ \###| + ~~ \#/ ___ https://aws.amazon.com/linux/amazon-linux-2023 + ~~ V~' '-> + ~~~ / + ~~._. _/ + _/ _/ + _/m/' +Last login: Tue Feb 18 10:14:06 2025 from 182.230.60.93 +[ec2-user@ip-192-168-3-80 ~]$ iostat -xmdz 1 +``` + +![Image](https://github.com/user-attachments/assets/20614297-8fc2-4d6b-abb8-659fd44817cd) + +- 1์ดˆ ๋‹จ์œ„๋กœ ๋””์Šคํฌ ๋ถ€ํ•˜ ๋ชจ๋‹ˆํ„ฐ๋ง (`nvme0n1`) + +### **5. Kubestr๋ฅผ ํ™œ์šฉํ•œ ๋””์Šคํฌ ์„ฑ๋Šฅ ์ธก์ •** + +**๋žœ๋ค ์ฝ๊ธฐ ํ…Œ์ŠคํŠธ** + +**(1) FIO ํ…Œ์ŠคํŠธ ์„ค์ • ํŒŒ์ผ ์ƒ์„ฑ** + +```bash +(eks-user:default) [root@operator-host ~]# cat << EOF > fio-read.fio +> [global] +> ioengine=libaio +> direct=1 +> bs=4k +> runtime=120 +> time_based=1 +> iodepth=16 +> numjobs=4 +> group_reporting +> size=1g +> rw=randread +> [read] +> EOF +``` + +**(2) Kubestr๋ฅผ ์ด์šฉํ•œ ์„ฑ๋Šฅ ํ…Œ์ŠคํŠธ ์‹คํ–‰** + +```bash +(eks-user:default) [root@operator-host ~]# kubestr fio -f fio-read.fio -s local-path --size 10G +``` + +โœ… **์ถœ๋ ฅ** + +```bash +PVC created kubestr-fio-pvc-lsl6j +Pod created kubestr-fio-pod-xdnwx +Running FIO test (fio-read.fio) on StorageClass (local-path) with a PVC of Size (10G) +``` + +- PV, PVC, Pod๋ฅผ ์ƒ์„ฑ ํ›„ ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค(`local-path`)์˜ ์„ฑ๋Šฅ ํ…Œ์ŠคํŠธ ์ง„ํ–‰ + + +![Image](https://github.com/user-attachments/assets/ef20a2d3-ccd0-432a-82b5-9a1afbdb527d) + +![Image](https://github.com/user-attachments/assets/8cbbf677-9053-4db0-9c93-cf4be41de8f8) + +- ํ…Œ์ŠคํŠธ๊ฐ€ **192.168.1.207 ๋…ธ๋“œ์—์„œ ์‹คํ–‰๋จ** + +![Image](https://github.com/user-attachments/assets/88dc6565-f94f-4718-94e6-4a3f87a039ea) + + +**(3) ์„ฑ๋Šฅ ์ธก์ • ๊ฒฐ๊ณผ (IOPS ๋ฐ Bandwidth)** + +โœ… **์ถœ๋ ฅ** + +```bash +FIO test results: + +FIO version - fio-3.36 +Global options - ioengine=libaio verify= direct=1 gtod_reduce= + +JobName: + blocksize= filesize= iodepth= rw= +read: + IOPS=3023.845947 BW(KiB/s)=12095 + iops: min=2220 max=9001 avg=3025.832520 + bw(KiB/s): min=8880 max=36007 avg=12103.798828 + +Disk stats (read/write): + nvme0n1: ios=362379/201 merge=0/30 ticks=6373605/3593 in_queue=6377198, util=95.415291% + - OK +``` + +- **IOPS (์ดˆ๋‹น ์ž…์ถœ๋ ฅ ์ž‘์—… ์ˆ˜)** : **ํ‰๊ท  3024 (์ตœ์†Œ 2220, ์ตœ๋Œ€ 9001)** +- **Bandwidth (Throughput)** : **์•ฝ 11.8 MB/s (์ตœ์†Œ 8.7 MB/s, ์ตœ๋Œ€ 35.2 MB/s)** +- **AWS EBS `gp3`์˜ ๊ธฐ๋ณธ ์„ฑ๋Šฅ(3000 IOPS, 125 MB/s)๊ณผ ์œ ์‚ฌ** +- **๋””์Šคํฌ ์‚ฌ์šฉ๋ฅ ** : **95.4% ํ™œ์šฉ๋จ** + +**๋žœ๋ค์“ฐ๊ธฐ ์„ฑ๋Šฅ ํ…Œ์ŠคํŠธ** + +**(1) ๋žœ๋ค ์“ฐ๊ธฐ ํ…Œ์ŠคํŠธ ์„ค์ •** + +```bash +(eks-user:default) [root@operator-host ~]# cat << EOF > fio-write.fio +> [global] +> ioengine=libaio +> numjobs=16 +> iodepth=16 +> direct=1 +> bs=4k +> runtime=120 +> time_based=1 +> size=1g +> group_reporting +> rw=randrw +> rwmixread=0 +> rwmixwrite=100 +> [write] +> EOF +``` + +**(2) ๋žœ๋ค ์“ฐ๊ธฐ ์„ฑ๋Šฅ ํ…Œ์ŠคํŠธ ์‹คํ–‰** + +- StorageClass `local-path`๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ 20GB PVC ์ƒ์„ฑ ํ›„ ํ…Œ์ŠคํŠธ ์ง„ํ–‰ + +```bash +(eks-user:default) [root@operator-host ~]# kubestr fio -f fio-write.fio -s local-path --size 20G +``` + +โœ… **์ถœ๋ ฅ** + +```bash +PVC created kubestr-fio-pvc-ntvm4 +Pod created kubestr-fio-pod-lhstx +``` + +- ํ…Œ์ŠคํŠธ๊ฐ€ **192.168.1.207 ๋…ธ๋“œ์—์„œ ์‹คํ–‰๋จ** + + +![Image](https://github.com/user-attachments/assets/5992d68b-aefc-475d-b4d3-1008fe4c96be) + +![Image](https://github.com/user-attachments/assets/ce0905ae-a503-407f-b354-b4a4abc6e8a2) + + +**(3) ์„ฑ๋Šฅ ์ธก์ • ๊ฒฐ๊ณผ (IOPS ๋ฐ Bandwidth)** + +โœ… **์ถœ๋ ฅ** + +```bash +Running FIO test (fio-write.fio) on StorageClass (local-path) with a PVC of Size (20G) +Elapsed time- 4m10.953340777s +FIO test results: + +FIO version - fio-3.36 +Global options - ioengine=libaio verify= direct=1 gtod_reduce= + +JobName: + blocksize= filesize= iodepth= rw= +write: + IOPS=3024.511475 BW(KiB/s)=12098 + iops: min=1456 max=8625 avg=3024.983154 + bw(KiB/s): min=5824 max=34517 avg=12101.912109 + +Disk stats (read/write): + nvme0n1: ios=0/362366 merge=0/8 ticks=0/7063240 in_queue=7063240, util=94.799950% + - OK +``` + +- **IOPS: ํ‰๊ท  3024.98, ์ตœ์†Œ 1456, ์ตœ๋Œ€ 8625** +- **Bandwidth: ํ‰๊ท  12,101 KiB/s, ์ตœ์†Œ 5824 KiB/s, ์ตœ๋Œ€ 34,517 KiB/s** +- **๋””์Šคํฌ ํ™œ์šฉ๋ฅ  94.8%** + +--- + +## **๐Ÿ’พ EBS CSI ์ปจํŠธ๋กค๋Ÿฌ ์„ค์ • ๋ฐ ๊ตฌ์„ฑ** + +### **1. EBS CSI ์ปจํŠธ๋กค๋Ÿฌ ๊ฐœ์š”** + +- **HostPath ๋ณผ๋ฅจ์˜ ํ•œ๊ณ„**: ๋…ธ๋“œ์˜ ๋””์Šคํฌ๊ฐ€ ๊ฝ‰ ์ฐจ๋ฉด ์‚ฌ์šฉ ๋ถˆ๊ฐ€ +- **AWS EBS์˜ ์žฅ์ **: ๋ธ”๋ก ์Šคํ† ๋ฆฌ์ง€ ์ œ๊ณต, Pod์—์„œ ์‰ฝ๊ฒŒ ๋ณผ๋ฅจ์„ ์ƒ์„ฑ ๋ฐ ๋ถ€์ฐฉ ๊ฐ€๋Šฅ +- **EBS CSI ์ปจํŠธ๋กค๋Ÿฌ ์—ญํ• ** + - EBS ๋ณผ๋ฅจ์„ ์ƒ์„ฑํ•˜๊ณ  Pod์— Attach + - Kubernetes API ์„œ๋ฒ„๋ฅผ ํ†ตํ•ด ์š”์ฒญ์„ ์ฒ˜๋ฆฌํ•˜๋Š” **CSI ์ปจํŠธ๋กค๋Ÿฌ** ์—ญํ•  ์ˆ˜ํ–‰ + +### **2. EBS CSI ๋“œ๋ผ์ด๋ฒ„ ๋ฒ„์ „ ํ™•์ธ** + +- **EKS์—์„œ ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ EBS CSI ๋“œ๋ผ์ด๋ฒ„ ๋ฒ„์ „ ์กฐํšŒ** + +```bash +(eks-user:default) [root@operator-host ~]# aws eks describe-addon-versions \ +> --addon-name aws-ebs-csi-driver \ +> --kubernetes-version 1.31 \ +> --query "addons[].addonVersions[].[addonVersion, compatibilities[].defaultVersion]" \ +> --output text +``` + +โœ… **์ถœ๋ ฅ** + +```bash +v1.39.0-eksbuild.1 +True +v1.38.1-eksbuild.2 +False +v1.38.1-eksbuild.1 +False +v1.37.0-eksbuild.2 +False +v1.37.0-eksbuild.1 +False +... +``` + +### **3. IAM Role for Service Account (IRSA) ์ƒ์„ฑ** + +- EBS CSI ๋“œ๋ผ์ด๋ฒ„๊ฐ€ AWS EBS์— ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ๋„๋ก IAM ์—ญํ•  ์ƒ์„ฑ +- AWS EKS ํด๋Ÿฌ์Šคํ„ฐ์— IAM ServiceAccount ์ถ”๊ฐ€ + +```bash +eksctl create iamserviceaccount \ + --name ebs-csi-controller-sa \ + --namespace kube-system \ + --cluster ${CLUSTER_NAME} \ + --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \ + --approve \ + --role-only \ + --role-name AmazonEKS_EBS_CSI_DriverRole +``` + +โœ… **์ถœ๋ ฅ** + +```bash +2025-02-18 20:00:11 [โ„น] 1 existing iamserviceaccount(s) (kube-system/aws-load-balancer-controller) will be excluded +2025-02-18 20:00:11 [โ„น] 1 iamserviceaccount (kube-system/ebs-csi-controller-sa) was included (based on the include/exclude rules) +2025-02-18 20:00:11 [!] serviceaccounts in Kubernetes will not be created or modified, since the option --role-only is used +2025-02-18 20:00:11 [โ„น] 1 task: { create IAM role for serviceaccount "kube-system/ebs-csi-controller-sa" } +2025-02-18 20:00:11 [โ„น] building iamserviceaccount stack "eksctl-myeks-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" +2025-02-18 20:00:11 [โ„น] deploying stack "eksctl-myeks-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" +2025-02-18 20:00:11 [โ„น] waiting for CloudFormation stack "eksctl-myeks-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" +2025-02-18 20:00:42 [โ„น] waiting for CloudFormation stack "eksctl-myeks-addon-iamserviceaccount-kube-system-ebs-csi-controller-sa" +``` + +- CloudFormation์— ์ƒˆ๋กœ์šด Stack์ด ์ƒ์„ฑ๋จ + +![Image](https://github.com/user-attachments/assets/5056118f-bf72-47d1-98c6-50cce70f1ad6) + +- **ISRA ํ™•์ธ** + +```bash +eksctl get iamserviceaccount --cluster ${CLUSTER_NAME} +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAMESPACE NAME ROLE ARN +kube-system aws-load-balancer-controller arn:aws:iam::378102432899:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-O6YEYsN7iVeQ +kube-system ebs-csi-controller-sa arn:aws:iam::378102432899:role/AmazonEKS_EBS_CSI_DriverRole +``` + +### **4. Amazon EBS CSI ๋“œ๋ผ์ด๋ฒ„ ๋ฐฐํฌ** + +**(1) EBS CSI ๋“œ๋ผ์ด๋ฒ„ Add-on ์„ค์น˜** + +```bash +export ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text) +eksctl create addon --name aws-ebs-csi-driver --cluster ${CLUSTER_NAME} --service-account-role-arn arn:aws:iam::${ACCOUNT_ID}:role/AmazonEKS_EBS_CSI_DriverRole --force +``` + +โœ… **์ถœ๋ ฅ** + +```bash +2025-02-18 20:05:22 [โ„น] Kubernetes version "1.31" in use by cluster "myeks" +2025-02-18 20:05:23 [โ„น] IRSA is set for "aws-ebs-csi-driver" addon; will use this to configure IAM permissions +2025-02-18 20:05:23 [!] the recommended way to provide IAM permissions for "aws-ebs-csi-driver" addon is via pod identity associations; after addon creation is completed, run `eksctl utils migrate-to-pod-identity` +2025-02-18 20:05:23 [โ„น] using provided ServiceAccountRoleARN "arn:aws:iam::378102432899:role/AmazonEKS_EBS_CSI_DriverRole" +2025-02-18 20:05:23 [โ„น] creating addon +``` + +**(2) ์„ค์น˜๋œ Add-on ํ™•์ธ** + +```bash +eksctl get addon --cluster ${CLUSTER_NAME} +``` + +โœ… **์ถœ๋ ฅ** + +```bash +2025-02-18 20:09:13 [โ„น] Kubernetes version "1.31" in use by cluster "myeks" +2025-02-18 20:09:13 [โ„น] getting all addons +2025-02-18 20:09:15 [โ„น] to see issues for an addon run `eksctl get addon --name --cluster ` +NAME VERSION STATUS ISSUES IAMROLE UPDATE AVAILABLE CONFIGURATION VALUES POD IDENTITY ASSOCIATION ROLES +aws-ebs-csi-driver v1.39.0-eksbuild.1 ACTIVE 0 arn:aws:iam::378102432899:role/AmazonEKS_EBS_CSI_DriverRole +coredns v1.11.4-eksbuild.2 ACTIVE 0 +kube-proxy v1.31.3-eksbuild.2 ACTIVE 0 +metrics-server v0.7.2-eksbuild.2 ACTIVE 0 +vpc-cni v1.19.2-eksbuild.5 ACTIVE 0 arn:aws:iam::378102432899:role/eksctl-myeks-addon-vpc-cni-Role1-ZTYxtOMDwfFu enableNetworkPolicy: "true" +``` + +### **5. EBS CSI ์ปจํŠธ๋กค๋Ÿฌ ๋ฐ DaemonSet ํ™•์ธ** + +```bash +kubectl get deploy,ds -l=app.kubernetes.io/name=aws-ebs-csi-driver -n kube-system +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/ebs-csi-controller 2/2 2 2 5m36s + +NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE +daemonset.apps/ebs-csi-node 3 3 3 3 3 kubernetes.io/os=linux 5m36s +daemonset.apps/ebs-csi-node-windows 0 0 0 0 0 kubernetes.io/os=windows 5m36s +``` + +- **Pod ๋ชฉ๋ก ํ™•์ธ** + +```bash +kubectl get pod -n kube-system -l app=ebs-csi-controller -o jsonpath='{.items[0].spec.containers[*].name}' ; echo +``` + +โœ… **์ถœ๋ ฅ** + +```bash +ebs-plugin csi-provisioner csi-attacher csi-snapshotter csi-resizer liveness-probe +``` + +- **์ „์ฒด Pod ๋ชฉ๋ก ํ™•์ธ** + +```bash +k get pod -A +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAMESPACE NAME READY STATUS RESTARTS AGE +kube-system aws-load-balancer-controller-554fbd9d-vk2p8 1/1 Running 0 5h26m +kube-system aws-load-balancer-controller-554fbd9d-xnx6r 1/1 Running 0 5h26m +kube-system aws-node-rf9bf 2/2 Running 0 8h +kube-system aws-node-tbbhl 2/2 Running 0 8h +kube-system aws-node-xb7dt 2/2 Running 0 8h +kube-system coredns-86f5954566-mskq6 1/1 Running 0 8h +kube-system coredns-86f5954566-wxwqw 1/1 Running 0 8h +kube-system ebs-csi-controller-7f8f8cb84-fd2bm 6/6 Running 0 7m35s +kube-system ebs-csi-controller-7f8f8cb84-tsvk8 6/6 Running 0 7m35s +kube-system ebs-csi-node-8d77m 3/3 Running 0 7m35s +kube-system ebs-csi-node-b2qcp 3/3 Running 0 7m35s +kube-system ebs-csi-node-rkk64 3/3 Running 0 7m35s +kube-system external-dns-dc4878f5f-mvnt9 1/1 Running 0 5h24m +kube-system kube-ops-view-657dbc6cd8-fgbqc 1/1 Running 0 5h29m +kube-system kube-proxy-6bc4m 1/1 Running 0 8h +kube-system kube-proxy-qsd8t 1/1 Running 0 8h +kube-system kube-proxy-rvw86 1/1 Running 0 8h +kube-system metrics-server-6bf5998d9c-nt4ks 1/1 Running 0 8h +kube-system metrics-server-6bf5998d9c-prz6f 1/1 Running 0 8h +local-path-storage local-path-provisioner-84967477f-g6xvh 1/1 Running 0 3h3m +``` + +### **6. CSI Node ๋ฐ Driver ํ™•์ธ** + +**(1) CSI ๊ด€๋ จ ๋ฆฌ์†Œ์Šค ํ™•์ธ** + +```bash +kubectl api-resources | grep -i csi +``` + +โœ… **์ถœ๋ ฅ** + +```bash +csidrivers storage.k8s.io/v1 false CSIDriver +csinodes storage.k8s.io/v1 false CSINode +csistoragecapacities storage.k8s.io/v1 true CSIStorageCapacity +``` + +**(2) CSI ๋…ธ๋“œ ์ƒํƒœ ํ™•์ธ** + +```bash +kubectl get csinodes +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME DRIVERS AGE +ip-192-168-1-207.ap-northeast-2.compute.internal 1 8h +ip-192-168-2-84.ap-northeast-2.compute.internal 1 8h +ip-192-168-3-80.ap-northeast-2.compute.internal 1 8h +``` + +- ๊ฐ ๋…ธ๋“œ์— `ebs.csi.aws.com` ๋“œ๋ผ์ด๋ฒ„๊ฐ€ ์„ค์น˜๋จ + +**(3) CSI Node ์ƒ์„ธ ์ •๋ณด ํ™•์ธ** + +```bash +kubectl describe csinodes +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: ip-192-168-1-207.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:46 +0900 +Spec: + Drivers: + ebs.csi.aws.com: + Node ID: i-093ad32d5ff5a8770 + Allocatables: + Count: 25 + Topology Keys: [kubernetes.io/os topology.ebs.csi.aws.com/zone topology.kubernetes.io/zone] +Events: + +Name: ip-192-168-2-84.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:49 +0900 +Spec: + Drivers: + ebs.csi.aws.com: + Node ID: i-0a80fdc36a856f394 + Allocatables: + Count: 25 + Topology Keys: [kubernetes.io/os topology.ebs.csi.aws.com/zone topology.kubernetes.io/zone] +Events: + +Name: ip-192-168-3-80.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:42 +0900 +Spec: + Drivers: + ebs.csi.aws.com: + Node ID: i-0484d2b724be33973 + Allocatables: + Count: 25 + Topology Keys: [kubernetes.io/os topology.ebs.csi.aws.com/zone topology.kubernetes.io/zone] +Events: +``` + +- ๊ฐ ๋…ธ๋“œ์—์„œ ์ตœ๋Œ€ 25๊ฐœ์˜ EBS ๋ณผ๋ฅจ์„ ๋ถ€์ฐฉ ๊ฐ€๋Šฅ + +**(4) CSI ๋“œ๋ผ์ด๋ฒ„ ๋ชฉ๋ก ์กฐํšŒ** + +```bash +kubectl get csidrivers +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE +ebs.csi.aws.com true false false false Persistent 12m +efs.csi.aws.com false false false false Persistent 8h +``` + +- EBS CSI ๋“œ๋ผ์ด๋ฒ„(`ebs.csi.aws.com`)๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ๋“ฑ๋ก๋จ + +**(5) CSI ๋“œ๋ผ์ด๋ฒ„ ์ƒ์„ธ ์ •๋ณด ํ™•์ธ** + +```bash +kubectl describe csidrivers ebs.csi.aws.com +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: ebs.csi.aws.com +Namespace: +Labels: app.kubernetes.io/component=csi-driver + app.kubernetes.io/managed-by=EKS + app.kubernetes.io/name=aws-ebs-csi-driver + app.kubernetes.io/version=1.39.0 +Annotations: +API Version: storage.k8s.io/v1 +Kind: CSIDriver +Metadata: + Creation Timestamp: 2025-02-18T11:05:31Z + Resource Version: 112312 + UID: 4db549f5-c7ef-42b5-8dbc-7611b898b6fc +Spec: + Attach Required: true + Fs Group Policy: ReadWriteOnceWithFSType + Pod Info On Mount: false + Requires Republish: false + Se Linux Mount: false + Storage Capacity: false + Volume Lifecycle Modes: + Persistent +Events: +``` + +### **7. ๋…ธ๋“œ์˜ ์ตœ๋Œ€ EBS ๋ถ€์ฐฉ ์ˆ˜๋Ÿ‰ ๋ณ€๊ฒฝ** + +- **๊ธฐ๋ณธ์ ์œผ๋กœ ๋…ธ๋“œ๋‹น EBS ๋ณผ๋ฅจ ์ตœ๋Œ€ 25๊ฐœ๊นŒ์ง€ ๋ถ€์ฐฉ ๊ฐ€๋Šฅ** +- **์ตœ๋Œ€ ๋ถ€์ฐฉ ์ˆ˜๋Ÿ‰์„ 31๊ฐœ๋กœ ์ฆ๊ฐ€** + +```bash +aws eks update-addon --cluster-name ${CLUSTER_NAME} --addon-name aws-ebs-csi-driver \ + --addon-version v1.39.0-eksbuild.1 --configuration-values '{ + "node": { + "volumeAttachLimit": 31, + "enableMetrics": true + } + }' +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "update": { + "id": "31bc4279-d50f-30e0-aebe-de2971095881", + "status": "InProgress", + "type": "AddonUpdate", + "params": [ + { + "type": "AddonVersion", + "value": "v1.39.0-eksbuild.1" + }, + { + "type": "ConfigurationValues", + "value": "{\n \"node\": {\n \"volumeAttachLimit\": 31,\n \"enableMetrics\": true\n }\n }" + } + ], + "createdAt": "2025-02-18T20:19:34.938000+09:00", + "errors": [] + } + +``` + +- **EKS Add-ons์—์„œ `aws-ebs-csi-driver`์˜ ์„ค์ • ๋ณ€๊ฒฝ ๊ฐ€๋Šฅ** + +![Image](https://github.com/user-attachments/assets/2e785633-b22c-4ab8-a010-3f64dcb70bea) + +### **8. EBS CSI Node ๋ฐ๋ชฌ์…‹ ์„ค์ • ํ™•์ธ** + +```bash +kubectl get ds -n kube-system ebs-csi-node -o yaml +``` + +โœ… **์ถœ๋ ฅ (์ผ๋ถ€)** + +```bash +containers: + - args: + - node + - --endpoint=$(CSI_ENDPOINT) + - --http-endpoint=0.0.0.0:3302 + - --csi-mount-point-prefix=/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/ + - --volume-attach-limit=31 +``` + +- **EBS CSI Node ๋ฐ๋ชฌ์…‹์ด ์žฌ๊ธฐ๋™๋˜๋ฉด์„œ Argument ๊ฐ’์ด ์ ์šฉ๋จ** +- **๊ฐ ๋…ธ๋“œ์—์„œ `-volume-attach-limit=31` ์„ค์ •์ด ๋ฐ˜์˜๋จ** + +### **9. CSI Node ๋ณผ๋ฅจ ๋ถ€์ฐฉ ํ•œ๋„ ์ฆ๊ฐ€ ํ™•์ธ** + +```bash +kubectl describe csinodes +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: ip-192-168-1-207.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:46 +0900 +Spec: + Drivers: + ebs.csi.aws.com: + Node ID: i-093ad32d5ff5a8770 + Allocatables: + Count: 31 + Topology Keys: [kubernetes.io/os topology.ebs.csi.aws.com/zone topology.kubernetes.io/zone] +Events: + +Name: ip-192-168-2-84.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:49 +0900 +Spec: + Drivers: + ebs.csi.aws.com: + Node ID: i-0a80fdc36a856f394 + Allocatables: + Count: 31 + Topology Keys: [kubernetes.io/os topology.ebs.csi.aws.com/zone topology.kubernetes.io/zone] +Events: + +Name: ip-192-168-3-80.ap-northeast-2.compute.internal +Labels: +Annotations: storage.alpha.kubernetes.io/migrated-plugins: + kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd,kubernetes.io/portworx-v... +CreationTimestamp: Tue, 18 Feb 2025 11:46:42 +0900 +Spec: + Drivers: + ebs.csi.aws.com: + Node ID: i-0484d2b724be33973 + Allocatables: + Count: 31 + Topology Keys: [kubernetes.io/os topology.ebs.csi.aws.com/zone topology.kubernetes.io/zone] +Events: +``` + +### **10. ๊ธฐ์กด ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ํ™•์ธ** + +```bash +kubectl get sc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 8h +local-path rancher.io/local-path Delete WaitForFirstConsumer false 3h19m +``` + +### **11. gp3 ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ์ƒ์„ฑ** + +```bash +cat < +ReclaimPolicy: Delete +VolumeBindingMode: WaitForFirstConsumer +Events: +``` + +- ๋ณผ๋ฅจ ํ™•์žฅ ๊ฐ€๋Šฅ (`AllowVolumeExpansion: True`) +- ๊ธฐ๋ณธ ํŒŒ์ผ ์‹œ์Šคํ…œ `xfs` ์ ์šฉ + +### **13. EBS ๋ณผ๋ฅจ ์ƒ์„ฑ ๋ชจ๋‹ˆํ„ฐ๋ง** + +```bash +while true; do aws ec2 describe-volumes --filters Name=tag:ebs.csi.aws.com/cluster,Values=true --query "Volumes[].{VolumeId: VolumeId, VolumeType: VolumeType, InstanceId: Attachments[0].InstanceId, State: Attachments[0].State}" --output text; date; sleep 1; done +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 08:36:24 PM KST 2025 +Tue Feb 18 08:36:25 PM KST 2025 +Tue Feb 18 08:36:27 PM KST 2025 +Tue Feb 18 08:36:28 PM KST 2025 +``` + +### **14. EBS PVC ์ƒ์„ฑ** + +**(1) StorageClass `gp3`๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ PVC๋ฅผ ์ƒ์„ฑ** + +```bash +cat < 96s +``` + +### **15. PVC๋ฅผ ์‚ฌ์šฉํ•˜๋Š” Pod ์ƒ์„ฑ** + +```bash +cat <> /data/out.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: ebs-claim +EOF + +# ๊ฒฐ๊ณผ +pod/app created +``` + +### **16. PVC์™€ PV ๋ฐ”์ธ๋”ฉ ์ƒํƒœ ํ™•์ธ** + +```bash +kubectl get pvc,pv,pod +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +persistentvolumeclaim/ebs-claim Bound pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5 4Gi RWO gp3 4m22s + +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE +persistentvolume/pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5 4Gi RWO Delete Bound default/ebs-claim gp3 26s + +NAME READY STATUS RESTARTS AGE +pod/app 1/1 Running 0 29s +``` + +- PVC(`ebs-claim`)๊ฐ€ ์ž๋™์œผ๋กœ PV(`pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5`)์™€ ๋ฐ”์ธ๋”ฉ๋จ +- Pod(`app`)์ด ์ •์ƒ์ ์œผ๋กœ ์‹คํ–‰๋จ (`Status: Running`) + +- Pod ์‹คํ–‰ ํ›„ EBS ๋ณผ๋ฅจ์ด ์ž๋™ ์ƒ์„ฑ ๋ฐ ๋ถ€์ฐฉ๋จ + +![Image](https://github.com/user-attachments/assets/be0e5ab7-0ba4-4554-9435-2b7b4f9f4575) + +### **17. EBS ๋ณผ๋ฅจ์ด ํŠน์ • ๋…ธ๋“œ์— ๋ถ€์ฐฉ๋˜์—ˆ๋Š”์ง€ ํ™•์ธ** + +```bash +kubectl get VolumeAttachment +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME ATTACHER PV NODE ATTACHED AGE +csi-2c549400779c2b0340a9261869f4798e1239dc965d8a47cf3b3c29fe8b2b4fd4 ebs.csi.aws.com pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5 ip-192-168-1-207.ap-northeast-2.compute.internal true 7m45s +``` + +- EBS ๋ณผ๋ฅจ์ด `ip-192-168-1-207` ๋…ธ๋“œ์— ๋ถ€์ฐฉ๋จ (`ATTACHED: true`) + +![Image](https://github.com/user-attachments/assets/a5a96977-d42a-443d-a2c0-3d8b8398b86c) + +### **18. EBS ๋ณผ๋ฅจ ์‚ฌ์šฉ๋Ÿ‰ ํ™•์ธ** + +```bash +kubectl df-pv +``` + +โœ… **์ถœ๋ ฅ** + +```bash + PV NAME PVC NAME NAMESPACE NODE NAME POD NAME VOLUME MOUNT NAME SIZE USED AVAILABLE %USED IUSED IFREE %IUSED + pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5 ebs-claim default ip-192-168-1-207.ap-northeast-2.compute.internal app persistent-storage 3Gi 60Mi 3Gi 1.50 4 2097148 0.00 +``` + +- **ํ˜„์žฌ `1.5%` ์ •๋„์˜ ๋ณผ๋ฅจ ๊ณต๊ฐ„์ด ์‚ฌ์šฉ๋จ** +- **Pod๊ฐ€ ์‹คํ–‰ ์ค‘์ธ ์›Œ์ปค ๋…ธ๋“œ(`192.168.1.207`)์—์„œ ํ•ด๋‹น ๋ณผ๋ฅจ์„ ์‚ฌ์šฉ ์ค‘** + +- **AWS ์ฝ˜์†”์—์„œ ๋ณผ๋ฅจ ์‚ฌ์šฉ๋Ÿ‰ ํ™•์ธ** + +![Image](https://github.com/user-attachments/assets/66e5b1fa-dae7-4a8f-ac75-515d39e254f1) + +### **19. EBS ๋ณผ๋ฅจ์˜ Node Affinity ํ™•์ธ** + +- ํ˜„์žฌ PV ์„ค์ • ํ™•์ธ + +```bash +kubectl get pv -o yaml +``` + +โœ… **์ถœ๋ ฅ (์ผ๋ถ€)** + +```bash +spec: + nodeAffinity: + required: + nodeSelectorTerms: + - matchExpressions: + - key: topology.kubernetes.io/zone + operator: In + values: + - ap-northeast-2a +``` + +- ํ˜„์žฌ `ap-northeast-2a` ๊ฐ€์šฉ ์˜์—ญ(AZ)์— ์žˆ๋Š” ๋…ธ๋“œ์—์„œ๋งŒ EBS ๋ถ€์ฐฉ ๊ฐ€๋Šฅ + +### **20. ํ˜„์žฌ ๋…ธ๋“œ ๋ชฉ๋ก ๋ฐ ๊ฐ€์šฉ ์˜์—ญ ํ™•์ธ** + +```bash +kubectl get node --label-columns=topology.ebs.csi.aws.com/zone,topology.k8s.aws/zone-id +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION ZONE ZONE-ID +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 9h v1.31.5-eks-5d632ec ap-northeast-2a apne2-az1 +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 9h v1.31.5-eks-5d632ec ap-northeast-2b apne2-az2 +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 9h v1.31.5-eks-5d632ec ap-northeast-2c apne2-az3 +``` + +- EBS ๋ณผ๋ฅจ์€ `ap-northeast-2a`์— ์žˆ๋Š” `ip-192-168-1-207` ๋…ธ๋“œ์—์„œ๋งŒ ์‚ฌ์šฉ ๊ฐ€๋Šฅ + +### **21. EBS ๋ณผ๋ฅจ ๋‚ด ๋ฐ์ดํ„ฐ ์ •์ƒ ์ €์žฅ ํ™•์ธ** + +```bash +kubectl exec app -- tail -f /data/out.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 12:05:17 UTC 2025 +Tue Feb 18 12:05:22 UTC 2025 +Tue Feb 18 12:05:27 UTC 2025 +Tue Feb 18 12:05:32 UTC 2025 +Tue Feb 18 12:05:37 UTC 2025 +Tue Feb 18 12:05:42 UTC 2025 +Tue Feb 18 12:05:47 UTC 2025 +... +``` + +- EBS ๋ณผ๋ฅจ์ด ์ •์ƒ์ ์œผ๋กœ `/data/out.txt` ํŒŒ์ผ์— ๋ฐ์ดํ„ฐ ๊ธฐ๋ก ์ค‘ + +### **22. EBS ๋ณผ๋ฅจ ๋งˆ์šดํŠธ ํ™•์ธ** + +**(1) Overlay ํŒŒ์ผ์‹œ์Šคํ…œ ํ™•์ธ** + +```bash +kubectl exec -it app -- sh -c 'df -hT --type=overlay' +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +overlay overlay 120G 4.5G 116G 4% / +``` + +**(2) XFS ํŒŒ์ผ์‹œ์Šคํ…œ ํ™•์ธ** + +```bash +kubectl exec -it app -- sh -c 'df -hT --type=xfs' +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +/dev/nvme1n1 xfs 4.0G 61M 3.9G 2% /data +/dev/nvme0n1p1 xfs 120G 4.5G 116G 4% /etc/hosts +``` + +- EBS ๋ณผ๋ฅจ(`/dev/nvme1n1`)์ด `/data`์— ๋งˆ์šดํŠธ๋จ +- ํ˜„์žฌ `4.0GiB` ์ค‘ `61MiB` ์‚ฌ์šฉ๋จ (`2%`) + +### **23. EBS ๋ณผ๋ฅจ ํฌ๊ธฐ ํ™•์žฅ ํ…Œ์ŠคํŠธ** + +**(1) ํ˜„์žฌ PVC ํฌ๊ธฐ ํ™•์ธ** + +```bash +kubectl get pvc ebs-claim -o jsonpath={.spec.resources.requests.storage} ; echo +``` + +โœ… **์ถœ๋ ฅ** + +```bash +4Gi +``` + +**(2) PVC ํฌ๊ธฐ ํ™•์žฅ ์š”์ฒญ (`4GiB โ†’ 10GiB`)** + +```bash +kubectl patch pvc ebs-claim -p '{"spec":{"resources":{"requests":{"storage":"10Gi"}}}}' + +# ๊ฒฐ๊ณผ +persistentvolumeclaim/ebs-claim patched +``` + +**(3) ๋ณ€๊ฒฝ๋œ ๋ณผ๋ฅจ ํฌ๊ธฐ ํ™•์ธ** + +```bash +kubectl exec -it app -- sh -c 'df -hT --type=xfs' +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +/dev/nvme1n1 xfs 10G 105M 9.9G 2% /data +/dev/nvme0n1p1 xfs 120G 4.5G 116G 4% /etc/hosts +``` + +- EBS ๋ณผ๋ฅจ ํฌ๊ธฐ๊ฐ€ `4GiB โ†’ 10GiB`๋กœ ์ •์ƒ ํ™•์žฅ๋จ + +**(4) PVC ์ƒํƒœ ํ™•์ธ (`df-pv` ํ™œ์šฉ)** + +```bash +kubectl df-pv +``` + +โœ… **์ถœ๋ ฅ** + +```bash + PV NAME PVC NAME NAMESPACE NODE NAME POD NAME VOLUME MOUNT NAME SIZE USED AVAILABLE %USED IUSED IFREE %IUSED + pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5 ebs-claim default ip-192-168-1-207.ap-northeast-2.compute.internal app persistent-storage 9Gi 104Mi 9Gi 1.02 4 5242876 0.00 +``` + +- ๋ณผ๋ฅจ ํฌ๊ธฐ `9GiB`, ์‚ฌ์šฉ๋Ÿ‰ `1.02%` ํ™•์ธ๋จ + +**(5) AWS์—์„œ ๋ณผ๋ฅจ ํฌ๊ธฐ ๋ณ€๊ฒฝ ์‚ฌํ•ญ ํ™•์ธ** + +```bash +aws ec2 describe-volumes --volume-ids $(kubectl get pv -o jsonpath="{.items[0].spec.csi.volumeHandle}") | jq +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "Volumes": [ + { + "Iops": 3000, + "Tags": [ + { + "Key": "kubernetes.io/created-for/pvc/name", + "Value": "ebs-claim" + }, + { + "Key": "ebs.csi.aws.com/cluster", + "Value": "true" + }, + { + "Key": "kubernetes.io/cluster/myeks", + "Value": "owned" + }, + { + "Key": "kubernetes.io/created-for/pvc/namespace", + "Value": "default" + }, + { + "Key": "KubernetesCluster", + "Value": "myeks" + }, + { + "Key": "CSIVolumeName", + "Value": "pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5" + }, + { + "Key": "Name", + "Value": "myeks-dynamic-pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5" + }, + { + "Key": "kubernetes.io/created-for/pv/name", + "Value": "pvc-ef2fe3fe-7117-44f2-94d0-cdb253c47af5" + } + ], + "VolumeType": "gp3", + "MultiAttachEnabled": false, + "Throughput": 125, + "Operator": { + "Managed": false + }, + "VolumeId": "vol-0b12360fbeebb9580", + "Size": 10, + "SnapshotId": "", + "AvailabilityZone": "ap-northeast-2a", + "State": "in-use", + "CreateTime": "2025-02-18T11:46:58.192000+00:00", + "Attachments": [ + { + "DeleteOnTermination": false, + "VolumeId": "vol-0b12360fbeebb9580", + "InstanceId": "i-093ad32d5ff5a8770", + "Device": "/dev/xvdaa", + "State": "attached", + "AttachTime": "2025-02-18T11:47:01+00:00" + } + ], + "Encrypted": true, + "KmsKeyId": "arn:aws:kms:ap-northeast-2:378102432899:key/8c9984ef-c009-4d66-bb63-428b05a0ed1e" + } + ] +} +``` + +- AWS ์ฝ˜์†”์—์„œ๋„ EBS ๋ณผ๋ฅจ ํฌ๊ธฐ `10GiB`๋กœ ํ™•์žฅ๋œ ๊ฒƒ ํ™•์ธ ๊ฐ€๋Šฅ + +![Image](https://github.com/user-attachments/assets/1676f878-276c-480a-9217-5de367769061) + +### **24. ๋ณผ๋ฅจ ํ™•์žฅ ํ›„ Pod ๋ฐ PVC ์‚ญ์ œ** + +```bash +kubectl delete pod app & kubectl delete pvc ebs-claim +``` + +โœ… **์ถœ๋ ฅ** + +```bash +[1] 208891 +pod "app" deleted +persistentvolumeclaim "ebs-claim" deleted +[1]+ Done kubecolor delete pod app +``` + +- Pod ์‚ญ์ œ ํ›„ PVC ์‚ญ์ œ ์‹œ, EBS ๋ณผ๋ฅจ๋„ ํ•จ๊ป˜ ์‚ญ์ œ๋จ +- AWS ์ฝ˜์†”์—์„œ EBS ๋ณผ๋ฅจ์ด ์ž๋™์œผ๋กœ ์ œ๊ฑฐ๋จ + +- **AWS ์ฝ˜์†”์—์„œ EBS ๋ณผ๋ฅจ ์‚ญ์ œ ํ™•์ธ** + +![Image](https://github.com/user-attachments/assets/30090d56-fcc3-4899-96e9-12594eb1583e) + +--- + +## **๐Ÿ“ธ EBS ๋ณผ๋ฅจ ์Šค๋ƒ…์ƒท ์ƒ์„ฑ ๋ฐ ๋ณต์›** + +### **1. EBS ๋ณผ๋ฅจ ์Šค๋ƒ…์ƒท ๊ธฐ๋Šฅ ๊ฐœ์š”** + +- AWS EBS ๋ณผ๋ฅจ์˜ ์Šค๋ƒ…์ƒท์„ ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค ๋„ค์ดํ‹ฐ๋ธŒ ๋ฐฉ์‹์œผ๋กœ ํ™œ์šฉ +- **AWS Volume SnapShots Controller**๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์Šค๋ƒ…์ƒท ์ƒ์„ฑ ๋ฐ ๋ณต์› ๊ฐ€๋Šฅ + +### 2. Volume Snapshot CRD ์„ค์น˜ + +```bash +kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml +kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml +kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io created +customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io created +customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io created +``` + +### **3. ์„ค์น˜ ํ™•์ธ** + +**(1) CRD์—์„œ ์Šค๋ƒ…์ƒท ๊ด€๋ จ ๋ฆฌ์†Œ์Šค ํ™•์ธ** + +```bash +kubectl get crd | grep snapshot +``` + +โœ… **์ถœ๋ ฅ** + +```bash +volumesnapshotclasses.snapshot.storage.k8s.io 2025-02-18T12:47:37Z +volumesnapshotcontents.snapshot.storage.k8s.io 2025-02-18T12:47:38Z +volumesnapshots.snapshot.storage.k8s.io 2025-02-18T12:47:36Z +``` + +**(2) API ๋ฆฌ์†Œ์Šค์—์„œ ์Šค๋ƒ…์ƒท ๊ด€๋ จ ๋ฆฌ์†Œ์Šค ํ™•์ธ** + +```bash +kubectl api-resources | grep snapshot +``` + +โœ… **์ถœ๋ ฅ** + +```bash +volumesnapshotclasses vsclass,vsclasses snapshot.storage.k8s.io/v1 false VolumeSnapshotClass +volumesnapshotcontents vsc,vscs snapshot.storage.k8s.io/v1 false VolumeSnapshotContent +volumesnapshots vs snapshot.storage.k8s.io/v1 true VolumeSnapshot +``` + +### **4. ์Šค๋ƒ…์ƒท ์ปจํŠธ๋กค๋Ÿฌ ๋ฐฐํฌ** + +```bash +kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml +kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/master/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +serviceaccount/snapshot-controller created +clusterrole.rbac.authorization.k8s.io/snapshot-controller-runner created +clusterrolebinding.rbac.authorization.k8s.io/snapshot-controller-role created +role.rbac.authorization.k8s.io/snapshot-controller-leaderelection created +rolebinding.rbac.authorization.k8s.io/snapshot-controller-leaderelection created +deployment.apps/snapshot-controller created +``` + +### **5. ๋ฐฐํฌ ํ™•์ธ** + +```bash +kubectl get deploy -n kube-system snapshot-controller +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME READY UP-TO-DATE AVAILABLE AGE +snapshot-controller 2/2 2 2 48s +``` + +### **6. AWS EBS ์Šค๋ƒ…์ƒท ํด๋ž˜์Šค ์ƒ์„ฑ** + +```bash +kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-ebs-csi-driver/master/examples/kubernetes/snapshot/manifests/classes/snapshotclass.yaml + +# ๊ฒฐ๊ณผ +volumesnapshotclass.snapshot.storage.k8s.io/csi-aws-vsc created +``` + +### **7. ์Šค๋ƒ…์ƒท ํด๋ž˜์Šค ํ™•์ธ** + +**(1) ๋ณผ๋ฅจ ์Šค๋ƒ…์ƒท ํด๋ž˜์Šค(VolumeSnapshotClass) ๋ชฉ๋ก ํ™•์ธ** + +```bash +kubectl get vsclass +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME DRIVER DELETIONPOLICY AGE +csi-aws-vsc ebs.csi.aws.com Delete 1s +``` + +**(2) ๋ณผ๋ฅจ ์Šค๋ƒ…์ƒท ํด๋ž˜์Šค(VolumeSnapshotClass) ์ƒ์„ธ ์ •๋ณด ํ™•์ธ** + +```bash +kubectl describe vsclass +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: csi-aws-vsc +Namespace: +Labels: +Annotations: +API Version: snapshot.storage.k8s.io/v1 +Deletion Policy: Delete +Driver: ebs.csi.aws.com +Kind: VolumeSnapshotClass +Metadata: + Creation Timestamp: 2025-02-18T12:53:56Z + Generation: 1 + Resource Version: 142886 + UID: e310327c-0673-4a0d-bf3f-fa1c2791b063 +Events: +``` + +### **8. PVC ๋ฐ Pod ์ƒ์„ฑ** + +**(1) ๋ชจ๋‹ˆํ„ฐ๋ง** + +```bash +watch -d kubectl get pv,pvc,pod + +Every 2.0s: kubectl get pv,pvc,pod gram88: 09:57:32 PM + in 0.813s (0) +No resources found +``` + +**(2) PVC ์ƒ์„ฑ** + +```bash +cat < 0s +``` + +**(4) Pod ์ƒ์„ฑ** + +```bash +cat <> /data/out.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: ebs-claim +EOF + +# ๊ฒฐ๊ณผ +pod/app created +``` + +### **9. ํŒŒ์ผ ์ €์žฅ ํ™•์ธ** + +```bash +kubectl exec app -- tail -f /data/out.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 13:00:45 UTC 2025 +Tue Feb 18 13:00:50 UTC 2025 +Tue Feb 18 13:00:55 UTC 2025 +Tue Feb 18 13:01:00 UTC 2025 +Tue Feb 18 13:01:05 UTC 2025 +Tue Feb 18 13:01:10 UTC 2025 +Tue Feb 18 13:01:15 UTC 2025 +Tue Feb 18 13:01:20 UTC 2025 +Tue Feb 18 13:01:25 UTC 2025 +... +``` + +### **10. EBS ๋ณผ๋ฅจ ์Šค๋ƒ…์ƒท ์ƒ์„ฑ** + +```bash +cat < +Annotations: +API Version: snapshot.storage.k8s.io/v1 +Kind: VolumeSnapshot +Metadata: + Creation Timestamp: 2025-02-18T13:03:27Z + Finalizers: + snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection + snapshot.storage.kubernetes.io/volumesnapshot-bound-protection + Generation: 1 + Resource Version: 146033 + UID: 9bd1cc5d-21d5-47f4-97de-c4b6a871ae01 +Spec: + Source: + Persistent Volume Claim Name: ebs-claim + Volume Snapshot Class Name: csi-aws-vsc +Status: + Bound Volume Snapshot Content Name: snapcontent-9bd1cc5d-21d5-47f4-97de-c4b6a871ae01 + Creation Time: 2025-02-18T13:03:28Z + Ready To Use: true + Restore Size: 4Gi +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal CreatingSnapshot 5m20s snapshot-controller Waiting for a snapshot default/ebs-volume-snapshot to be created by the CSI driver. + Normal SnapshotCreated 5m19s snapshot-controller Snapshot default/ebs-volume-snapshot was successfully created by the CSI driver. + Normal SnapshotReady 4m12s snapshot-controller Snapshot default/ebs-volume-snapshot is ready to use. +``` + +**(4) ์ƒ์„ฑ๋œ VolumeSnapshotContent ํ™•์ธ** + +```bash +kubectl get volumesnapshotcontents +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE +snapcontent-9bd1cc5d-21d5-47f4-97de-c4b6a871ae01 true 4294967296 Delete ebs.csi.aws.com csi-aws-vsc ebs-volume-snapshot default 5m52s +``` + +**(5) VolumeSnapshot ID ํ™•์ธ** + +```bash +kubectl get volumesnapshotcontents -o jsonpath='{.items[*].status.snapshotHandle}' ; echo +``` + +โœ… **์ถœ๋ ฅ** + +```bash +snap-0f1c3fa51d2fc9d33 +``` + +**(6) AWS EBS ์Šค๋ƒ…์ƒท ๋ชฉ๋ก ์กฐํšŒ (JSON ์ถœ๋ ฅ)** + +```bash +aws ec2 describe-snapshots --owner-ids self | jq +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "Snapshots": [ + { + "Tags": [ + { + "Key": "Name", + "Value": "myeks-dynamic-snapshot-9bd1cc5d-21d5-47f4-97de-c4b6a871ae01" + }, + { + "Key": "CSIVolumeSnapshotName", + "Value": "snapshot-9bd1cc5d-21d5-47f4-97de-c4b6a871ae01" + }, + { + "Key": "kubernetes.io/cluster/myeks", + "Value": "owned" + }, + { + "Key": "ebs.csi.aws.com/cluster", + "Value": "true" + } + ], + "StorageTier": "standard", + "TransferType": "standard", + "CompletionTime": "2025-02-18T13:04:04.793000+00:00", + "SnapshotId": "snap-0f1c3fa51d2fc9d33", + "VolumeId": "vol-090da41ed97dae65a", + "State": "completed", + "StartTime": "2025-02-18T13:03:28.688000+00:00", + "Progress": "100%", + "OwnerId": "378102432899", + "Description": "Created by AWS EBS CSI driver for volume vol-090da41ed97dae65a", + "VolumeSize": 4, + "Encrypted": true, + "KmsKeyId": "arn:aws:kms:ap-northeast-2:378102432899:key/8c9984ef-c009-4d66-bb63-428b05a0ed1e" + } + ] +} +``` + +**(7) AWS EBS ์Šค๋ƒ…์ƒท ๋ชฉ๋ก ์กฐํšŒ (ํ…Œ์ด๋ธ” ์ถœ๋ ฅ)** + +```bash +aws ec2 describe-snapshots --owner-ids self --query 'Snapshots[]' --output table +``` + +โœ… **์ถœ๋ ฅ** + +```bash +-------------------------------------------------------------------------------------------------------- +| DescribeSnapshots | ++----------------+-------------------------------------------------------------------------------------+ +| CompletionTime| 2025-02-18T13:04:04.793000+00:00 | +| Description | Created by AWS EBS CSI driver for volume vol-090da41ed97dae65a | +| Encrypted | True | +| KmsKeyId | arn:aws:kms:ap-northeast-2:378102432899:key/8c9984ef-c009-4d66-bb63-428b05a0ed1e | +| OwnerId | 378102432899 | +| Progress | 100% | +| SnapshotId | snap-0f1c3fa51d2fc9d33 | +| StartTime | 2025-02-18T13:03:28.688000+00:00 | +| State | completed | +| StorageTier | standard | +| TransferType | standard | +| VolumeId | vol-090da41ed97dae65a | +| VolumeSize | 4 | ++----------------+-------------------------------------------------------------------------------------+ +|| Tags || +|+--------------------------------+-------------------------------------------------------------------+| +|| Key | Value || +|+--------------------------------+-------------------------------------------------------------------+| +|| Name | myeks-dynamic-snapshot-9bd1cc5d-21d5-47f4-97de-c4b6a871ae01 || +|| CSIVolumeSnapshotName | snapshot-9bd1cc5d-21d5-47f4-97de-c4b6a871ae01 || +|| kubernetes.io/cluster/myeks | owned || +|| ebs.csi.aws.com/cluster | true || +|+--------------------------------+-------------------------------------------------------------------+| +``` + +### **12. EBS ๋ณผ๋ฅจ ์‚ญ์ œ (์‹ค์ˆ˜ ๊ฐ€์ •)** + +- **Pod ๋ฐ PVC ์‚ญ์ œ** +- ์‹ค์ˆ˜๋กœ ์ธํ•ด ๋ณผ๋ฅจ์ด ์‚ญ์ œ๋œ ์ƒํ™ฉ ๊ฐ€์ • + +```bash +kubectl delete pod app && kubectl delete pvc ebs-claim + +# ๊ฒฐ๊ณผ +pod "app" deleted +persistentvolumeclaim "ebs-claim" deleted +``` + +![Image](https://github.com/user-attachments/assets/d8e88669-7fe7-468f-ba28-33feb8b56d70) + +### **13. ์Šค๋ƒ…์ƒท์„ ํ™œ์šฉํ•œ PVC ๋ณต์›** + +```bash +cat <> /data/out.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: ebs-snapshot-restored-claim +EOF + +# ๊ฒฐ๊ณผ +pod/app created +``` + +### **15. ๋ณต์›๋œ ๋ฐ์ดํ„ฐ ํ™•์ธ** + +```bash +kubectl exec app -- cat /data/out.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 13:03:00 UTC 2025 +Tue Feb 18 13:03:05 UTC 2025 +Tue Feb 18 13:03:10 UTC 2025 +Tue Feb 18 13:03:15 UTC 2025 +Tue Feb 18 13:18:15 UTC 2025 +Tue Feb 18 13:18:20 UTC 2025 +Tue Feb 18 13:18:25 UTC 2025 +Tue Feb 18 13:18:30 UTC 2025 +Tue Feb 18 13:18:35 UTC 2025 +``` + +- **์Šค๋ƒ…์ƒท ์ƒ์„ฑ ์‹œ์ (`13:03:15`)๊นŒ์ง€์˜ ๋ฐ์ดํ„ฐ๊ฐ€ ๋ณต์›๋จ** +- **์ƒˆ๋กœ์šด ๋ฐ์ดํ„ฐ(`13:18:15` ์ดํ›„)๋„ ์ •์ƒ์ ์œผ๋กœ ์ €์žฅ๋จ** + +### **16. ์‚ญ์ œ** + +```bash +kubectl delete pod app && kubectl delete pvc ebs-snapshot-restored-claim && kubectl delete volumesnapshots ebs-volume-snapshot +``` + +โœ… **์ถœ๋ ฅ** + +```bash +pod "app" deleted +persistentvolumeclaim "ebs-snapshot-restored-claim" deleted +volumesnapshot.snapshot.storage.k8s.io "ebs-volume-snapshot" deleted +``` + +--- + +## **๐Ÿ“ AWS EFS Controller** + +### **1. EFS Controller ๊ฐœ์š”** + +- ๊ธฐ์กด EBS๋Š” **Block Storage**, EFS๋Š” **File System ๊ธฐ๋ฐ˜์˜ ์Šคํ† ๋ฆฌ์ง€** +- Amazon EFS ํŒŒ์ผ ์‹œ์Šคํ…œ์„ **CloudFormation**์„ ํ†ตํ•ด ์ƒ์„ฑ +- ์ดํ›„ EFS๋ฅผ ์‚ฌ์šฉํ•˜๊ธฐ ์œ„ํ•ด CSI ๋“œ๋ผ์ด๋ฒ„ ์„ค์ • ์ง„ํ–‰ + +### **2. ์ƒ์„ฑ๋œ EFS ํŒŒ์ผ ์‹œ์Šคํ…œ ํ™•์ธ** + +```bash +aws efs describe-file-systems --query "FileSystems[*].FileSystemId" --output text +``` + +โœ… **์ถœ๋ ฅ** + +```bash +fs-0aeb6f8c0c228b9d2 +``` + +### **3. EFS CSI ๋“œ๋ผ์ด๋ฒ„ ๋ฒ„์ „ ํ™•์ธ** + +```bash +aws eks describe-addon-versions \ + --addon-name aws-efs-csi-driver \ + --kubernetes-version 1.31 \ + --query "addons[].addonVersions[].[addonVersion, compatibilities[].defaultVersion]" \ + --output text +``` + +โœ… **์ถœ๋ ฅ** + +```bash +v2.1.4-eksbuild.1 +True +v2.1.3-eksbuild.1 +False +v2.1.2-eksbuild.1 +False +v2.1.1-eksbuild.1 +False +v2.1.0-eksbuild.1 +False +... +``` + +### **4. IAM Role for Service Account (IRSA) ์„ค์ •** + +```bash +eksctl create iamserviceaccount \ + --name efs-csi-controller-sa \ + --namespace kube-system \ + --cluster ${CLUSTER_NAME} \ + --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \ + --approve \ + --role-only \ + --role-name AmazonEKS_EFS_CSI_DriverRole +``` + +โœ… **์ถœ๋ ฅ** + +```bash +2025-02-18 22:32:31 [โ„น] 2 existing iamserviceaccount(s) (kube-system/aws-load-balancer-controller,kube-system/ebs-csi-controller-sa) will be excluded +2025-02-18 22:32:31 [โ„น] 1 iamserviceaccount (kube-system/efs-csi-controller-sa) was included (based on the include/exclude rules) +2025-02-18 22:32:31 [!] serviceaccounts in Kubernetes will not be created or modified, since the option --role-only is used +2025-02-18 22:32:31 [โ„น] 1 task: { create IAM role for serviceaccount "kube-system/efs-csi-controller-sa" } +2025-02-18 22:32:31 [โ„น] building iamserviceaccount stack "eksctl-myeks-addon-iamserviceaccount-kube-system-efs-csi-controller-sa" +2025-02-18 22:32:31 [โ„น] deploying stack "eksctl-myeks-addon-iamserviceaccount-kube-system-efs-csi-controller-sa" +2025-02-18 22:32:31 [โ„น] waiting for CloudFormation stack "eksctl-myeks-addon-iamserviceaccount-kube-system-efs-csi-controller-sa" +2025-02-18 22:33:01 [โ„น] waiting for CloudFormation stack "eksctl-myeks-addon-iamserviceaccount-kube-system-efs-csi-controller-sa" +``` + +- EFS CSI ๋“œ๋ผ์ด๋ฒ„๊ฐ€ **AWS EFS API์™€ ํ†ต์‹ **ํ•  ์ˆ˜ ์žˆ๋„๋ก IAM ์—ญํ•  ์ƒ์„ฑ +- Amazon์ด ์ œ๊ณตํ•˜๋Š” `AmazonEFSCSIDriverPolicy` IAM ์ •์ฑ…์„ ์—ฐ๊ฒฐ + +### **5. IAM Service Account ํ™•์ธ** + +```bash +eksctl get iamserviceaccount --cluster ${CLUSTER_NAME} +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAMESPACE NAME ROLE ARN +kube-system aws-load-balancer-controller arn:aws:iam::378102432899:role/eksctl-myeks-addon-iamserviceaccount-kube-sys-Role1-O6YEYsN7iVeQ +kube-system ebs-csi-controller-sa arn:aws:iam::378102432899:role/AmazonEKS_EBS_CSI_DriverRole +kube-system efs-csi-controller-sa arn:aws:iam::378102432899:role/AmazonEKS_EFS_CSI_DriverRole +``` + +- `efs-csi-controller-sa` ์„œ๋น„์Šค ๊ณ„์ •์ด ์˜ฌ๋ฐ”๋ฅด๊ฒŒ **IAM ์—ญํ• ๊ณผ ์—ฐ๊ฒฐ๋จ** ํ™•์ธ + +### **6. Amazon EFS CSI ๋“œ๋ผ์ด๋ฒ„ ์• ๋“œ์˜จ ๋ฐฐํฌ** + +```bash +export ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text) +eksctl create addon --name aws-efs-csi-driver --cluster ${CLUSTER_NAME} --service-account-role-arn arn:aws:iam::${ACCOUNT_ID}:role/AmazonEKS_EFS_CSI_DriverRole --force +``` + +โœ… **์ถœ๋ ฅ** + +```bash +2025-02-18 22:35:34 [โ„น] Kubernetes version "1.31" in use by cluster "myeks" +2025-02-18 22:35:34 [โ„น] IRSA is set for "aws-efs-csi-driver" addon; will use this to configure IAM permissions +2025-02-18 22:35:34 [!] the recommended way to provide IAM permissions for "aws-efs-csi-driver" addon is via pod identity associations; after addon creation is completed, run `eksctl utils migrate-to-pod-identity` +2025-02-18 22:35:34 [โ„น] using provided ServiceAccountRoleARN "arn:aws:iam::378102432899:role/AmazonEKS_EFS_CSI_DriverRole" +2025-02-18 22:35:34 [โ„น] creating addon +``` + +- EFS CSI Driver ์• ๋“œ์˜จ์„ EKS ํด๋Ÿฌ์Šคํ„ฐ์— ์„ค์น˜ + +### **7. ์• ๋“œ์˜จ ๋ฐฐํฌ ์ƒํƒœ ํ™•์ธ** + +```bash +eksctl get addon --cluster ${CLUSTER_NAME} +``` + +โœ… **์ถœ๋ ฅ** + +```bash +2025-02-18 22:36:24 [โ„น] Kubernetes version "1.31" in use by cluster "myeks" +2025-02-18 22:36:24 [โ„น] getting all addons +2025-02-18 22:36:26 [โ„น] to see issues for an addon run `eksctl get addon --name --cluster ` +NAME VERSION STATUS ISSUES IAMROLE UPDATE AVAILABLE CONFIGURATION VALUES POD IDENTITY ASSOCIATION ROLES +aws-ebs-csi-driver v1.39.0-eksbuild.1 ACTIVE 0 { + "node": { + "volumeAttachLimit": 31, + "enableMetrics": true + } + } +aws-efs-csi-driver v2.1.4-eksbuild.1 ACTIVE 0 arn:aws:iam::378102432899:role/AmazonEKS_EFS_CSI_DriverRole +coredns v1.11.4-eksbuild.2 ACTIVE 0 +kube-proxy v1.31.3-eksbuild.2 ACTIVE 0 +metrics-server v0.7.2-eksbuild.2 ACTIVE 0 +vpc-cni v1.19.2-eksbuild.5 ACTIVE 0 arn:aws:iam::378102432899:role/eksctl-myeks-addon-vpc-cni-Role1-ZTYxtOMDwfFu enableNetworkPolicy: "true" +``` + +- `aws-efs-csi-driver`๊ฐ€ **ACTIVE ์ƒํƒœ**์ž„์„ ํ™•์ธ + +### **8. EFS CSI Driver ๋ฐฐํฌ ํ™•์ธ** + +```bash +kubectl get csidrivers efs.csi.aws.com -o yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +apiVersion: storage.k8s.io/v1 +kind: CSIDriver +metadata: + annotations: + kubectl.kubernetes.io/last-applied-configuration: | + {"apiVersion":"storage.k8s.io/v1","kind":"CSIDriver","metadata":{"annotations":{},"name":"efs.csi.aws.com"},"spec":{"attachRequired":false}} + creationTimestamp: "2025-02-18T02:37:23Z" + name: efs.csi.aws.com + resourceVersion: "155201" + uid: 9b68ab1a-c2c2-40e2-84c8-b0f06f04b289 +spec: + attachRequired: false + fsGroupPolicy: ReadWriteOnceWithFSType + podInfoOnMount: false + requiresRepublish: false + seLinuxMount: false + storageCapacity: false + volumeLifecycleModes: + - Persistent +``` + +- `efs.csi.aws.com` CSI ๋“œ๋ผ์ด๋ฒ„๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ์„ค์น˜๋จ ํ™•์ธ + +### **9. EFS ์‚ฌ์šฉ ๋ฐฉ์‹: ์ „์ฒด ๊ณต์œ  vs Access Points** + +- **EFS ํŒŒ์ผ ์‹œ์Šคํ…œ์„ ์ „์ฒด ๊ณต์œ ** +- **Access Points๋ฅผ ์ด์šฉํ•ด ํŠน์ • ๋””๋ ‰ํ† ๋ฆฌ๋งŒ ๋ถ„๋ฆฌํ•˜์—ฌ ์‚ฌ์šฉ** +- ํ•„์š”์— ๋”ฐ๋ผ **์ „์ฒด ํŒŒ์ผ ์‹œ์Šคํ…œ ๊ณต์œ  ๋˜๋Š” ํŠน์ • ๋””๋ ‰ํ† ๋ฆฌ ๊ถŒํ•œ ๊ด€๋ฆฌ ๊ฐ€๋Šฅ** + +- EFS์˜ Access Points ๊ธฐ๋Šฅ์„ ์‚ฌ์šฉํ•˜๋ฉด ํŠน์ • ๊ฒฝ๋กœ๋งŒ ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•˜๋„๋ก ์ œํ•œํ•  ์ˆ˜ ์žˆ์Œ + +![Image](https://github.com/user-attachments/assets/22a03b3d-b98f-4c3b-b897-13561c677cf3) + +### **10. EFS CSI Driver ํด๋ก  ๋ฐ ํŒŒ์ผ ๊ตฌ์กฐ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# git clone https://github.com/kubernetes-sigs/aws-efs-csi-driver.git /root/efs-csi +(eks-user:default) [root@operator-host ~]# cd /root/efs-csi/examples/kubernetes/multiple_pods/specs && tree +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Cloning into '/root/efs-csi'... +remote: Enumerating objects: 29682, done. +remote: Counting objects: 100% (5145/5145), done. +remote: Compressing objects: 100% (1142/1142), done. +remote: Total 29682 (delta 4275), reused 4015 (delta 3999), pack-reused 24537 (from 3) +Receiving objects: 100% (29682/29682), 27.11 MiB | 17.36 MiB/s, done. +Resolving deltas: 100% (16140/16140), done. + +. +โ”œโ”€โ”€ claim.yaml +โ”œโ”€โ”€ pod1.yaml +โ”œโ”€โ”€ pod2.yaml +โ”œโ”€โ”€ pv.yaml +โ””โ”€โ”€ storageclass.yaml + +0 directories, 5 files +``` + +- **EFS CSI Driver** ์ €์žฅ์†Œ๋ฅผ ํด๋ก ํ•˜์—ฌ ์ƒ˜ํ”Œ ์„ค์ • ํŒŒ์ผ ๋‹ค์šด๋กœ๋“œ +- StorageClass, PV, PVC, Pod ๊ด€๋ จ YAML ํŒŒ์ผ ์กด์žฌ + +### **11. EFS StorageClass ์ƒ์„ฑ ๋ฐ ํ™•์ธ** + +**(1) StorageClass ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# cat storageclass.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: efs-sc +provisioner: efs.csi.aws.com +``` + +**(2) StorageClass ์ƒ์„ฑ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl apply -f storageclass.yaml + +# ๊ฒฐ๊ณผ +storageclass.storage.k8s.io/efs-sc created +``` + +**(3) StorageClass ์ƒ์„ฑ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl get sc efs-sc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +efs-sc efs.csi.aws.com Delete Immediate false 54s +``` + +### **12. EFS PersistentVolume(PV) ์„ค์ •** + +**(1) EFS PV ์„ค์ • ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# cat pv.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +apiVersion: v1 +kind: PersistentVolume +metadata: + name: efs-pv +spec: + capacity: + storage: 5Gi + volumeMode: Filesystem + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + storageClassName: efs-sc + csi: + driver: efs.csi.aws.com + volumeHandle: fs-4af69aab +``` + +- ๊ธฐ๋ณธ์ ์œผ๋กœ `volumeHandle: fs-4af69aab`๋กœ ์„ค์ •๋˜์–ด ์žˆ์Œ + +**(2) EFS ์‹œ์Šคํ…œ ID ์—…๋ฐ์ดํŠธ** + +```bash +(eks-user:default) [root@operator-host specs]# EfsFsId=$(aws efs describe-file-systems --query "FileSystems[*].FileSystemId" --output text) +(eks-user:default) [root@operator-host specs]# sed -i "s/fs-4af69aab/$EfsFsId/g" pv.yaml +(eks-user:default) [root@operator-host specs]# cat pv.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +apiVersion: v1 +kind: PersistentVolume +metadata: + name: efs-pv +spec: + capacity: + storage: 5Gi + volumeMode: Filesystem + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + storageClassName: efs-sc + csi: + driver: efs.csi.aws.com + volumeHandle: fs-0aeb6f8c0c228b9d2 +``` + +- ๊ธฐ์กด `fs-4af69aab` โ†’ ์‹ค์ œ ์‚ฌ์šฉ ์ค‘์ธ EFS ์‹œ์Šคํ…œ ID(`fs-0aeb6f8c0c228b9d2`)๋กœ ๋ณ€๊ฒฝ + +### **13. EFS PV ์ƒ์„ฑ ๋ฐ ํ™•์ธ** + +**(1) `efs-pv` ์ƒ์„ฑ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl apply -f pv.yaml +# ๊ฒฐ๊ณผ +persistentvolume/efs-pv created +``` + +**(2) `efs-pv` ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl get pv; kubectl describe pv +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE +efs-pv 5Gi RWX Retain Available efs-sc 31s + +Name: efs-pv +Labels: +Annotations: +Finalizers: [kubernetes.io/pv-protection] +StorageClass: efs-sc +Status: Available +Claim: +Reclaim Policy: Retain +Access Modes: RWX +VolumeMode: Filesystem +Capacity: 5Gi +Node Affinity: +Message: +Source: + Type: CSI (a Container Storage Interface (CSI) volume source) + Driver: efs.csi.aws.com + FSType: + VolumeHandle: fs-0aeb6f8c0c228b9d2 + ReadOnly: false + VolumeAttributes: +Events: +``` + +- **ACCESS MODES: RWX** +- IP ๊ธฐ๋ฐ˜ ์ ‘๊ทผ์ด๋ฏ€๋กœ ๋‹ค์ˆ˜์˜ ํŒŒ๋“œ๊ฐ€ ๋™์‹œ์— ์ ‘๊ทผ ๊ฐ€๋Šฅ + +### **14. EFS PersistentVolumeClaim(PVC) ์ƒ์„ฑ** + +**(1) PVC ์ƒ์„ฑ ๋ฐ ์ ์šฉ** + +```bash +(eks-user:default) [root@operator-host specs]# cat claim.yaml +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: efs-claim +spec: + accessModes: + - ReadWriteMany + storageClassName: efs-sc + resources: + requests: + storage: 5Gi + +(eks-user:default) [root@operator-host specs]# kubectl apply -f claim.yaml +# ๊ฒฐ๊ณผ +persistentvolumeclaim/efs-claim created +``` + +- `efs-pv`๋ฅผ ์‚ฌ์šฉํ•  `efs-claim` PVC๋ฅผ ์ƒ์„ฑ + +**(2) PVC ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl get pvc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +efs-claim Bound efs-pv 5Gi RWX efs-sc 32s +``` + +- `STATUS`๊ฐ€ `Bound`์ด๋ฉด PV์™€ PVC๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ์—ฐ๊ฒฐ๋จ +- `ACCESS MODES`๊ฐ€ `RWX`๋กœ ์„ค์ •๋˜์–ด ์—ฌ๋Ÿฌ Pod์—์„œ ๋™์‹œ์— ์ ‘๊ทผ ๊ฐ€๋Šฅ + +### **15. EFS ๊ธฐ๋ฐ˜ ๋‹ค์ค‘ Pod ์ƒ์„ฑ** + +**(1) ๋‹ค์ค‘ Pod ์ƒ์„ฑ ๋ฐ ์ ์šฉ** + +- ๋‘ ๊ฐœ์˜ Pod(`app1`, `app2`)๋ฅผ ์ƒ์„ฑํ•˜์—ฌ **EFS ๋ณผ๋ฅจ์„ ๊ณต์œ ** + +```bash +(eks-user:default) [root@operator-host specs]# cat pod1.yaml pod2.yaml +apiVersion: v1 +kind: Pod +metadata: + name: app1 +spec: + containers: + - name: app1 + image: busybox + command: ["/bin/sh"] + args: ["-c", "while true; do echo $(date -u) >> /data/out1.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: efs-claim +apiVersion: v1 +kind: Pod +metadata: + name: app2 +spec: + containers: + - name: app2 + image: busybox + command: ["/bin/sh"] + args: ["-c", "while true; do echo $(date -u) >> /data/out2.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: efs-claim +``` + +```bash +(eks-user:default) [root@operator-host specs]# kubectl apply -f pod1.yaml,pod2.yaml + +# ๊ฒฐ๊ณผ +pod/app1 created +pod/app2 created +``` + +**(2) ๋‹ค์ค‘ Pod ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl get pods +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME READY STATUS RESTARTS AGE +app1 1/1 Running 0 57s +app2 1/1 Running 0 57s +``` + +![Image](https://github.com/user-attachments/assets/a3357cf5-1e80-4d7e-9ae7-431bac8ef4ca) + +### **16. Pod ๋‚ด๋ถ€์—์„œ EFS ๋ณผ๋ฅจ ํฌ๊ธฐ ํ™•์ธ** + +**(1) EFS ๋ณผ๋ฅจ์ด ์ •์ƒ์ ์œผ๋กœ ๋งˆ์šดํŠธ๋˜์—ˆ๋Š”์ง€ ํ™•์ธ** + +- `app1`๊ณผ `app2` Pod ๋‚ด๋ถ€์—์„œ `/data` ๋””๋ ‰ํ„ฐ๋ฆฌ๊ฐ€ EFS ๋ณผ๋ฅจ์œผ๋กœ ์ •์ƒ ๋งˆ์šดํŠธ๋˜์—ˆ๋Š”์ง€ ํ™•์ธ + +```bash +(eks-user:default) [root@operator-host specs]# kubectl exec -ti app1 -- sh -c "df -hT -t nfs4" + +# โœ… ์ถœ๋ ฅ +Filesystem Type Size Used Available Use% Mounted on +127.0.0.1:/ nfs4 8.0E 0 8.0E 0% /data +``` + +```bash +(eks-user:default) [root@operator-host specs]# kubectl exec -ti app2 -- sh -c "df -hT -t nfs4" + +# โœ… ์ถœ๋ ฅ +Filesystem Type Size Used Available Use% Mounted on +127.0.0.1:/ nfs4 8.0E 0 8.0E 0% /data +``` + +**(2) EFS ๋ณผ๋ฅจ ํฌ๊ธฐ ์ฐจ์ด ๋ถ„์„** + +- `kubectl get pv`๋กœ ํ™•์ธํ•œ **EFS์˜ ์„ค์ •๋œ ์šฉ๋Ÿ‰์€ 5GiB** +- ๊ทธ๋Ÿฌ๋‚˜ **Pod ๋‚ด๋ถ€์—์„œ๋Š” EFS ํฌ๊ธฐ๊ฐ€ `8.0E`(์—‘์‚ฌ๋ฐ”์ดํŠธ)๋กœ ํ‘œ์‹œ๋จ** +- ์ด๋Š” **EFS๊ฐ€ ๋ธ”๋ก ์Šคํ† ๋ฆฌ์ง€๊ฐ€ ์•„๋‹Œ ํŒŒ์ผ ์‹œ์Šคํ…œ ๊ธฐ๋ฐ˜์œผ๋กœ ๊ด€๋ฆฌ๋˜๋ฉฐ, ํฌ๊ธฐ ์ œํ•œ ์—†์ด ์ž๋™ ํ™•์žฅ ๊ฐ€๋Šฅ**ํ•˜๊ธฐ ๋•Œ๋ฌธ +- ์ฆ‰, EFS๋Š” **๋…ผ๋ฆฌ์ ์œผ๋กœ ๋ฌดํ•œํ•œ ํฌ๊ธฐ๋กœ ์ธ์‹๋˜๋ฉฐ, ์‹ค์ œ ์‚ฌ์šฉ๋Ÿ‰์— ๋”ฐ๋ผ ์ž๋™์œผ๋กœ ์ฆ๊ฐ€** + +### **17. ๋‹ค์ค‘ Pod์—์„œ EFS ๋งˆ์šดํŠธ ์„ค์ •** + +- `app1`๊ณผ `app2` Pod๊ฐ€ ๋™์ผํ•œ EFS ๋ณผ๋ฅจ์„ `/data` ๊ฒฝ๋กœ์— ๋งˆ์šดํŠธํ•˜๋„๋ก ์„ค์ • +- ๊ฐ Pod๋Š” `/data/out1.txt` ๋ฐ `/data/out2.txt` ํŒŒ์ผ์— 5์ดˆ ๊ฐ„๊ฒฉ์œผ๋กœ ๋กœ๊ทธ๋ฅผ ๊ธฐ๋ก + +```bash +(eks-user:default) [root@operator-host specs]# cat pod1.yaml pod2.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +apiVersion: v1 +kind: Pod +metadata: + name: app1 +spec: + containers: + - name: app1 + image: busybox + command: ["/bin/sh"] + args: ["-c", "while true; do echo $(date -u) >> /data/out1.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: efs-claim + +apiVersion: v1 +kind: Pod +metadata: + name: app2 +spec: + containers: + - name: app2 + image: busybox + command: ["/bin/sh"] + args: ["-c", "while true; do echo $(date -u) >> /data/out2.txt; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: efs-claim +``` + +### **18. ์šด์˜์„œ๋ฒ„์—์„œ EFS ๋งˆ์šดํŠธ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# df -hT +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +devtmpfs devtmpfs 981M 0 981M 0% /dev +tmpfs tmpfs 990M 0 990M 0% /dev/shm +tmpfs tmpfs 990M 432K 989M 1% /run +tmpfs tmpfs 990M 0 990M 0% /sys/fs/cgroup +/dev/xvda1 xfs 30G 3.2G 27G 11% / +192.168.1.145:/ nfs4 8.0E 0 8.0E 0% /mnt/myefs +tmpfs tmpfs 198M 0 198M 0% /run/user/1000 +``` + +- ์šด์˜์„œ๋ฒ„์—์„œ๋„ ๋™์ผํ•œ EFS ๋ณผ๋ฅจ์„ `/mnt/myefs`์— ๋งˆ์šดํŠธํ•˜์—ฌ ๊ณต์œ  ์ €์žฅ์†Œ๋กœ ํ™œ์šฉ ๊ฐ€๋Šฅ + +### **19. ๊ณต์œ  ์ €์žฅ์†Œ ๋‚ด ํŒŒ์ผ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# tree /mnt/myefs +``` + +โœ… **์ถœ๋ ฅ** + +```bash +/mnt/myefs +โ”œโ”€โ”€ memo.txt +โ”œโ”€โ”€ out1.txt +โ””โ”€โ”€ out2.txt + +0 directories, 3 files +``` + +- `out1.txt`์™€ `out2.txt`๋Š” ๊ฐ๊ฐ `app1`๊ณผ `app2`๊ฐ€ ์ƒ์„ฑํ•œ ํŒŒ์ผ +- ์šด์˜์„œ๋ฒ„์—์„œ๋„ ๋™์ผํ•œ ๋ฐ์ดํ„ฐ๋ฅผ ํ™•์ธ ๊ฐ€๋Šฅ + +### **20. EFS ๊ณต์œ  ์ €์žฅ์†Œ ๋™์ž‘ ํ™•์ธ** + +์šด์˜์„œ๋ฒ„์™€ `app1`, `app2`์—์„œ ๊ฐ™์€ ํŒŒ์ผ์„ ์กฐํšŒ + +**(1) ์šด์˜์„œ๋ฒ„์—์„œ ํŒŒ์ผ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# tail -f /mnt/myefs/out1.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 14:02:30 UTC 2025 +Tue Feb 18 14:02:35 UTC 2025 +Tue Feb 18 14:02:40 UTC 2025 +Tue Feb 18 14:02:45 UTC 2025 +Tue Feb 18 14:02:50 UTC 2025 +... +``` + +**(2) Pod ๋‚ด๋ถ€์—์„œ ๋™์ผํ•œ ํŒŒ์ผ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl exec -ti app1 -- tail -f /data/out1.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 14:02:55 UTC 2025 +Tue Feb 18 14:03:00 UTC 2025 +Tue Feb 18 14:03:05 UTC 2025 +Tue Feb 18 14:03:10 UTC 2025 +Tue Feb 18 14:03:15 UTC 2025 +... +``` + +- ์šด์˜์„œ๋ฒ„์™€ Pod ๊ฐ„์— ๋™์ผํ•œ ํŒŒ์ผ์„ ๊ณต์œ ํ•˜๋ฉฐ ์‹ค์‹œ๊ฐ„์œผ๋กœ ๋ฐ์ดํ„ฐ๊ฐ€ ์ €์žฅ๋จ + +### **21. EFS ๋งˆ์šดํŠธ ํ•ด์ œ ๋ฐ ๋ฆฌ์†Œ์Šค ์ •๋ฆฌ** + +**(1) Pod ๋ฐ EFS ๊ด€๋ จ ๋ฆฌ์†Œ์Šค ์‚ญ์ œ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl delete pod app1 app2 +# ๊ฒฐ๊ณผ +pod "app1" deleted +pod "app2" deleted +``` + +```bash +(eks-user:default) [root@operator-host specs]# kubectl delete pvc efs-claim && kubectl delete pv efs-pv && kubectl delete sc efs-sc +# ๊ฒฐ๊ณผ +persistentvolumeclaim "efs-claim" deleted +persistentvolume "efs-pv" deleted +storageclass.storage.k8s.io "efs-sc" deleted +``` + +--- + +## **๐Ÿ”‘ EFS AccessPoints๋ฅผ ํ™œ์šฉํ•œ ๋™์  ํ”„๋กœ๋น„์ €๋‹ ๋ฐฐํฌ** + +### **1. EFS AccessPoints ๊ธฐ๋ฐ˜ StorageClass ์ƒ์„ฑ** + +EFS AccessPoints๋ฅผ ํ™œ์šฉํ•˜์—ฌ ํŠน์ • ๋””๋ ‰ํ† ๋ฆฌ์™€ ๊ถŒํ•œ์„ ์„ค์ •ํ•  ์ˆ˜ ์žˆ๋Š” StorageClass ์ƒ์„ฑ + +```bash +(eks-user:default) [root@operator-host specs]# curl -s -O https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/examples/kubernetes/dynamic_provisioning/specs/storageclass.yaml +(eks-user:default) [root@operator-host specs]# cat storageclass.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: efs-sc +provisioner: efs.csi.aws.com +parameters: + provisioningMode: efs-ap + fileSystemId: fs-92107410 + directoryPerms: "700" + gidRangeStart: "1000" # optional + gidRangeEnd: "2000" # optional + basePath: "/dynamic_provisioning" # optional + subPathPattern: "${.PVC.namespace}/${.PVC.name}" # optional + ensureUniqueDirectory: "true" # optional + reuseAccessPoint: "false" # optional +``` + +**์„ค์ • ํ•ญ๋ชฉ** + +- `fileSystemId`: ์‚ฌ์šฉ ์ค‘์ธ EFS ID๋ฅผ ๋™์ ์œผ๋กœ ์ ์šฉ +- `basePath`: ๋™์  ์ƒ์„ฑ๋  ๋””๋ ‰ํ† ๋ฆฌ ๊ฒฝ๋กœ ์„ค์ • +- `subPathPattern`: PVC๋ณ„ ๊ฐœ๋ณ„ ๋””๋ ‰ํ† ๋ฆฌ ํ• ๋‹น + +```bash +(eks-user:default) [root@operator-host specs]# sed -i "s/fs-92107410/$EfsFsId/g" storageclass.yaml +(eks-user:default) [root@operator-host specs]# kubectl apply -f storageclass.yaml + +# ๊ฒฐ๊ณผ +storageclass.storage.k8s.io/efs-sc created +``` + +### **2. StorageClass ์ƒ์„ฑ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl get sc efs-sc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +efs-sc efs.csi.aws.com Delete Immediate false 5s +``` + +### **3. PVC ๋ฐ Pod ์ƒ์„ฑ** + +**(1) PVC ๋ฐ Pod ์ •์˜ ํŒŒ์ผ ๋‹ค์šด๋กœ๋“œ** + +```bash +(eks-user:default) [root@operator-host specs]# curl -s -O https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/examples/kubernetes/dynamic_provisioning/specs/pod.yaml +``` + +**(2) PVC ๋ฐ Pod ์„ค์ • ๋‚ด์šฉ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# cat pod.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: efs-claim +spec: + accessModes: + - ReadWriteMany + storageClassName: efs-sc + resources: + requests: + storage: 5Gi +--- +apiVersion: v1 +kind: Pod +metadata: + name: efs-app +spec: + containers: + - name: app + image: centos + command: ["/bin/sh"] + args: ["-c", "while true; do echo $(date -u) >> /data/out; sleep 5; done"] + volumeMounts: + - name: persistent-storage + mountPath: /data + volumes: + - name: persistent-storage + persistentVolumeClaim: + claimName: efs-claim +``` + +**(3) PVC ๋ฐ Pod ์ƒ์„ฑ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl apply -f pod.yaml + +# ๊ฒฐ๊ณผ +persistentvolumeclaim/efs-claim created +pod/efs-app created +``` + +### **4. PVC, PV, Pod ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl get pvc,pv,pod +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +persistentvolumeclaim/efs-claim Bound pvc-4ea92c76-1bda-4425-b183-77f8c6ea11ef 5Gi RWX efs-sc 2s + +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE +persistentvolume/pvc-4ea92c76-1bda-4425-b183-77f8c6ea11ef 5Gi RWX Delete Bound default/efs-claim efs-sc 2s + +NAME READY STATUS RESTARTS AGE +pod/efs-app 0/1 ContainerCreating 0 2s +``` + +### **5. PVC, PV ์ƒ์„ฑ ๋กœ๊ทธ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl krew install stern +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Updated the local copy of plugin index. +Installing plugin: stern +Installed plugin: stern +\ + | Use this plugin: + | kubectl stern + | Documentation: + | https://github.com/stern/stern +/ +WARNING: You installed plugin "stern" from the krew-index plugin repository. + These plugins are not audited for security by the Krew maintainers. + Run them at your own risk. +``` + +**EFS CSI ๋“œ๋ผ์ด๋ฒ„ ๋กœ๊ทธ ํ™•์ธ** + +```bash +kubectl stern -n kube-system -l app=efs-csi-controller -c csi-provisioner +``` + +![Image](https://github.com/user-attachments/assets/c203a6f2-3f8b-40d9-9191-4b45fbaa38c8) + +### **6. Pod ๋‚ด๋ถ€์—์„œ EFS ๋งˆ์šดํŠธ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl exec -it efs-app -- sh -c "df -hT -t nfs4" +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +127.0.0.1:/ nfs4 8.0E 0 8.0E 0% /data +``` + +- EFS๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ๋งˆ์šดํŠธ๋จ +- ๋™์ ์œผ๋กœ ํฌ๊ธฐ ํ™•์žฅ ๊ฐ€๋Šฅ + +### **7. EFS AccessPoints๋ฅผ ํ†ตํ•œ ์ €์žฅ์†Œ ๊ด€๋ฆฌ** + +- **AccessPoints๋ฅผ ํ™œ์šฉํ•˜์—ฌ ํ™˜๊ฒฝ๋ณ„ ์ €์žฅ์†Œ ๋ถ„๋ฆฌ ๊ฐ€๋Šฅ** + +![Image](https://github.com/user-attachments/assets/07ece9ff-6f58-47b7-bbcb-7178ce9fa4bc) + +### **8. ๊ณต์œ  ์ €์žฅ์†Œ ๋‚ด ๋ฐ์ดํ„ฐ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host specs]# tree /mnt/myefs +``` + +โœ… **์ถœ๋ ฅ** + +```bash +/mnt/myefs +โ”œโ”€โ”€ dynamic_provisioning +โ”‚ย ย  โ””โ”€โ”€ default +โ”‚ย ย  โ””โ”€โ”€ efs-claim-1a0017bc-6165-44fb-ac74-5d623fd39925 +โ”‚ย ย  โ””โ”€โ”€ out +โ”œโ”€โ”€ memo.txt +โ”œโ”€โ”€ out1.txt +โ””โ”€โ”€ out2.txt + +3 directories, 4 files +``` + +- PVC๋ณ„ ๊ฐœ๋ณ„ ๋””๋ ‰ํ† ๋ฆฌ๊ฐ€ ์ž๋™ ์ƒ์„ฑ๋จ +- ๋™์  ๋ณผ๋ฅจ ํ• ๋‹น์ด ์ •์ƒ์ ์œผ๋กœ ์ˆ˜ํ–‰๋จ + +### **9. ๋ฆฌ์†Œ์Šค ์ •๋ฆฌ** + +```bash +(eks-user:default) [root@operator-host specs]# kubectl delete -f pod.yaml +persistentvolumeclaim "efs-claim" deleted +pod "efs-app" deleted +``` + +```bash +(eks-user:default) [root@operator-host specs]# kubectl delete -f storageclass.yaml +storageclass.storage.k8s.io "efs-sc" deleted + +(eks-user:default) [root@operator-host specs]# cd $HOME +``` + +--- + +## **๐Ÿ—๏ธ EKS ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด ๊ธฐ๋ฐ˜ Persistent Volumes ๋ฐ NodeGroup ์ถ”๊ฐ€** + +### **1. ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด(Instance Store)๋ž€?** + +- ๋ฌผ๋ฆฌ ์„œ๋ฒ„์— ๋กœ์ปฌ ์ €์žฅ์†Œ๋ฅผ ์ œ๊ณตํ•˜๋Š” EC2 ์ธ์Šคํ„ด์Šค์˜ ์ €์žฅ์†Œ +- ์ธ์Šคํ„ด์Šค๊ฐ€ ์ข…๋ฃŒ๋˜๋ฉด ๋ฐ์ดํ„ฐ๊ฐ€ ์‚ญ์ œ๋  ๊ฐ€๋Šฅ์„ฑ์ด ์žˆ์Œ +- **IO ์„ฑ๋Šฅ์ด ๋›ฐ์–ด๋‚˜๋ฉฐ ๋งค์šฐ ๋น ๋ฅธ ์†๋„๋กœ ๋™์ž‘** + +### **2. ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด ์ง€์› ์ธ์Šคํ„ด์Šค ์œ ํ˜• ๋ฐ ์šฉ๋Ÿ‰ ์กฐํšŒ** + +```bash +aws ec2 describe-instance-types \ + --filters "Name=instance-type,Values=c5*" "Name=instance-storage-supported,Values=true" \ + --query "InstanceTypes[].[InstanceType, InstanceStorageInfo.TotalSizeInGB]" \ + --output table +``` + +โœ… **์ถœ๋ ฅ** + +```bash +-------------------------- +| DescribeInstanceTypes | ++---------------+--------+ +| c5d.large | 50 | +| c5d.12xlarge | 1800 | +| c5d.2xlarge | 200 | +| c5d.24xlarge | 3600 | +| c5d.4xlarge | 400 | +| c5d.18xlarge | 1800 | +| c5d.xlarge | 100 | +| c5d.metal | 3600 | +| c5d.9xlarge | 900 | ++---------------+--------+ +``` + +### **3. ์„œ๋ธŒ๋„ท ๋ฐ SSH ํ‚คํŽ˜์–ด ์„ค์ •** + +- ๋…ธ๋“œ ๊ทธ๋ฃน์ด ๋ฐฐํฌ๋  ์„œ๋ธŒ๋„ท ๋ฐ SSH ํ‚คํŽ˜์–ด ์„ค์ • + +```bash +export PubSubnet1=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet1" --query "Subnets[0].[SubnetId]" --output text) +export PubSubnet2=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet2" --query "Subnets[0].[SubnetId]" --output text) +export PubSubnet3=$(aws ec2 describe-subnets --filters Name=tag:Name,Values="$CLUSTER_NAME-Vpc1PublicSubnet3" --query "Subnets[0].[SubnetId]" --output text) +echo $PubSubnet1 $PubSubnet2 $PubSubnet3 + +SSHKEYNAME=kp-aews +``` + +### **4. ์‹ ๊ทœ NodeGroup ๊ตฌ์„ฑ ํŒŒ์ผ ์ƒ์„ฑ** + +```bash +cat << EOF > myng2.yaml +apiVersion: eksctl.io/v1alpha5 +kind: ClusterConfig +metadata: + name: myeks + region: ap-northeast-2 + version: "1.31" + +managedNodeGroups: +- amiFamily: AmazonLinux2 + desiredCapacity: 1 + instanceType: c5d.large + labels: + alpha.eksctl.io/cluster-name: myeks + alpha.eksctl.io/nodegroup-name: ng2 + disk: instancestore + maxPodsPerNode: 110 + maxSize: 1 + minSize: 1 + name: ng2 + ssh: + allow: true + publicKeyName: $SSHKEYNAME + subnets: + - $PubSubnet1 + - $PubSubnet2 + - $PubSubnet3 + tags: + alpha.eksctl.io/nodegroup-name: ng2 + alpha.eksctl.io/nodegroup-type: managed + volumeIOPS: 3000 + volumeSize: 30 + volumeThroughput: 125 + volumeType: gp3 + preBootstrapCommands: + - | + # Install Tools + yum install nvme-cli links tree jq tcpdump sysstat -y + + # Filesystem & Mount + mkfs -t xfs /dev/nvme1n1 + echo /dev/nvme1n1 /data xfs defaults,noatime 0 2 >> /etc/fstab +EOF +``` + +- **์ธ์Šคํ„ด์Šค ํƒ€์ž…**: `c5d.large` (50GB ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด ์ œ๊ณต) +- **๋””์Šคํฌ ๋งˆ์šดํŠธ**: `/dev/nvme1n1 โ†’ /data` +- **์‚ฌ์ „ ์„ค์น˜ ํŒจํ‚ค์ง€**: `nvme-cli, jq, tree, tcpdump, sysstat` +- **๋ณผ๋ฅจ ํฌ๊ธฐ ๋ฐ ์„ฑ๋Šฅ ์„ค์ •**: 30GB (gp3, 3000 IOPS, 125MB/s) + +### **5. ์‹ ๊ทœ NodeGroup ๋ฐฐํฌ** + +```bash +eksctl create nodegroup -f myng2.yaml +``` + +โœ… **์ถœ๋ ฅ** + +```bash +2025-02-18 23:40:47 [โ„น] nodegroup "ng2" will use "" [AmazonLinux2/1.31] +2025-02-18 23:40:47 [โ„น] using EC2 key pair "kp-aews" +2025-02-18 23:40:48 [โ„น] 1 existing nodegroup(s) (ng1) will be excluded +2025-02-18 23:40:48 [โ„น] 1 nodegroup (ng2) was included (based on the include/exclude rules) +2025-02-18 23:40:48 [โ„น] will create a CloudFormation stack for each of 1 managed nodegroups in cluster "myeks" +2025-02-18 23:40:49 [โ„น] +2 sequential tasks: { fix cluster compatibility, 1 task: { 1 task: { create managed nodegroup "ng2" } } +} +2025-02-18 23:40:49 [โ„น] checking cluster stack for missing resources +2025-02-18 23:40:49 [โ„น] cluster stack has all required resources +2025-02-18 23:40:49 [โ„น] building managed nodegroup stack "eksctl-myeks-nodegroup-ng2" +2025-02-18 23:40:49 [โ„น] deploying stack "eksctl-myeks-nodegroup-ng2" +2025-02-18 23:40:49 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng2" +2025-02-18 23:41:20 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng2" +2025-02-18 23:42:18 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng2" +2025-02-18 23:43:00 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng2" +2025-02-18 23:43:38 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng2" +2025-02-18 23:43:38 [โ„น] no tasks +2025-02-18 23:43:38 [โœ”] created 0 nodegroup(s) in cluster "myeks" +2025-02-18 23:43:38 [โ„น] nodegroup "ng2" has 1 node(s) +2025-02-18 23:43:38 [โ„น] node "ip-192-168-3-139.ap-northeast-2.compute.internal" is ready +2025-02-18 23:43:38 [โ„น] waiting for at least 1 node(s) to become ready in "ng2" +2025-02-18 23:43:38 [โ„น] nodegroup "ng2" has 1 node(s) +2025-02-18 23:43:38 [โ„น] node "ip-192-168-3-139.ap-northeast-2.compute.internal" is ready +2025-02-18 23:43:38 [โœ”] created 1 managed nodegroup(s) in cluster "myeks" +2025-02-18 23:43:38 [โ„น] checking security group configuration for all nodegroups +2025-02-18 23:43:38 [โ„น] all nodegroups have up-to-date cloudformation templates +``` + +### **6. ๋ฐฐํฌ๋œ ๋…ธ๋“œ ํ™•์ธ** + +```bash +kubectl get node --label-columns=node.kubernetes.io/instance-type,eks.amazonaws.com/capacityType,topology.kubernetes.io/zone +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION INSTANCE-TYPE CAPACITYTYPE ZONE +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 11h v1.31.5-eks-5d632ec t3.medium ON_DEMAND ap-northeast-2a +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 11h v1.31.5-eks-5d632ec t3.medium ON_DEMAND ap-northeast-2b +ip-192-168-3-139.ap-northeast-2.compute.internal Ready 2m59s v1.31.5-eks-5d632ec c5d.large ON_DEMAND ap-northeast-2c +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 11h v1.31.5-eks-5d632ec t3.medium ON_DEMAND ap-northeast-2c +``` + +- ๋…ธ๋“œ `c5d.large`๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ์ถ”๊ฐ€๋จ + +```bash +kubectl get node -l disk=instancestore +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION +ip-192-168-3-139.ap-northeast-2.compute.internal Ready 3m29s v1.31.5-eks-5d632ec +``` + +- ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด๊ฐ€ ์žˆ๋Š” ๋…ธ๋“œ ํ™•์ธ ์™„๋ฃŒ + +### **7. ๋ณด์•ˆ ๊ทธ๋ฃน ์„ค์ • (ng2-remoteAccess ํฌํ•จ)** + +```bash +export NG2SGID=$(aws ec2 describe-security-groups --filters "Name=group-name,Values=*ng2-remoteAccess*" --query 'SecurityGroups[*].GroupId' --output text) +aws ec2 authorize-security-group-ingress --group-id $NG2SGID --protocol '-1' --cidr $(curl -s ipinfo.io/ip)/32 +aws ec2 authorize-security-group-ingress --group-id $NG2SGID --protocol '-1' --cidr 172.20.1.100/32 +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "Return": true, + "SecurityGroupRules": [ + { + "SecurityGroupRuleId": "sgr-01263a4b3dd1797db", + "GroupId": "sg-0e186958bfe3d2895", + "GroupOwnerId": "378102432899", + "IsEgress": false, + "IpProtocol": "-1", + "FromPort": -1, + "ToPort": -1, + "CidrIpv4": "182.230.60.93/32", + "SecurityGroupRuleArn": "arn:aws:ec2:ap-northeast-2:378102432899:security-group-rule/sgr-01263a4b3dd1797db" + } + ] +} +{ + "Return": true, + "SecurityGroupRules": [ + { + "SecurityGroupRuleId": "sgr-004bfcccf4933bca4", + "GroupId": "sg-0e186958bfe3d2895", + "GroupOwnerId": "378102432899", + "IsEgress": false, + "IpProtocol": "-1", + "FromPort": -1, + "ToPort": -1, + "CidrIpv4": "172.20.1.100/32", + "SecurityGroupRuleArn": "arn:aws:ec2:ap-northeast-2:378102432899:security-group-rule/sgr-004bfcccf4933bca4" + } + ] +} +``` + +- ๋ณด์•ˆ ๊ทธ๋ฃน ๊ทœ์น™ ์ถ”๊ฐ€ ์™„๋ฃŒ + + +c5d.large ์ธ์Šคํ„ด์Šค์˜ ๊ณต์ธ IP: 3.34.90.173 + +![Image](https://github.com/user-attachments/assets/2759253f-bc7b-48de-9af0-ac2fc1eef997) + +### **8. SSH๋ฅผ ํ†ตํ•œ ์›Œ์ปค ๋…ธ๋“œ ์ ‘์†** + +```bash +N4=3.34.90.173 +ssh ec2-user@$N4 hostname +``` + +โœ… **์ถœ๋ ฅ** + +```bash +The authenticity of host '3.34.90.173 (3.34.90.173)' can't be established. +ED25519 key fingerprint is SHA256:8982ohpoaUv/4ImQwqxA8Ye4HDQZbziZ+n7vcW++NKw. +This key is not known by any other names. +Are you sure you want to continue connecting (yes/no/[fingerprint])? yes +Warning: Permanently added '3.34.90.173' (ED25519) to the list of known hosts. +ip-192-168-3-139.ap-northeast-2.compute.internal +``` + +- SSH ์ ‘์† ํ™•์ธ + +### **9. ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด ๋””์Šคํฌ ํ™•์ธ** + +์ธ์Šคํ„ด์Šค์—์„œ ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ NVMe ๋””์Šคํฌ ๋ชฉ๋ก ์กฐํšŒ + +```bash +ssh ec2-user@$N4 sudo nvme list +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Node SN Model Namespace Usage Format FW Rev +---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- +/dev/nvme0n1 vol0c28a9e99805af630 Amazon Elastic Block Store 1 32.21 GB / 32.21 GB 512 B + 0 B 1.0 +/dev/nvme1n1 AWS3E4B64A6880B81A2C Amazon EC2 NVMe Instance Storage 1 50.00 GB / 50.00 GB 512 B + 0 B 0 +``` + +- EBS(30GB) ์™ธ์—๋„ 50GB ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด ์ถ”๊ฐ€ ํ™•์ธ๋จ + +### **10. ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด ๋งˆ์šดํŠธ ํ™•์ธ** + +`nvme1n1` ๋””์Šคํฌ๊ฐ€ `/data`์— ๋งˆ์šดํŠธ๋˜์–ด ์žˆ๋Š”์ง€ ํ™•์ธ + +```bash +ssh ec2-user@$N4 sudo lsblk -e 7 -d +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT +nvme1n1 259:0 0 46.6G 0 disk /data +nvme0n1 259:1 0 30G 0 disk +``` + +- 50GB ์ธ์Šคํ„ด์Šค ์Šคํ† ์–ด ๋””์Šคํฌ๊ฐ€ `/data`์— ๋งˆ์šดํŠธ๋จ + +```bash +ssh ec2-user@$N4 sudo df -hT -t xfs +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Filesystem Type Size Used Avail Use% Mounted on +/dev/nvme0n1p1 xfs 30G 3.7G 27G 13% / +/dev/nvme1n1 xfs 47G 365M 47G 1% /data +``` + +- ๋””์Šคํฌ๊ฐ€ `XFS` ํŒŒ์ผ ์‹œ์Šคํ…œ์œผ๋กœ ํฌ๋งท๋˜์–ด `/data`์— ์ •์ƒ์ ์œผ๋กœ ๋งˆ์šดํŠธ๋จ + +```bash +ssh ec2-user@$N4 sudo tree /data +``` + +โœ… **์ถœ๋ ฅ** + +```bash +/data + +0 directories, 0 files +``` + +- ๋ฐ์ดํ„ฐ ๋””๋ ‰ํ† ๋ฆฌ์— ์•„๋ฌด ํŒŒ์ผ๋„ ์—†์Œ + +### **11. fstab ์„ค์ • ํ™•์ธ (๋ถ€ํŒ… ์‹œ ์ž๋™ ๋งˆ์šดํŠธ ์—ฌ๋ถ€)** + +```bash +ssh ec2-user@$N4 sudo cat /etc/fstab +``` + +โœ… **์ถœ๋ ฅ** + +```bash +UUID=1dfdfe0d-276a-4d52-8572-ceb3b011d9ea / xfs defaults,noatime 1 1 +/dev/nvme1n1 /data xfs defaults,noatime 0 2 +``` + +- ์žฌ๋ถ€ํŒ… ์‹œ `/data`๊ฐ€ ์ž๋™ ๋งˆ์šดํŠธ๋˜๋„๋ก ์„ค์ •๋จ + +### **12. ๋…ธ๋“œ ๋ฆฌ์†Œ์Šค ํ™•์ธ** + +```bash +kubectl describe node -l disk=instancestore | grep Allocatable: -A7 +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Allocatable: + cpu: 1930m + ephemeral-storage: 27905944324 + hugepages-1Gi: 0 + hugepages-2Mi: 0 + memory: 3097488Ki + pods: 110 +System Info: +``` + +- `ephemeral-storage` ๋ฐ `pods` ๊ฐœ์ˆ˜ ํ™•์ธ ๊ฐ€๋Šฅ + +### **13. Kubelet ํŒŒ๋ผ๋ฏธํ„ฐ ํ™•์ธ** + +```bash +ssh ec2-user@$N4 sudo ps -ef | grep kubelet +``` + +โœ… **์ถœ๋ ฅ** + +```bash +root 3014 1 0 14:42 ? 00:00:07 /usr/bin/kubelet --config /etc/kubernetes/kubelet/kubelet-config.json --kubeconfig /var/lib/kubelet/kubeconfig --container-runtime-endpoint unix:///run/containerd/containerd.sock --image-credential-provider-config /etc/eks/image-credential-provider/config.json --image-credential-provider-bin-dir /etc/eks/image-credential-provider --node-ip=192.168.3.139 --pod-infra-container-image=602401143452.dkr.ecr.ap-northeast-2.amazonaws.com/eks/pause:3.5 --v=2 --hostname-override=ip-192-168-3-139.ap-northeast-2.compute.internal --cloud-provider=external --node-labels=eks.amazonaws.com/sourceLaunchTemplateVersion=1,alpha.eksctl.io/cluster-name=myeks,alpha.eksctl.io/nodegroup-name=ng2,disk=instancestore,eks.amazonaws.com/nodegroup-image=ami-0fa05db9e3c145f63,eks.amazonaws.com/capacityType=ON_DEMAND,eks.amazonaws.com/nodegroup=ng2,eks.amazonaws.com/sourceLaunchTemplateId=lt-0d2cf44115bd914f2 --max-pods=29 --max-pods=110 +root 3602 3110 0 14:42 ? 00:00:00 /csi-node-driver-registrar --csi-address=/csi/csi.sock --kubelet-registration-path=/var/lib/kubelet/plugins/efs.csi.aws.com/csi.sock --v=2 +root 4015 3921 0 14:42 ? 00:00:00 /bin/aws-ebs-csi-driver node --endpoint=unix:/csi/csi.sock --http-endpoint=0.0.0.0:3302 --csi-mount-point-prefix=/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/ --volume-attach-limit=31 --logging-format=text --v=2 +root 4063 3921 0 14:42 ? 00:00:00 /csi-node-driver-registrar --csi-address=/csi/csi.sock --kubelet-registration-path=/var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock --v=2 +``` + +- `--max-pods=110` ๊ฐ’์ด ์ตœ์ข… ์ ์šฉ๋จ + +### **14. ๊ธฐ์กด local-path ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ์‚ญ์ œ** + +```bash +k get sc +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE +gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 12h +gp3 (default) ebs.csi.aws.com Delete WaitForFirstConsumer true 3h32m +local-path rancher.io/local-path Delete WaitForFirstConsumer false 6h53m +``` + +```bash +kubectl delete -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.31/deploy/local-path-storage.yaml + +# ๊ฒฐ๊ณผ +namespace "local-path-storage" deleted +serviceaccount "local-path-provisioner-service-account" deleted +role.rbac.authorization.k8s.io "local-path-provisioner-role" deleted +clusterrole.rbac.authorization.k8s.io "local-path-provisioner-role" deleted +rolebinding.rbac.authorization.k8s.io "local-path-provisioner-bind" deleted +clusterrolebinding.rbac.authorization.k8s.io "local-path-provisioner-bind" deleted +deployment.apps "local-path-provisioner" deleted +storageclass.storage.k8s.io "local-path" deleted +configmap "local-path-config" deleted +``` + +### **15. ์ƒˆ๋กœ์šด local-path ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ๋ฐฐํฌ** + +```bash +curl -sL https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.31/deploy/local-path-storage.yaml | sed 's/opt/data/g' | kubectl apply -f - + +# ๊ฒฐ๊ณผ +namespace/local-path-storage created +serviceaccount/local-path-provisioner-service-account created +role.rbac.authorization.k8s.io/local-path-provisioner-role created +clusterrole.rbac.authorization.k8s.io/local-path-provisioner-role created +rolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created +clusterrolebinding.rbac.authorization.k8s.io/local-path-provisioner-bind created +deployment.apps/local-path-provisioner created +storageclass.storage.k8s.io/local-path created +configmap/local-path-config created +``` + +### **16. local-path ์„ค์ • ํ™•์ธ** + +```bash +kubectl describe cm -n local-path-storage local-path-config +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: local-path-config +Namespace: local-path-storage +Labels: +Annotations: + +Data +==== +config.json: +---- +{ + "nodePathMap":[ + { + "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES", + "paths":["/data/local-path-provisioner"] + } + ] +} + +helperPod.yaml: +---- +apiVersion: v1 +kind: Pod +metadata: + name: helper-pod +spec: + priorityClassName: system-node-critical + tolerations: + - key: node.kubernetes.io/disk-pressure + operator: Exists + effect: NoSchedule + containers: + - name: helper-pod + image: busybox + imagePullPolicy: IfNotPresent + +setup: +---- +#!/bin/sh +set -eu +mkdir -m 0777 -p "$VOL_DIR" + +teardown: +---- +#!/bin/sh +set -eu +rm -rf "$VOL_DIR" + +BinaryData +==== + +Events: +``` + +- ๋กœ์ปฌ ์Šคํ† ๋ฆฌ์ง€์˜ ๋งˆ์šดํŠธ ๊ฒฝ๋กœ `/data/local-path-provisioner`๋กœ ๋ณ€๊ฒฝ๋จ + +### **17. ์ธ์Šคํ„ด์Šค ์ ‘์†** + +```bash +ssh ec2-user@$N4 +Last login: Tue Feb 18 15:07:08 2025 from 182.230.60.93 + , #_ + ~\_ ####_ Amazon Linux 2 + ~~ \_#####\ + ~~ \###| AL2 End of Life is 2026-06-30. + ~~ \#/ ___ + ~~ V~' '-> + ~~~ / A newer version of Amazon Linux is available! + ~~._. _/ + _/ _/ Amazon Linux 2023, GA and supported until 2028-03-15. + _/m/' https://aws.amazon.com/linux/amazon-linux-2023/ + +[ec2-user@ip-192-168-3-139 ~]$ +``` + +### **18. ๋””์Šคํฌ ์„ฑ๋Šฅ ๋ชจ๋‹ˆํ„ฐ๋ง (IOPS ํ™•์ธ)** + +```bash +[ec2-user@ip-192-168-3-139 ~]$ iostat -xmdz 1 -p nvme1n1 +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Linux 5.10.233-224.894.amzn2.x86_64 (ip-192-168-3-139.ap-northeast-2.compute.internal) 02/18/2025 _x86_64_ (2 CPU) + +Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util +nvme1n1 0.00 0.00 0.19 0.16 0.00 0.02 118.97 0.00 0.23 0.08 0.41 0.28 0.01 + +Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util + +Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util +``` + +- nvme1n1 ๋””์Šคํฌ์˜ ์ž…์ถœ๋ ฅ(IO) ์†๋„๋ฅผ ์‹ค์‹œ๊ฐ„์œผ๋กœ ํ™•์ธ ๊ฐ€๋Šฅ + +### **19. FIO ํ…Œ์ŠคํŠธ๋กœ IOPS ์„ฑ๋Šฅ ์ธก์ •** + +```bash +(eks-user:default) [root@operator-host ~]# kubestr fio -f fio-read.fio -s local-path --size 10G --nodeselector disk=instancestore +``` + +- FIO ๋ฒค์น˜๋งˆํฌ ํ…Œ์ŠคํŠธ๊ฐ€ ์ง„ํ–‰๋จ + +![Image](https://github.com/user-attachments/assets/a8e229cb-f43d-4b4c-9c2f-801c3deb746b) + +โœ… **์ถœ๋ ฅ** + +```bash +PVC created kubestr-fio-pvc-cvqpl +Pod created kubestr-fio-pod-xzw7g +Running FIO test (fio-read.fio) on StorageClass (local-path) with a PVC of Size (10G) +Elapsed time- 3m42.67154584s +FIO test results: + +FIO version - fio-3.36 +Global options - ioengine=libaio verify= direct=1 gtod_reduce= + +JobName: + blocksize= filesize= iodepth= rw= +read: + IOPS=20308.564453 BW(KiB/s)=81234 + iops: min=15898 max=93734 avg=20316.953125 + bw(KiB/s): min=63592 max=374940 avg=81267.796875 + +Disk stats (read/write): + nvme1n1: ios=2433492/10 merge=0/3 ticks=7648125/15 in_queue=7648141, util=99.949951% + - OK +``` + +- IOPS๊ฐ€ ๊ธฐ์กด ๋Œ€๋น„ 7๋ฐฐ ์ฆ๊ฐ€ (`IOPS=20308.564453`), ํ‰๊ท  ๋Œ€์—ญํญ 81MB/s ๋‹ฌ์„ฑ + +### **19. ๋ฐฐํฌ ๋ฆฌ์†Œ์Šค ์ •๋ฆฌ ๋ฐ ์‚ญ์ œ** + +```bash +# local-path ์Šคํ† ๋ฆฌ์ง€ ํด๋ž˜์Šค ์‚ญ์ œ +kubectl delete -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.31/deploy/local-path-storage.yaml +namespace "local-path-storage" deleted +serviceaccount "local-path-provisioner-service-account" deleted +role.rbac.authorization.k8s.io "local-path-provisioner-role" deleted +clusterrole.rbac.authorization.k8s.io "local-path-provisioner-role" deleted +rolebinding.rbac.authorization.k8s.io "local-path-provisioner-bind" deleted +clusterrolebinding.rbac.authorization.k8s.io "local-path-provisioner-bind" deleted +deployment.apps "local-path-provisioner" deleted +storageclass.storage.k8s.io "local-path" deleted +configmap "local-path-config" deleted + +# ng2 ๋…ธ๋“œ๊ทธ๋ฃน ์‚ญ์ œ +eksctl delete nodegroup -c $CLUSTER_NAME -n ng2 +2025-02-19 00:18:37 [โ„น] 1 nodegroup (ng2) was included (based on the include/exclude rules) +2025-02-19 00:18:37 [โ„น] will drain 1 nodegroup(s) in cluster "myeks" +2025-02-19 00:18:37 [โ„น] starting parallel draining, max in-flight of 1 +2025-02-19 00:18:37 [โ„น] cordon node "ip-192-168-3-139.ap-northeast-2.compute.internal" +2025-02-19 00:18:37 [โœ”] drained all nodes: [ip-192-168-3-139.ap-northeast-2.compute.internal] +2025-02-19 00:18:37 [โ„น] will delete 1 nodegroups from cluster "myeks" +2025-02-19 00:18:38 [โ„น] 1 task: { 1 task: { delete nodegroup "ng2" [async] } } +2025-02-19 00:18:38 [โ„น] will delete stack "eksctl-myeks-nodegroup-ng2" +2025-02-19 00:18:38 [โœ”] deleted 1 nodegroup(s) from cluster "myeks" +``` + +--- + +## **๐Ÿ”„ ๋ฉ€ํ‹ฐํ”Œ๋žซํผ ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ ๋ฐ ECR ๋ฐฐํฌ** + +### **1. ๋…ธ๋“œ ๊ทธ๋ฃน ๋ฐ CPU ์•„ํ‚คํ…์ฒ˜ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# arch +# ๊ฒฐ๊ณผ +x86_64 +``` + +**๋‹ค๋ฅธ ์•„ํ‚คํ…์ฒ˜(ARM, RISC-V) ๊ธฐ๋ฐ˜ ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰ ์‹œ ์˜ค๋ฅ˜ ๋ฐœ์ƒ** + +```bash +(eks-user:default) [root@operator-host ~]# docker run --rm -it riscv64/ubuntu bash +# ๊ฒฐ๊ณผ +Unable to find image 'riscv64/ubuntu:latest' locally +latest: Pulling from riscv64/ubuntu +docker: no matching manifest for linux/amd64 in the manifest list entries. +See 'docker run --help'. + +(eks-user:default) [root@operator-host ~]# docker run --rm -it arm64v8/ubuntu bash +# ๊ฒฐ๊ณผ +Unable to find image 'arm64v8/ubuntu:latest' locally +latest: Pulling from arm64v8/ubuntu +docker: no matching manifest for linux/amd64 in the manifest list entries. +See 'docker run --help' +``` + +- CPU ์•„ํ‚คํ…์ฒ˜๊ฐ€ ๋‹ค๋ฅผ ๊ฒฝ์šฐ ๊ธฐ๋ณธ์ ์œผ๋กœ ์‹คํ–‰ ๋ถˆ๊ฐ€๋Šฅ + +### **2. ๋ฉ€ํ‹ฐํ”Œ๋žซํผ ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ ํ™˜๊ฒฝ ๊ตฌ์ถ•** + +**(1) `buildx` ๋นŒ๋” ์ƒํƒœ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# docker buildx ls +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS +default * docker + default default running v0.12.5 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386 +``` + +- ๊ธฐ๋ณธ์ ์œผ๋กœ `amd64` ํ”Œ๋žซํผ๋งŒ ์ง€์›๋จ + +**(2) QEMU๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋‹ค๋ฅธ ์•„ํ‚คํ…์ฒ˜ ์ง€์› ํ™œ์„ฑํ™”** + +```bash +(eks-user:default) [root@operator-host ~]# docker run --rm --privileged multiarch/qemu-user-static --reset -p yes +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Unable to find image 'multiarch/qemu-user-static:latest' locally +latest: Pulling from multiarch/qemu-user-static +205dae5015e7: Pull complete +816739e52091: Pull complete +30abb83a18eb: Pull complete +0657daef200b: Pull complete +30c9c93f40b9: Pull complete +Digest: sha256:fe60359c92e86a43cc87b3d906006245f77bfc0565676b80004cc666e4feb9f0 +Status: Downloaded newer image for multiarch/qemu-user-static:latest +Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha +Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm +Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb +Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc +Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus +Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64 +Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc +Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64 +Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le +Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k +Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips +Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel +Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32 +Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el +Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64 +Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el +Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4 +Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb +Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x +Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64 +Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be +Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa +Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32 +Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64 +Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa +Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb +Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze +Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel +Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k +Setting /usr/bin/qemu-hexagon-static as binfmt interpreter for hexagon +``` + +- QEMU๋ฅผ ํ†ตํ•ด ๋‹ค์–‘ํ•œ ์•„ํ‚คํ…์ฒ˜ ์ง€์› ์ถ”๊ฐ€๋จ + +**(3) QEMU ์„ค์น˜ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# docker images +``` + +โœ… **์ถœ๋ ฅ** + +```bash +REPOSITORY TAG IMAGE ID CREATED SIZE +multiarch/qemu-user-static latest 3539aaa87393 2 years ago 305MB +``` + +### **3. ๋นŒ๋“œ ํ™˜๊ฒฝ ์ƒ์„ฑ ๋ฐ ํ™•์ธ** + +**(1) `buildx` ๋นŒ๋” ์ƒ์„ฑ** + +```bash +(eks-user:default) [root@operator-host ~]# docker buildx create --use --name mybuilder +# ๊ฒฐ๊ณผ +mybuilder +``` + +**(2) `buildx` ๋นŒ๋” ๋ชฉ๋ก ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# docker buildx ls +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS +mybuilder * docker-container + mybuilder0 unix:///var/run/docker.sock inactive +default docker + default default running v0.12.5 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386, linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6 +``` + +- **๋ฉ€ํ‹ฐํ”Œ๋žซํผ ๋นŒ๋“œ ํ™˜๊ฒฝ์ด ํ™œ์„ฑํ™”๋จ** + +**(3) `buildx` ๋นŒ๋” ๋ถ€ํŠธ์ŠคํŠธ๋žฉ ์‹คํ–‰** + +```bash +(eks-user:default) [root@operator-host ~]# docker buildx inspect --bootstrap +``` + +โœ… **์ถœ๋ ฅ** + +```bash +[+] Building 8.6s (1/1) FINISHED + => [internal] booting buildkit 8.6s + => => pulling image moby/buildkit:buildx-stable-1 7.9s + => => creating container buildx_buildkit_mybuilder0 0.7s +Name: mybuilder +Driver: docker-container +Last Activity: 2025-02-18 15:33:47 +0000 UTC + +Nodes: +Name: mybuilder0 +Endpoint: unix:///var/run/docker.sock +Status: running +Buildkit: v0.19.0 +Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6 +Labels: + org.mobyproject.buildkit.worker.executor: oci + org.mobyproject.buildkit.worker.hostname: 9dd2e75695e3 + org.mobyproject.buildkit.worker.network: host + org.mobyproject.buildkit.worker.oci.process-mode: sandbox + org.mobyproject.buildkit.worker.selinux.enabled: false + org.mobyproject.buildkit.worker.snapshotter: overlayfs +GC Policy rule#0: + All: false + Filters: type==source.local,type==exec.cachemount,type==source.git.checkout + Keep Duration: 48h0m0s +GC Policy rule#1: + All: false + Keep Duration: 1440h0m0s + Keep Bytes: 2.794GiB +GC Policy rule#2: + All: false + Keep Bytes: 2.794GiB +GC Policy rule#3: + All: true + Keep Bytes: 2.794GiB +``` + +- ์ด์ œ ์—ฌ๋Ÿฌ ์•„ํ‚คํ…์ฒ˜์˜ ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ ๊ฐ€๋Šฅ + +**(4) `buildx` ๋นŒ๋” ๋ชฉ๋ก ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# docker buildx ls +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS +mybuilder * docker-container + mybuilder0 unix:///var/run/docker.sock running v0.19.0 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6 +default docker + default default running v0.12.5 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386, linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6 +``` + +- `linux/amd64`, `linux/arm64`, `linux/riscv64` ๋“ฑ ๋‹ค์ค‘ ํ”Œ๋žซํผ ์ง€์› ํ™•์ธ ๊ฐ€๋Šฅ + +**(5) `buildx` ์‹คํ–‰ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host ~]# docker ps +``` + +โœ… **์ถœ๋ ฅ** + +```bash +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +9dd2e75695e3 moby/buildkit:buildx-stable-1 "buildkitd" 58 seconds ago Up 57 seconds buildx_buildkit_mybuilder0 +``` + +- moby/buildkit ์ปจํ…Œ์ด๋„ˆ๊ฐ€ ์‹คํ–‰๋˜๋ฉฐ ๋ฉ€ํ‹ฐํ”Œ๋žซํผ ๋นŒ๋“œ ๊ฐ€๋Šฅ + +### **4. ์ƒ˜ํ”Œ ์ปจํ…Œ์ด๋„ˆ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ์ƒ์„ฑ** + +**(1) ํ”„๋กœ์ ํŠธ ๋””๋ ‰ํ† ๋ฆฌ ์ƒ์„ฑ** + +```bash +(eks-user:default) [root@operator-host ~]# mkdir myweb && cd myweb +``` + +**(2) `server.py` ์ž‘์„ฑ** + +```bash +(eks-user:default) [root@operator-host myweb]# cat > server.py < from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler +> from datetime import datetime +> import socket +> +> class RequestHandler(BaseHTTPRequestHandler): +> def do_GET(self): +> self.send_response(200) +> self.send_header('Content-type', 'text/plain') +> self.end_headers() +> +> now = datetime.now() +> hostname = socket.gethostname() +> response_string = now.strftime("The time is %-I:%M:%S %p, VERSION 0.0.1\n") +> response_string += f"Server hostname: {hostname}\n" +> self.wfile.write(bytes(response_string, "utf-8")) +> +> def startServer(): +> try: +> server = ThreadingHTTPServer(('', 80), RequestHandler) +> print("Listening on " + ":".join(map(str, server.server_address))) +> server.serve_forever() +> except KeyboardInterrupt: +> server.shutdown() +> +> if __name__ == "__main__": +> startServer() +> EOF +``` + +**(3) `Dockerfile` ์ž‘์„ฑ** + +```bash +(eks-user:default) [root@operator-host myweb]# cat > Dockerfile < FROM python:3.12 +> ENV PYTHONUNBUFFERED 1 +> COPY . /app +> WORKDIR /app +> CMD python3 server.py +> EOF +``` + +### **5. ๋‹จ์ผ ํ”Œ๋žซํผ ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ ๋ฐ ์‹คํ–‰** + +**(1) Python 3.12 ๊ธฐ๋ฐ˜ ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ๋‹ค์šด๋กœ๋“œ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker pull python:3.12 + +# ๊ฒฐ๊ณผ +3.12: Pulling from library/python +a492eee5e559: Pull complete +32b550be6cb6: Pull complete +35af2a7690f2: Pull complete +7576b00d9bb1: Pull complete +07612085660d: Pull complete +60fd44efca0f: Pull complete +4a12975a6131: Pull complete +Digest: sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 +Status: Downloaded newer image for python:3.12 +docker.io/library/python:3.12 +``` + +**(2) Python 3.12 ๊ธฐ๋ฐ˜ ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker build -t myweb:1 -t myweb:latest . + +# ๊ฒฐ๊ณผ +[+] Building 0.3s (8/8) FINISHED docker:default + => [internal] load build definition from Dockerfile 0.0s + => => transferring dockerfile: 125B 0.0s + => [internal] load metadata for docker.io/library/python:3.12 0.0s + => [internal] load .dockerignore 0.0s + => => transferring context: 2B 0.0s + => [internal] load build context 0.0s + => => transferring context: 1.04kB 0.0s + => [1/3] FROM docker.io/library/python:3.12 0.1s + => [2/3] COPY . /app 0.2s + => [3/3] WORKDIR /app 0.0s + => exporting to image 0.0s + => => exporting layers 0.0s + => => writing image sha256:e8e1bb0cc5209d01fa642497dc930b06db1e6317d3eacc9b94c40033b7436efd 0.0s + => => naming to docker.io/library/myweb:1 0.0s + => => naming to docker.io/library/myweb:latest 0.0s +``` + +- `myweb:1` ๋ฐ `myweb:latest` ํƒœ๊ทธ๋กœ ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ๋นŒ๋“œ ์™„๋ฃŒ + +### **6. ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker images +``` + +โœ… **์ถœ๋ ฅ** + +```bash +REPOSITORY TAG IMAGE ID CREATED SIZE +myweb 1 e8e1bb0cc520 27 seconds ago 1.02GB +myweb latest e8e1bb0cc520 27 seconds ago 1.02GB +python 3.12 149b9784258f 13 days ago 1.02GB +moby/buildkit buildx-stable-1 23b5a9d195cf 4 weeks ago 208MB +multiarch/qemu-user-static latest 3539aaa87393 2 years ago 305MB +``` + +- ๋นŒ๋“œ๋œ `myweb:1` ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ํ™•์ธ + +### **7. ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰ ๋ฐ ์„œ๋น„์Šค ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker run -d -p 8080:80 --name=timeserver myweb + +# ๊ฒฐ๊ณผ +2651f49a0fe29182098c0c6e185ee81e63816f1ddf034fa2f32cc58383f7badb +``` + +- `timeserver` ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰ +- 8080 ํฌํŠธ๋กœ HTTP ์š”์ฒญ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Œ + +```bash +(eks-user:default) [root@operator-host myweb]# curl http://localhost:8080 +``` + +โœ… **์ถœ๋ ฅ** + +```bash +The time is 3:51:20 PM, VERSION 0.0.1 +Server hostname: 2651f49a0fe2 +``` + +- ํ˜„์žฌ ์‹œ๊ฐ„๊ณผ ์ปจํ…Œ์ด๋„ˆ ํ˜ธ์ŠคํŠธ๋ช…์„ ๋ฐ˜ํ™˜ํ•˜๋Š” ๊ฐ„๋‹จํ•œ ์›น ์„œ๋ฒ„ ๋™์ž‘ ํ™•์ธ + +### **8. ์ปจํ…Œ์ด๋„ˆ ์ •๋ฆฌ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker rm -f timeserver + +# ๊ฒฐ๊ณผ +timeserver +``` + +### **9. Docker Hub ๋กœ๊ทธ์ธ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker login +Log in with your Docker ID or email address to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com/ to create one. +You can log in with your password or a Personal Access Token (PAT). Using a limited-scope PAT grants better security and is required for organizations using SSO. Learn more at https://docs.docker.com/go/access-tokens/ + +Username: shinminjin +Password: +WARNING! Your password will be stored unencrypted in /root/.docker/config.json. +Configure a credential helper to remove this warning. See +https://docs.docker.com/engine/reference/commandline/login/#credentials-store + +Login Succeeded +``` + +### **10. Docker ์‚ฌ์šฉ์ž๋ช… ๋ณ€์ˆ˜ ์„ค์ •** + +```bash +(eks-user:default) [root@operator-host myweb]# DOCKERNAME=shinminjin +``` + +### **11. ๋‹ค์ค‘ ์•„ํ‚คํ…์ฒ˜ ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ ๋ฐ ํ‘ธ์‹œ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker buildx build --platform linux/amd64,linux/arm64 --push --tag $DOCKERNAME/myweb:multi . +``` + +โœ… **์ถœ๋ ฅ** + +```bash +[+] Building 106.5s (14/14) FINISHED docker-container:mybuilder + => [internal] load build definition from Dockerfile 0.0s + => => transferring dockerfile: 125B 0.0s + => [linux/arm64 internal] load metadata for docker.io/library/python:3.12 2.6s + => [linux/amd64 internal] load metadata for docker.io/library/python:3.12 2.5s + => [auth] library/python:pull token for registry-1.docker.io 0.0s + => [internal] load .dockerignore 0.0s + => => transferring context: 2B 0.0s + => [linux/amd64 1/3] FROM docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 36.9s + => => resolve docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 0.0s + => => sha256:4a12975a6131fb8b18f6c80441a8533d18ec06d744cd9dd26431ae147a1b7552 248B / 248B 0.5s + => => sha256:60fd44efca0fdc711987188c7b289674aea93ed94d2beff8a1e4d92a8e1fbe7f 25.66MB / 25.66MB 1.1s + => => sha256:7576b00d9bb10cc967bb5bdeeb3d5fa078ac8800e112aa03ed15ec199662d4f7 211.33MB / 211.33MB 5.7s + => => sha256:07612085660d86eae935f91c31ea91065995815b395c798b5c0f8df260c7e2a8 6.16MB / 6.16MB 0.7s + => => sha256:35af2a7690f2b43e7237d1fae8e3f2350dfb25f3249e9cf65121866f9c56c772 64.39MB / 64.39MB 2.6s + => => sha256:32b550be6cb62359a0f3a96bc0dc289f8b45d097eaad275887f163c6780b4108 24.06MB / 24.06MB 1.7s + => => sha256:a492eee5e55976c7d3feecce4c564aaf6f14fb07fdc5019d06f4154eddc93fde 48.48MB / 48.48MB 1.9s + => => extracting sha256:a492eee5e55976c7d3feecce4c564aaf6f14fb07fdc5019d06f4154eddc93fde 4.3s + => => extracting sha256:32b550be6cb62359a0f3a96bc0dc289f8b45d097eaad275887f163c6780b4108 1.5s + => => extracting sha256:35af2a7690f2b43e7237d1fae8e3f2350dfb25f3249e9cf65121866f9c56c772 5.4s + => => extracting sha256:7576b00d9bb10cc967bb5bdeeb3d5fa078ac8800e112aa03ed15ec199662d4f7 15.7s + => => extracting sha256:07612085660d86eae935f91c31ea91065995815b395c798b5c0f8df260c7e2a8 0.6s + => => extracting sha256:60fd44efca0fdc711987188c7b289674aea93ed94d2beff8a1e4d92a8e1fbe7f 1.6s + => => extracting sha256:4a12975a6131fb8b18f6c80441a8533d18ec06d744cd9dd26431ae147a1b7552 0.0s + => [linux/arm64 1/3] FROM docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 36.1s + => => resolve docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 0.0s + => => sha256:305935ae2ee22ec786106269d7da84b1df25782c691db84256ebce68190e1c79 250B / 250B 0.5s + => => sha256:d708adbcc8ee3a9b1354ffe945eb8875d5e843a6aceb8397559caf8ef2ffd214 24.91MB / 24.91MB 1.1s + => => sha256:7d49e4574da2d8f2419ffdbd2cfc65c6be873a05dab52159fb02b63d8be816fe 6.24MB / 6.24MB 1.0s + => => sha256:9611c2b713640ce0f9156445b244c4da5e621183b56c0901d97a8b6d54ce10d7 202.72MB / 202.72MB 6.2s + => => sha256:c9d3572a68af0b860060b7ea84adfa8406fa20cfd1337c947dfb661aa965eee7 64.36MB / 64.36MB 2.8s + => => sha256:193c44006e77abbadfdd7be72b4ab6d7a5c08640ef575970f722b798ee7800ac 23.60MB / 23.60MB 1.3s + => => sha256:106abeaee908db66722312b3379ae398e2bcc5b2fdad0cc248509efa14a819ff 48.31MB / 48.31MB 2.2s + => => extracting sha256:106abeaee908db66722312b3379ae398e2bcc5b2fdad0cc248509efa14a819ff 8.1s + => => extracting sha256:193c44006e77abbadfdd7be72b4ab6d7a5c08640ef575970f722b798ee7800ac 1.6s + => => extracting sha256:c9d3572a68af0b860060b7ea84adfa8406fa20cfd1337c947dfb661aa965eee7 5.4s + => => extracting sha256:9611c2b713640ce0f9156445b244c4da5e621183b56c0901d97a8b6d54ce10d7 14.6s + => => extracting sha256:7d49e4574da2d8f2419ffdbd2cfc65c6be873a05dab52159fb02b63d8be816fe 0.6s + => => extracting sha256:d708adbcc8ee3a9b1354ffe945eb8875d5e843a6aceb8397559caf8ef2ffd214 2.2s + => => extracting sha256:305935ae2ee22ec786106269d7da84b1df25782c691db84256ebce68190e1c79 0.0s + => [internal] load build context 0.0s + => => transferring context: 1.04kB 0.0s + => [linux/arm64 2/3] COPY . /app 0.2s + => [linux/arm64 3/3] WORKDIR /app 0.0s + => [linux/amd64 2/3] COPY . /app 0.1s + => [linux/amd64 3/3] WORKDIR /app 0.0s + => exporting to image 66.8s + => => exporting layers 0.1s + => => exporting manifest sha256:b73a3a9ae85ac83166587bc398a9cda723b1296cd200e7bf61a2f8fdc03f22d6 0.0s + => => exporting config sha256:2e766855d2c0dc9c8a005866ec139d251b4f3cabd38586fff3fa9df874b1b14f 0.0s + => => exporting attestation manifest sha256:be4b65e573ddbae236084237c32bb3e61598865f2378464082b008a268349bd7 0.0s + => => exporting manifest sha256:5d18951aa10d7db756c39875c1360499c3a6643a097f9ea81ed8fdc7479c9b49 0.0s + => => exporting config sha256:c319f11eb4c8a52e492b75bd068715a6703f6d0afcf6d6240dbed9c0ad1011e8 0.0s + => => exporting attestation manifest sha256:eaa3e8e820eb8ee42165d8d674562cc887cdf8b0e9ba7d26d24e05bde231ce07 0.0s + => => exporting manifest list sha256:1c6d6bc4f4383a11c8d033222ba28e68073abd45e0de31272d0b94020916ee56 0.0s + => => pushing layers 62.4s + => => pushing manifest for docker.io/shinminjin/myweb:multi@sha256:1c6d6bc4f4383a11c8d033222ba28e68073abd45e0de31272d0b94020916ee56 4.2s + => [auth] shinminjin/myweb:pull,push token for registry-1.docker.io 0.0s + + 2 warnings found (use --debug to expand): + - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 2) + - JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals (line 5) +``` + +- `linux/amd64`, `linux/arm64` ์•„ํ‚คํ…์ฒ˜๋ฅผ ์ง€์›ํ•˜๋Š” ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ๋นŒ๋“œ +- ๋นŒ๋“œ๋œ ์ด๋ฏธ์ง€๋ฅผ Docker Hub๋กœ ํ‘ธ์‹œ + +### **12. ๋นŒ๋“œ๋œ ์ปจํ…Œ์ด๋„ˆ ์•„ํ‚คํ…์ฒ˜ ํ™•์ธ** + +- Docker Hub์— ํ‘ธ์‹œ๋œ ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€์˜ OS ๋ฐ ์•„ํ‚คํ…์ฒ˜ ํ™•์ธ ๊ฐ€๋Šฅ +- โœ… **์šด์˜ ์ฒด์ œ(OS):** `linux` +- โœ… **์ง€์› ์•„ํ‚คํ…์ฒ˜(Architecture):** `amd64`, `arm64` + +![Image](https://github.com/user-attachments/assets/6a269eff-0773-417c-a7e0-81ae6bed4963) + +- ๋‹ค์ค‘ ํ”Œ๋žซํผ ์ง€์›์œผ๋กœ ์šด์˜ ํšจ์œจ์„ฑ ์ฆ๊ฐ€ + +![Image](https://github.com/user-attachments/assets/2afffb7e-efdd-4d6e-ab4f-679e9f3a4837) + +```bash +(eks-user:default) [root@operator-host myweb]# docker manifest inspect $DOCKERNAME/myweb:multi | jq +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "schemaVersion": 2, + "mediaType": "application/vnd.oci.image.index.v1+json", + "manifests": [ + { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "size": 2010, + "digest": "sha256:b73a3a9ae85ac83166587bc398a9cda723b1296cd200e7bf61a2f8fdc03f22d6", + "platform": { + "architecture": "amd64", + "os": "linux" + } + }, + { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "size": 2010, + "digest": "sha256:5d18951aa10d7db756c39875c1360499c3a6643a097f9ea81ed8fdc7479c9b49", + "platform": { + "architecture": "arm64", + "os": "linux" + } + }, + { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "size": 566, + "digest": "sha256:be4b65e573ddbae236084237c32bb3e61598865f2378464082b008a268349bd7", + "platform": { + "architecture": "unknown", + "os": "unknown" + } + }, + { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "size": 566, + "digest": "sha256:eaa3e8e820eb8ee42165d8d674562cc887cdf8b0e9ba7d26d24e05bde231ce07", + "platform": { + "architecture": "unknown", + "os": "unknown" + } + } + ] +} +``` + +- ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€๊ฐ€ `amd64`, `arm64` ์•„ํ‚คํ…์ฒ˜๋ฅผ ๋ชจ๋‘ ์ง€์›ํ•˜๋Š”์ง€ ํ™•์ธ ๊ฐ€๋Šฅ + +### **13. ์ปจํ…Œ์ด๋„ˆ ์‚ญ์ œ** + +```bash +docker rm -f timeserver +``` + +--- + +## **๐Ÿข AWS ECR ํ”„๋ผ์ด๋น— ์ €์žฅ์†Œ ์‚ฌ์šฉํ•˜๊ธฐ** + +### **1. AWS ECR ๋กœ๊ทธ์ธ ๋ฐ ์ธ์ฆ ์„ค์ •** + +**(1) AWS ๊ณ„์ • ID๋ฅผ ํ™˜๊ฒฝ ๋ณ€์ˆ˜์— ์ €์žฅ** + +```bash +(eks-user:default) [root@operator-host myweb]# export ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text +``` + +**(2) ECR์— Docker ์ธ์ฆ ์ •๋ณด ์„ค์ •** + +```bash +(eks-user:default) [root@operator-host myweb]# aws ecr get-login-password \ +> --region ap-northeast-2 | docker login \ +> --username AWS \ +> --password-stdin ${ACCOUNT_ID}.dkr.ecr.ap-northeast-2.amazonaws.com +``` + +โœ… **์ถœ๋ ฅ** + +```bash +WARNING! Your password will be stored unencrypted in /root/.docker/config.json. +Configure a credential helper to remove this warning. See +https://docs.docker.com/engine/reference/commandline/login/#credentials-store + +Login Succeeded +``` + +### **2. ECR ์ธ์ฆ ์ •๋ณด ํ™•์ธ** + +```bash +(eks-user:default) [root@operator-host myweb]# cat /root/.docker/config.json | jq +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "auths": { + "378102432899.dkr.ecr.ap-northeast-2.amazonaws.com": { + "auth": "xxxxxxxxxxxxxxxxxx" + }, + "https://index.docker.io/v1/": { + "auth": "xxxxxxxxxxxxxxxxxx" + } + } +} +``` + +- `auths` ํ•ญ๋ชฉ์— ECR ์ €์žฅ์†Œ URL์ด ํฌํ•จ๋˜์–ด ์žˆ๋Š”์ง€ ํ™•์ธ + +### **3. ECR ํ”„๋ผ์ด๋น— ์ €์žฅ์†Œ ์ƒ์„ฑ** + +- ์ƒˆ๋กœ์šด ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ์ €์žฅ์†Œ๋ฅผ AWS ECR์—์„œ ์ƒ์„ฑ + +```bash +(eks-user:default) [root@operator-host myweb]# aws ecr create-repository --repository-name myweb +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "repository": { + "repositoryArn": "arn:aws:ecr:ap-northeast-2:378102432899:repository/myweb", + "registryId": "378102432899", + "repositoryName": "myweb", + "repositoryUri": "378102432899.dkr.ecr.ap-northeast-2.amazonaws.com/myweb", + "createdAt": "2025-02-19T01:17:53.137000+09:00", + "imageTagMutability": "MUTABLE", + "imageScanningConfiguration": { + "scanOnPush": false + }, + "encryptionConfiguration": { + "encryptionType": "AES256" + } + } +} +``` + +### **4. ๋‹ค์ค‘ ์•„ํ‚คํ…์ฒ˜ ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ ๋ฐ ECR ํ‘ธ์‹œ** + +- `linux/amd64`, `linux/arm64` ์•„ํ‚คํ…์ฒ˜ ์ง€์› ์ปจํ…Œ์ด๋„ˆ ๋นŒ๋“œ +- ๋นŒ๋“œ๋œ ์ด๋ฏธ์ง€๋ฅผ AWS ECR๋กœ ๋ฐ”๋กœ ํ‘ธ์‹œ + +```bash +(eks-user:default) [root@operator-host myweb]# docker buildx build --platform linux/amd64,linux/arm64 --push --tag ${ACCOUNT_ID}.dkr.ecr.ap-northeast-2.amazonaws.com/myweb:multi . +``` + +โœ… **์ถœ๋ ฅ** + +```bash +[+] Building 13.3s (14/14) FINISHED docker-container:mybuilder + => [internal] load build definition from Dockerfile 0.0s + => => transferring dockerfile: 125B 0.0s + => [linux/arm64 internal] load metadata for docker.io/library/python:3.12 1.4s + => [linux/amd64 internal] load metadata for docker.io/library/python:3.12 1.4s + => [auth] library/python:pull token for registry-1.docker.io 0.0s + => [internal] load .dockerignore 0.0s + => => transferring context: 2B 0.0s + => [linux/amd64 1/3] FROM docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 0.0s + => => resolve docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 0.0s + => [linux/arm64 1/3] FROM docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 0.0s + => => resolve docker.io/library/python:3.12@sha256:f61c61fb2a8967599fb0874746c93530c3d2a4583478528eda06584abc736ea0 0.0s + => [internal] load build context 0.0s + => => transferring context: 60B 0.0s + => CACHED [linux/arm64 2/3] COPY . /app 0.0s + => CACHED [linux/arm64 3/3] WORKDIR /app 0.0s + => CACHED [linux/amd64 2/3] COPY . /app 0.0s + => CACHED [linux/amd64 3/3] WORKDIR /app 0.0s + => exporting to image 11.8s + => => exporting layers 0.0s + => => exporting manifest sha256:b73a3a9ae85ac83166587bc398a9cda723b1296cd200e7bf61a2f8fdc03f22d6 0.0s + => => exporting config sha256:2e766855d2c0dc9c8a005866ec139d251b4f3cabd38586fff3fa9df874b1b14f 0.0s + => => exporting attestation manifest sha256:75d8f32e66dee7b5268c1882a93f0107bf972a16c7972903f0fefbde843b0e08 0.0s + => => exporting manifest sha256:5d18951aa10d7db756c39875c1360499c3a6643a097f9ea81ed8fdc7479c9b49 0.0s + => => exporting config sha256:c319f11eb4c8a52e492b75bd068715a6703f6d0afcf6d6240dbed9c0ad1011e8 0.0s + => => exporting attestation manifest sha256:388d86fbeda42c224fd181d385228fa7d7ff1d0ff2c3379c5b7f6f21e4e2f8f2 0.0s + => => exporting manifest list sha256:2bfb7328872d6ac549cfdc136c70bca7f133b7d2da51da1a3f456ff256d774ba 0.0s + => => pushing layers 10.4s + => => pushing manifest for 378102432899.dkr.ecr.ap-northeast-2.amazonaws.com/myweb:multi@sha256:2bfb7328872d6ac549cfdc136c70bca7f133b7d2da 1.4s + => [auth] sharing credentials for 378102432899.dkr.ecr.ap-northeast-2.amazonaws.com 0.0s + + 2 warnings found (use --debug to expand): + - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 2) + - JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals (line 5) +``` + +- `multi-architecture` ์ง€์› ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€๊ฐ€ ECR์— ํ‘ธ์‹œ๋จ + +![Image](https://github.com/user-attachments/assets/472257ad-a6cc-4796-96e3-e46307b1abc2) + +### **5. AWS ECR ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰** + +- ECR์—์„œ ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€๋ฅผ ํ’€ํ•˜์—ฌ ์‹คํ–‰ +- ์œˆ๋„์šฐPC(amd64), macOS(arm64)์—์„œ ๋™์ผํ•œ ์ด๋ฏธ์ง€ ์‚ฌ์šฉ ๊ฐ€๋Šฅ + +```bash +(eks-user:default) [root@operator-host myweb]# docker run -d -p 8080:80 --name=timeserver ${ACCOUNT_ID}.dkr.ecr.ap-northeast-2.amazonaws.com/myweb:multi +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Unable to find image '378102432899.dkr.ecr.ap-northeast-2.amazonaws.com/myweb:multi' locally +multi: Pulling from myweb +a492eee5e559: Already exists +32b550be6cb6: Already exists +35af2a7690f2: Already exists +7576b00d9bb1: Already exists +07612085660d: Already exists +60fd44efca0f: Already exists +4a12975a6131: Already exists +0a18c01708c3: Pull complete +4f4fb700ef54: Pull complete +Digest: sha256:2bfb7328872d6ac549cfdc136c70bca7f133b7d2da51da1a3f456ff256d774ba +Status: Downloaded newer image for 378102432899.dkr.ecr.ap-northeast-2.amazonaws.com/myweb:multi +c981958a36336d01c5ee98d23a9953b03261bbeceeab19e9c4b910598a9bad2e +``` + +- ์ปจํ…Œ์ด๋„ˆ ์‹คํ–‰ ์ƒํƒœ ํ™•์ธ + +```bash +(eks-user:default) [root@operator-host myweb]# curl http://localhost:8080 +``` + +โœ… **์ถœ๋ ฅ** + +```bash +The time is 4:21:54 PM, VERSION 0.0.1 +Server hostname: c981958a3633 +``` + +### **6. ECR ์ปจํ…Œ์ด๋„ˆ ๋ฐ ์ €์žฅ์†Œ ์ •๋ฆฌ** + +**(1) ECR ์ €์žฅ์†Œ ์‚ญ์ œ** + +![Image](https://github.com/user-attachments/assets/e3f8af1c-841e-422d-af9e-f323b6decd23) + +**(2) ์ปจํ…Œ์ด๋„ˆ ์‚ญ์ œ** + +```bash +(eks-user:default) [root@operator-host myweb]# docker rm -f timeserver +# ๊ฒฐ๊ณผ +timeserver +``` + +--- + +## โšก AWS Graviton (ARM) ๋…ธ๋“œ๊ทธ๋ฃน ๋ฐ ๋ฐฐํฌ ์‹ค์Šต + +### **1. ๊ธฐ์กด ๋…ธ๋“œ ์•„ํ‚คํ…์ฒ˜ ํ™•์ธ** + +```bash +kubectl get nodes -L kubernetes.io/arch +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION ARCH +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 13h v1.31.5-eks-5d632ec amd64 +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 13h v1.31.5-eks-5d632ec amd64 +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 13h v1.31.5-eks-5d632ec amd64 +``` + +- ํ˜„์žฌ EKS ํด๋Ÿฌ์Šคํ„ฐ์˜ ๋ชจ๋“  ๋…ธ๋“œ๋Š” `amd64` ์•„ํ‚คํ…์ฒ˜ + +### **2. Graviton(ARM) ๊ธฐ๋ฐ˜ ๋…ธ๋“œ๊ทธ๋ฃน ์ƒ์„ฑ** + +**(1) YAML ํŒŒ์ผ ์ƒ์„ฑ (`myng3.yaml`)** + +```bash +eksctl create nodegroup -c $CLUSTER_NAME -r ap-northeast-2 --subnet-ids "$PubSubnet1","$PubSubnet2","$PubSubnet3" \ + -n ng3 -t t4g.medium -N 1 -m 1 -M 1 --node-volume-size=30 --node-labels family=graviton --dry-run > myng3.yaml +``` + +**(2) ๋…ธ๋“œ๊ทธ๋ฃน ์ƒ์„ฑ** + +```bash +eksctl create nodegroup -f myng3.yaml + +# ๊ฒฐ๊ณผ +2025-02-19 01:32:09 [โ„น] nodegroup "ng3" will use "" [AmazonLinux2/1.31] +2025-02-19 01:32:10 [โ„น] 1 existing nodegroup(s) (ng1) will be excluded +2025-02-19 01:32:10 [โ„น] 1 nodegroup (ng3) was included (based on the include/exclude rules) +2025-02-19 01:32:10 [โ„น] will create a CloudFormation stack for each of 1 managed nodegroups in cluster "myeks" +2025-02-19 01:32:11 [โ„น] +2 sequential tasks: { fix cluster compatibility, 1 task: { 1 task: { create managed nodegroup "ng3" } } +} +2025-02-19 01:32:11 [โ„น] checking cluster stack for missing resources +2025-02-19 01:32:11 [โ„น] cluster stack has all required resources +2025-02-19 01:32:12 [โ„น] building managed nodegroup stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 01:32:12 [โ„น] deploying stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 01:32:12 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 01:32:42 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 01:33:17 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 01:33:57 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 01:34:38 [โ„น] waiting for CloudFormation stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 01:34:38 [โ„น] no tasks +2025-02-19 01:34:38 [โœ”] created 0 nodegroup(s) in cluster "myeks" +2025-02-19 01:34:38 [โ„น] nodegroup "ng3" has 1 node(s) +2025-02-19 01:34:38 [โ„น] node "ip-192-168-3-99.ap-northeast-2.compute.internal" is ready +2025-02-19 01:34:38 [โ„น] waiting for at least 1 node(s) to become ready in "ng3" +2025-02-19 01:34:38 [โ„น] nodegroup "ng3" has 1 node(s) +2025-02-19 01:34:38 [โ„น] node "ip-192-168-3-99.ap-northeast-2.compute.internal" is ready +2025-02-19 01:34:38 [โœ”] created 1 managed nodegroup(s) in cluster "myeks" +2025-02-19 01:34:38 [โ„น] checking security group configuration for all nodegroups +2025-02-19 01:34:38 [โ„น] all nodegroups have up-to-date cloudformation templates +``` + +### **3. ๋ฐฐํฌ๋œ ๋…ธ๋“œ ํ™•์ธ** + +```bash +kubectl get nodes --label-columns eks.amazonaws.com/nodegroup,kubernetes.io/arch,eks.amazonaws.com/capacityType +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION NODEGROUP ARCH CAPACITYTYPE +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 13h v1.31.5-eks-5d632ec ng1 amd64 ON_DEMAND +ip-192-168-1-92.ap-northeast-2.compute.internal Ready 7m41s v1.31.5-eks-5d632ec managed-spot amd64 SPOT +ip-192-168-2-145.ap-northeast-2.compute.internal Ready 99s v1.31.4-eks-0f56d01 ng-bottlerocket amd64 ON_DEMAND +ip-192-168-2-164.ap-northeast-2.compute.internal Ready 7m40s v1.31.5-eks-5d632ec managed-spot amd64 SPOT +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 13h v1.31.5-eks-5d632ec ng1 amd64 ON_DEMAND +ip-192-168-3-73.ap-northeast-2.compute.internal Ready 73s v1.31.4-eks-0f56d01 ng-bottlerocket-ssh amd64 ON_DEMAND +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 13h v1.31.5-eks-5d632ec ng1 amd64 ON_DEMAND +ip-192-168-3-99.ap-northeast-2.compute.internal Ready 12m v1.31.5-eks-5d632ec ng3 arm64 ON_DEMAND +``` + +- `ng3` ๋…ธ๋“œ๊ฐ€ `arm64`๋กœ ๋ฐฐํฌ๋จ +![Image](https://github.com/user-attachments/assets/4d147f6c-ef29-4d0d-a6fc-e6589ea63042) + +### **4. ํ˜„์žฌ Taints ์„ค์ • ํ™•์ธ** + +```bash +aws eks describe-nodegroup --cluster-name $CLUSTER_NAME --nodegroup-name ng3 | jq .nodegroup.taints +``` + +โœ… **์ถœ๋ ฅ** + +```bash +null +``` + +![Image](https://github.com/user-attachments/assets/4b6533a0-1ecc-4a95-a136-70a47f64d10a) + +### **5. Graviton ๋…ธ๋“œ์— Taints ์„ค์ •** + +- `ng3` ๋…ธ๋“œ์— `frontend=true:NoExecute` Taint ์ ์šฉ +- ํŠน์ • Pod๋งŒ ํ•ด๋‹น ๋…ธ๋“œ์— ์Šค์ผ€์ค„๋ง ๊ฐ€๋Šฅ + +```bash +aws eks update-nodegroup-config --cluster-name $CLUSTER_NAME --nodegroup-name ng3 --taints "addOrUpdateTaints=[{key=frontend, value=true, effect=NO_EXECUTE}]" +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "update": { + "id": "eabe8298-9d6b-3ef6-965f-00120771faa6", + "status": "InProgress", + "type": "ConfigUpdate", + "params": [ + { + "type": "TaintsToAdd", + "value": "[{\"effect\":\"NO_EXECUTE\",\"value\":\"true\",\"key\":\"frontend\"}]" + }, + { + "type": "TaintsToRemove", + "value": "[]" + } + ], + "createdAt": "2025-02-19T01:52:40.946000+09:00", + "errors": [] + } +} +``` + +- **Taints ์„ค์ • ํ™•์ธ** + +```bash +kubectl describe nodes --selector family=graviton | grep Taints +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Taints: frontend=true:NoExecute +``` + +![Image](https://github.com/user-attachments/assets/30054f48-8332-495f-ab1b-a8335bfd96fc) + +### **6. Graviton ๋…ธ๋“œ์— Tolerations์„ ํฌํ•จํ•œ Pod ๋ฐฐํฌ** + +- `tolerations`์„ ์„ค์ •ํ•˜์—ฌ Taint๊ฐ€ ์žˆ์–ด๋„ Pod๊ฐ€ ๋ฐฐํฌ๋  ์ˆ˜ ์žˆ๋„๋ก ์„ค์ • + +```bash +cat <> /home/pod-out.txt; cd /home; sync; sync; sleep 10; done" + tolerations: + - effect: NoExecute + key: frontend + operator: Exists + nodeSelector: + family: graviton +EOF + +# ๊ฒฐ๊ณผ +pod/busybox created +``` + +![Image](https://github.com/user-attachments/assets/70663d42-56ff-431e-83bf-06fee29fc0e7) + +### **7. Pod ์ •๋ณด ํ™•์ธ** + +**(1) Pod ์„ธ๋ถ€ ์ •๋ณด ํ™•์ธ** + +```bash +kubectl describe pod busybox +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Name: busybox +Namespace: default +Priority: 0 +Service Account: default +Node: ip-192-168-3-99.ap-northeast-2.compute.internal/192.168.3.99 +Start Time: Wed, 19 Feb 2025 01:59:49 +0900 +Labels: +Annotations: +Status: Running +IP: 192.168.3.40 +IPs: + IP: 192.168.3.40 +Containers: + busybox: + Container ID: containerd://620e686393b9dee8876708ba15aac6f8523fd6fa9dd0507183dd230b9277bca9 + Image: busybox + Image ID: docker.io/library/busybox@sha256:a5d0ce49aa801d475da48f8cb163c354ab95cab073cd3c138bd458fc8257fbf1 + Port: + Host Port: + Command: + /bin/sh + -c + while true; do date >> /home/pod-out.txt; cd /home; sync; sync; sleep 10; done + State: Running + Started: Wed, 19 Feb 2025 01:59:54 +0900 + Ready: True + Restart Count: 0 + Environment: + Mounts: + /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5sq9z (ro) +Conditions: + Type Status + PodReadyToStartContainers True + Initialized True + Ready True + ContainersReady True + PodScheduled True +Volumes: + kube-api-access-5sq9z: + Type: Projected (a volume that contains injected data from multiple sources) + TokenExpirationSeconds: 3607 + ConfigMapName: kube-root-ca.crt + ConfigMapOptional: + DownwardAPI: true +QoS Class: BestEffort +Node-Selectors: family=graviton +Tolerations: frontend:NoExecute op=Exists + node.kubernetes.io/not-ready:NoExecute op=Exists for 300s + node.kubernetes.io/unreachable:NoExecute op=Exists for 300s +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Scheduled 2m27s default-scheduler Successfully assigned default/busybox to ip-192-168-3-99.ap-northeast-2.compute.internal + Normal Pulling 2m26s kubelet Pulling image "busybox" + Normal Pulled 2m22s kubelet Successfully pulled image "busybox" in 4.274s (4.274s including waiting). Image size: 1855561 bytes. + Normal Created 2m22s kubelet Created container busybox + Normal Started 2m22s kubelet Started container busybox +``` + +**(2) Pod ๋‚ด๋ถ€์—์„œ ์•„ํ‚คํ…์ฒ˜ ํ™•์ธ** + +```bash +kubectl exec -it busybox -- arch +# ๊ฒฐ๊ณผ +aarch64 +``` + +**(3) Pod ๋‚ด๋ถ€์—์„œ ๋กœ๊ทธ ํ™•์ธ** + +```bash +kubectl exec -it busybox -- tail -f /home/pod-out.txt +``` + +โœ… **์ถœ๋ ฅ** + +```bash +Tue Feb 18 17:01:24 UTC 2025 +Tue Feb 18 17:01:34 UTC 2025 +Tue Feb 18 17:01:44 UTC 2025 +Tue Feb 18 17:01:54 UTC 2025 +Tue Feb 18 17:02:04 UTC 2025 +Tue Feb 18 17:02:14 UTC 2025 +Tue Feb 18 17:02:24 UTC 2025 +Tue Feb 18 17:02:34 UTC 2025 +Tue Feb 18 17:02:44 UTC 2025 +... +``` + +- `busybox` ์ปจํ…Œ์ด๋„ˆ๊ฐ€ ์ •์ƒ์ ์œผ๋กœ ์‹คํ–‰ ์ค‘ + +### **8. Pod ์‚ญ์ œ** + +```bash +kubectl delete pod busybox +# ๊ฒฐ๊ณผ +pod "busybox" deleted +``` + +### **9. ๋‹ค์ค‘ ์•„ํ‚คํ…์ฒ˜ ์ปจํ…Œ์ด๋„ˆ ๋ฐฐํฌ (x86_64 & ARM64)** + +- ์šด์˜์„œ๋ฒ„ EC2์—์„œ ๋นŒ๋“œํ•œ `myweb` ์ปจํ…Œ์ด๋„ˆ๋ฅผ ๋‹ค์ค‘ ์•„ํ‚คํ…์ฒ˜ ์ง€์›ํ•˜๋Š” Pod๋กœ ๋ฐฐํฌ +- x86_64(AMD) ๋ฐ ARM64(Graviton) ๋…ธ๋“œ์—์„œ ๊ฐ๊ฐ ์‹คํ–‰ + +```bash +cat < +myweb-arm 1/1 Running 0 77s 192.168.3.40 ip-192-168-3-99.ap-northeast-2.compute.internal +``` + +### **11. ๊ฐ ์ปจํ…Œ์ด๋„ˆ์˜ ์•„ํ‚คํ…์ฒ˜ ํ™•์ธ** + +```bash +kubectl exec -it myweb-arm -- arch +aarch64 + +kubectl exec -it myweb-amd -- arch +x86_64 +``` + +### **12. ์›น ์„œ๋น„์Šค ์ •์ƒ ๋™์ž‘ ํ™•์ธ** + +```bash +kubectl exec -it myweb-arm -- curl localhost +# ๊ฒฐ๊ณผ +The time is 5:09:21 PM, VERSION 0.0.1 +Server hostname: myweb-arm + +kubectl exec -it myweb-amd -- curl localhost +# ๊ฒฐ๊ณผ +The time is 5:09:30 PM, VERSION 0.0.1 +Server hostname: myweb-amd +``` + +### **13. Pod ์‚ญ์ œ ๋ฐ ng3 ๋…ธ๋“œ ๊ทธ๋ฃน ์‚ญ์ œ** + +```bash +kubectl delete pod myweb-arm myweb-amd +# ๊ฒฐ๊ณผ +pod "myweb-arm" deleted +pod "myweb-amd" deleted +``` + +```bash +eksctl delete nodegroup -c $CLUSTER_NAME -n ng3 +# ๊ฒฐ๊ณผ +2025-02-19 02:12:21 [โ„น] 1 nodegroup (ng3) was included (based on the include/exclude rules) +2025-02-19 02:12:21 [โ„น] will drain 1 nodegroup(s) in cluster "myeks" +2025-02-19 02:12:21 [โ„น] starting parallel draining, max in-flight of 1 +2025-02-19 02:12:21 [โ„น] cordon node "ip-192-168-3-99.ap-northeast-2.compute.internal" +2025-02-19 02:12:22 [โœ”] drained all nodes: [ip-192-168-3-99.ap-northeast-2.compute.internal] +2025-02-19 02:12:22 [โ„น] will delete 1 nodegroups from cluster "myeks" +2025-02-19 02:12:22 [โ„น] 1 task: { 1 task: { delete nodegroup "ng3" [async] } } +2025-02-19 02:12:22 [โ„น] will delete stack "eksctl-myeks-nodegroup-ng3" +2025-02-19 02:12:22 [โœ”] deleted 1 nodegroup(s) from cluster "myeks" +``` + +--- + +## **๐Ÿ’ฐ ์ŠคํŒŸ ๋…ธ๋“œ๊ทธ๋ฃน ๋ฐ ๋ฐฐํฌ ์‹ค์Šต** + +### **1. ๋…ธ๋“œ ์—ญํ•  ์กฐํšŒ** + +```bash +NODEROLEARN=$(aws iam list-roles --query "Roles[?contains(RoleName, 'nodegroup-ng1')].Arn" --output text) +echo $NODEROLEARN +``` + +โœ… **์ถœ๋ ฅ** + +```bash +arn:aws:iam::378102432899:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-rGyQG9rZlOwl +``` + +### **2. ์ŠคํŒŸ ๋…ธ๋“œ๊ทธ๋ฃน ์ƒ์„ฑ** + +```bash +aws eks create-nodegroup \ + --cluster-name $CLUSTER_NAME \ + --nodegroup-name managed-spot \ + --subnets $PubSubnet1 $PubSubnet2 $PubSubnet3 \ + --node-role $NODEROLEARN \ + --instance-types c5.large c5d.large c5a.large \ + --capacity-type SPOT \ + --scaling-config minSize=2,maxSize=3,desiredSize=2 \ + --disk-size 20 +``` + +โœ… **์ถœ๋ ฅ** + +```bash +{ + "nodegroup": { + "nodegroupName": "managed-spot", + "nodegroupArn": "arn:aws:eks:ap-northeast-2:378102432899:nodegroup/myeks/managed-spot/56ca8cf5-e169-efa6-7b36-7922fce5434f", + "clusterName": "myeks", + "version": "1.31", + "releaseVersion": "1.31.5-20250212", + "createdAt": "2025-02-19T01:37:18.476000+09:00", + "modifiedAt": "2025-02-19T01:37:18.476000+09:00", + "status": "CREATING", + "capacityType": "SPOT", + "scalingConfig": { + "minSize": 2, + "maxSize": 3, + "desiredSize": 2 + }, + "instanceTypes": [ + "c5.large", + "c5d.large", + "c5a.large" + ], + "subnets": [ + "subnet-0fed28a1b3e108719", + "subnet-0e4fb63cb543698fe", + "subnet-0861bd68771150000" + ], + "amiType": "AL2023_x86_64_STANDARD", + "nodeRole": "arn:aws:iam::378102432899:role/eksctl-myeks-nodegroup-ng1-NodeInstanceRole-rGyQG9rZlOwl", + "diskSize": 20, + "health": { + "issues": [] + }, + "updateConfig": { + "maxUnavailable": 1 + }, + "tags": {} + } +} +``` + +- `managed-spot` ๋…ธ๋“œ๊ทธ๋ฃน์ด **์ŠคํŒŸ ์ธ์Šคํ„ด์Šค๋กœ ์ƒ์„ฑ๋จ** + +### **3. ์ ์ ˆํ•œ ์ŠคํŒŸ ์ธ์Šคํ„ด์Šค ์„ ํƒ** + +- `ec2-instance-selector` ์„ค์น˜ +- ์š”๊ตฌํ•˜๋Š” **CPU, ๋ฉ”๋ชจ๋ฆฌ, ์•„ํ‚คํ…์ฒ˜**์— ๋งž๋Š” ์ธ์Šคํ„ด์Šค๋ฅผ ์„ ํƒ ๊ฐ€๋Šฅ + +```bash +curl -Lo ec2-instance-selector https://github.com/aws/amazon-ec2-instance-selector/releases/download/v2.4.1/ec2-instance-selector-`uname | tr '[:upper:]' '[:lower:]'`-amd64 && chmod +x ec2-instance-selector +sudo mv ec2-instance-selector /usr/local/bin/ +ec2-instance-selector --version +``` + +โœ… **์ถœ๋ ฅ** + +```bash +v2.4.1 +``` + +- `ec2-instance-selector` ์ •์ƒ ์„ค์น˜ ์™„๋ฃŒ + +### **4. ์ธ์Šคํ„ด์Šค ์ถ”์ฒœ ์กฐํšŒ (vCPU 2, RAM 4GB, GPU ์—†์Œ, x86_64 ์•„ํ‚คํ…์ฒ˜)** + +```bash +ec2-instance-selector --vcpus 2 --memory 4 --gpus 0 --current-generation -a x86_64 --deny-list 't.*' --output table-wide +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NOTE: Could not retrieve 30 day avg hourly spot price for instance type p2.16xlarge +Instance Type VCPUs Mem (GiB) Hypervisor Current Gen Hibernation Support CPU Arch Network Performance ENIs GPUs GPU Mem (GiB) GPU Info On-Demand Price/Hr Spot Price/Hr (30d avg) +------------- ----- --------- ---------- ----------- ------------------- -------- ------------------- ---- ---- ------------- -------- ------------------ ----------------------- +c5.large 2 4 nitro true true x86_64 Up to 10 Gigabit 3 0 0 none $0.096 $0.02993 +c5a.large 2 4 nitro true false x86_64 Up to 10 Gigabit 3 0 0 none $0.086 $0.0397 +c5d.large 2 4 nitro true true x86_64 Up to 10 Gigabit 3 0 0 none $0.11 $0.03067 +c6i.large 2 4 nitro true true x86_64 Up to 12.5 Gigabit 3 0 0 none $0.096 $0.03346 +c6id.large 2 4 nitro true true x86_64 Up to 12.5 Gigabit 3 0 0 none $0.1155 $0.03071 +c6in.large 2 4 nitro true true x86_64 Up to 25 Gigabit 3 0 0 none $0.1281 $0.05098 +c7i-flex.large 2 4 nitro true true x86_64 Up to 12.5 Gigabit 3 0 0 none $0.09576 $0.02884 +c7i.large 2 4 nitro true true x86_64 Up to 12.5 Gigabit 3 0 0 none $0.1008 $0.02964 +``` + +- `c5.large` ๊ณ„์—ด ์ธ์Šคํ„ด์Šค๊ฐ€ **์ €๋ ดํ•œ ์ŠคํŒŸ ๊ฐ€๊ฒฉ**์„ ์ œ๊ณตํ•จ + +### **5. ์ŠคํŒŸ ๋ฐ ์˜จ๋””๋งจ๋“œ ๋…ธ๋“œ ํ™•์ธ** + +**(1) ์˜จ๋””๋งจ๋“œ ๋…ธ๋“œ๋งŒ ์กฐํšŒ** + +```bash +kubectl get nodes -l eks.amazonaws.com/capacityType=ON_DEMAND +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec +ip-192-168-2-145.ap-northeast-2.compute.internal Ready 35m v1.31.4-eks-0f56d01 +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec +ip-192-168-3-73.ap-northeast-2.compute.internal Ready 34m v1.31.4-eks-0f56d01 +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec +``` + +**(2) ์ „์ฒด ๋…ธ๋“œ (์˜จ๋””๋งจ๋“œ vs ์ŠคํŒŸ) ์กฐํšŒ** + +```bash +kubectl get nodes -L eks.amazonaws.com/capacityType +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION CAPACITYTYPE +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec ON_DEMAND +ip-192-168-1-92.ap-northeast-2.compute.internal Ready 41m v1.31.5-eks-5d632ec SPOT +ip-192-168-2-145.ap-northeast-2.compute.internal Ready 35m v1.31.4-eks-0f56d01 ON_DEMAND +ip-192-168-2-164.ap-northeast-2.compute.internal Ready 41m v1.31.5-eks-5d632ec SPOT +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec ON_DEMAND +ip-192-168-3-73.ap-northeast-2.compute.internal Ready 35m v1.31.4-eks-0f56d01 ON_DEMAND +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec ON_DEMAND +``` + +![Image](https://github.com/user-attachments/assets/789ecc35-6c8a-4ad7-a2a5-dfc0b7dc436e) + +### **6. ์ŠคํŒŸ ๋…ธ๋“œ์— Pod ๋ฐฐํฌ** + +์ŠคํŒŸ ๋…ธ๋“œ์—์„œ๋งŒ ์‹คํ–‰๋˜๋Š” `busybox` Pod ๋ฐฐํฌ + +```bash +cat <> /home/pod-out.txt; cd /home; sync; sync; sleep 10; done" + nodeSelector: + eks.amazonaws.com/capacityType: SPOT +EOF + +# ๊ฒฐ๊ณผ +pod/busybox created +``` + +### **7. Pod ๋ฐฐํฌ ๋…ธ๋“œ ํ™•์ธ** + +```bash +kubectl get pod -owide +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES +busybox 1/1 Running 0 50s 192.168.2.75 ip-192-168-2-164.ap-northeast-2.compute.internal +``` + +```bash +k get nodes -L eks.amazonaws.com/capacityType +``` + +โœ… **์ถœ๋ ฅ** + +```bash +NAME STATUS ROLES AGE VERSION CAPACITYTYPE +ip-192-168-1-207.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec ON_DEMAND +ip-192-168-1-92.ap-northeast-2.compute.internal Ready 47m v1.31.5-eks-5d632ec SPOT +ip-192-168-2-145.ap-northeast-2.compute.internal Ready 41m v1.31.4-eks-0f56d01 ON_DEMAND +ip-192-168-2-164.ap-northeast-2.compute.internal Ready 47m v1.31.5-eks-5d632ec SPOT +ip-192-168-2-84.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec ON_DEMAND +ip-192-168-3-73.ap-northeast-2.compute.internal Ready 41m v1.31.4-eks-0f56d01 ON_DEMAND +ip-192-168-3-80.ap-northeast-2.compute.internal Ready 14h v1.31.5-eks-5d632ec ON_DEMAND +``` + +- `busybox` Pod๊ฐ€ **์ŠคํŒŸ ์ธ์Šคํ„ด์Šค์—์„œ ์‹คํ–‰๋จ** + +### **8. ์ŠคํŒŸ ์ธ์Šคํ„ด์Šค ๋ฆฌ์†Œ์Šค ์‚ญ์ œ** + +**(1) Pod ์‚ญ์ œ** + +```bash +kubectl delete pod busybox +# ๊ฒฐ๊ณผ +pod "busybox" deleted +``` + +**(2) ng3 ๋…ธ๋“œ ๊ทธ๋ฃน ์‚ญ์ œ** + +```bash +eksctl delete nodegroup -c $CLUSTER_NAME -n managed-spot +# ๊ฒฐ๊ณผ +2025-02-19 02:30:04 [โ„น] 1 nodegroup (managed-spot) was included (based on the include/exclude rules) +2025-02-19 02:30:04 [โ„น] will drain 1 nodegroup(s) in cluster "myeks" +2025-02-19 02:30:04 [โ„น] starting parallel draining, max in-flight of 1 +2025-02-19 02:30:04 [โ„น] cordon node "ip-192-168-1-92.ap-northeast-2.compute.internal" +2025-02-19 02:30:04 [โ„น] cordon node "ip-192-168-2-164.ap-northeast-2.compute.internal" +2025-02-19 02:30:05 [โœ”] drained all nodes: [ip-192-168-1-92.ap-northeast-2.compute.internal ip-192-168-2-164.ap-northeast-2.compute.internal] +2025-02-19 02:30:05 [โ„น] will delete 1 nodegroups from cluster "myeks" +2025-02-19 02:30:05 [โ„น] +2 parallel tasks: { delete unowned nodegroup managed-spot, no tasks +} +2025-02-19 02:30:05 [โœ”] deleted 1 nodegroup(s) from cluster "myeks" +```