> For the complete documentation index, see [llms.txt](https://kubernetes.youngwjung.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://kubernetes.youngwjung.com/advanced-topics/storage/undefined.md).

# 실습

### EBS

1. 현재 설정된 StorageClass 목록 확인

   ```
   kubectl get sc
   ```
2. ClusterConfig를 아래와 같이 수정

   ```
   apiVersion: eksctl.io/v1alpha5
   kind: ClusterConfig

   metadata:
     name: mycluster
     region: ap-northeast-2
     version: "1.29"

   availabilityZones: 
   - ap-northeast-2a 
   - ap-northeast-2b
   - ap-northeast-2c
   - ap-northeast-2d

   managedNodeGroups:
   - name: nodegroup
     instanceType: t3.small
     minSize: 2
     desiredCapacity: 2
     maxSize: 5
     volumeSize: 20
     iam:
       attachPolicyARNs:
       - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
       - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
       - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
       - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
   iam:
     withOIDC: true
   addons:
   - name: aws-ebs-csi-driver
     wellKnownPolicies:
       ebsCSIController: true
   ```
3. Amazon EBS CSI driver 설치&#x20;

   ```
   eksctl create addon -f mycluster.yaml
   ```
4. 설치 확인&#x20;

   ```
   kubectl get pod -n kube-system \
   -l app.kubernetes.io/name=aws-ebs-csi-driver
   ```
5. StorageClass 생성 - <https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md>

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: ebs
   provisioner: ebs.csi.aws.com
   volumeBindingMode: WaitForFirstConsumer
   parameters:
     type: gp3
     encrypted: "true"
   EOF
   ```
6. 생성된 StorageClass 확인&#x20;

   ```
   kubectl get sc
   ```
7. 기본 StorageClass 변경

   ```
   {
     kubectl annotate sc gp2 storageclass.kubernetes.io/is-default-class- --overwrite
     kubectl annotate sc ebs storageclass.kubernetes.io/is-default-class=true --overwrite
   }
   ```
8. StorageClass 확인&#x20;

   ```
   kubectl get sc
   ```
9. 데모 애플리케이션 배포

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: my-pvc
   spec:
     accessModes:
     - ReadWriteOnce
     resources:
        requests:
          storage: 1Gi
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     labels:
       app: nginx
     name: nginx
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: nginx
           ports:
           - containerPort: 80
           volumeMounts:
           - name: data
             mountPath: /mnt
         volumes:
         - name: data
           persistentVolumeClaim:
             claimName: my-pvc
   EOF
   ```
10. Pod 상태 확인

    ```
    kubectl get pod -l app=nginx
    ```
11. Pod에 발생한 이벤트 확인&#x20;

    ```
    kubectl describe pod -l app=nginx
    ```
12. PVC 상태 확인

    ```
    kubectl get pvc my-pvc
    ```
13. PVC에 발생한 이벤트 확인&#x20;

    ```
    kubectl describe pvc my-pvc
    ```
14. 생성된 PV 확인&#x20;

    ```
    kubectl get pv $(kubectl get pvc my-pvc -o=jsonpath='{.spec.volumeName}')
    ```
15. PV에 연동된 EBS 볼륨를 확인하고 볼륨 ID를 환경변수로 지정

    ```
    {
        export VOL_ID=$(kubectl get pv $(kubectl get pvc my-pvc \
        -o=jsonpath='{.spec.volumeName}') \
        -o=jsonpath='{.spec.csi.volumeHandle}')

        echo $VOL_ID
    }
    ```
16. 위에서 확인한 EBS 볼륨 상태 확인

    ```
    aws ec2 describe-volumes --volume-ids $VOL_ID --no-cli-pager
    ```
17. PV(EBS)가 마운트된 경로에 파일 생성

    ```
    kubectl exec -it deploy/nginx -- \
    bash -c "echo hello EBS > /mnt/message.txt"
    ```
18. 파일 내용 확인&#x20;

    ```
    kubectl exec -it deploy/nginx -- cat /mnt/message.txt
    ```
19. Pod 삭제&#x20;

    ```
    kubectl scale deploy nginx --replicas=0
    ```
20. Pod 재생성&#x20;

    ```
    kubectl scale deploy nginx --replicas=1
    ```
21. Pod 상태 확인&#x20;

    ```
    kubectl get pod
    ```
22. PV에 생성했던 파일이 존재하는지 확인 &#x20;

    ```
    kubectl exec -it deploy/nginx -- cat /mnt/message.txt
    ```
23. 리소스 삭제

    ```
    {
        kubectl delete deploy nginx
        kubectl delete pvc my-pvc
    }
    ```
24. PV가 삭제되었는지 확인 - *삭제 될때까지 시간이 걸릴수도 있음*

    ```
    kubectl get pv
    ```
25. PV에 연동된 EBS 볼륨이 삭제되었는지 확인

    ```
    aws ec2 describe-volumes --volume-ids $VOL_ID
    ```

### EFS

1. ClusterConfig를 아래와 같이 수정

   ```
   apiVersion: eksctl.io/v1alpha5
   kind: ClusterConfig

   metadata:
     name: mycluster
     region: ap-northeast-2
     version: "1.29"

   availabilityZones: 
   - ap-northeast-2a 
   - ap-northeast-2b
   - ap-northeast-2c
   - ap-northeast-2d

   managedNodeGroups:
   - name: nodegroup
     instanceType: t3.small
     minSize: 2
     desiredCapacity: 2
     maxSize: 5
     volumeSize: 20
     iam:
       attachPolicyARNs:
       - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
       - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
       - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
       - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
   iam:
     withOIDC: true
   addons:
   - name: aws-ebs-csi-driver
     wellKnownPolicies:
       ebsCSIController: true
   - name: aws-efs-csi-driver
     wellKnownPolicies:
       efsCSIController: true
   ```
2. Amazon EFS CSI driver 설치&#x20;

   ```
   eksctl create addon -f mycluster.yaml
   ```
3. 설치 확인&#x20;

   ```
   kubectl get pod -n kube-system \
   -l app.kubernetes.io/name=aws-efs-csi-driver
   ```
4. EFS 파일 시스템 생성

   ```
   aws efs create-file-system --tags Key=Name,Value=my-efs --no-cli-pager
   ```
5. 생성된 EFS 파일 시스템 확인&#x20;

   ```
   {
     export EFS_ID=$(aws efs describe-file-systems \
     --query 'FileSystems[? Tags[? (Key==`Name`) && Value==`my-efs`]].FileSystemId' \
     --output text)
     
     echo $EFS_ID
   }
   ```
6. EKS 클러스터가 생성된 VPC의 프라이빗 서브넷 목록 확인

   ```
   {
     export EKS_SUBNETS=$(aws ec2 describe-subnets \
     --filters "Name=tag:alpha.eksctl.io/cluster-name,Values=mycluster" \
     --filters "Name=tag-key,Values=kubernetes.io/role/internal-elb" \
     --query "Subnets[*].SubnetId" \
     --output text)

     echo $EKS_SUBNETS
   }
   ```
7. EFS 마운트 대상에 부여할 보안 그룹 생성

   ```
   {
     export EKS_VPC=$(aws ec2 describe-vpcs \
     --filters "Name=tag:alpha.eksctl.io/cluster-name,Values=mycluster" \
     --query "Vpcs[*].VpcId" \
     --output text)
     
     aws ec2 create-security-group \
     --description "my efs security group" \
     --group-name my-efs-sg \
     --vpc-id $EKS_VPC
   }
   ```
8. EKS 노드에서 EFS로 접근이 가능하도록 인바운드 규칙 추가 &#x20;

   ```
   {
     export EFS_SG=$(aws ec2 describe-security-groups \
     --filters "Name=vpc-id,Values=$EKS_VPC" \
     --filters "Name=group-name,Values=my-efs-sg" \
     --query "SecurityGroups[*].GroupId" \
     --output text)
     
     export EKS_SG=$(aws eks describe-cluster --name mycluster \
     --query 'cluster.resourcesVpcConfig.clusterSecurityGroupId' \
     --output text)
     
     aws ec2 authorize-security-group-ingress \
     --group-id $EFS_SG \
     --protocol tcp \
     --port 2049 \
     --source-group $EKS_SG \
     --no-cli-pager
   }
   ```
9. EFS 마운트 대상 생성&#x20;

   ```
   {
     for subnet in $EKS_SUBNETS
     do
       aws efs create-mount-target \
       --file-system-id $EFS_ID \
       --subnet-id $subnet \
       --security-groups $EFS_SG \
       --no-cli-pager
     done
   }
   ```
10. EFS 마운트 대상 상태 확인&#x20;

    ```
    aws efs describe-mount-targets --file-system-id $EFS_ID --no-cli-pager
    ```
11. StorageClass 생성&#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: efs
    provisioner: efs.csi.aws.com
    EOF
    ```
12. PV 생성 &#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: my-efs
    spec:
      storageClassName: efs
      capacity:
        storage: 1Gi
      volumeMode: Filesystem
      accessModes:
      - ReadWriteMany
      persistentVolumeReclaimPolicy: Retain
      csi:
        driver: efs.csi.aws.com
        volumeHandle: $EFS_ID
    EOF
    ```
13. PVC 생성&#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-efs
    spec:
      storageClassName: efs
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 1Gi
      volumeName: my-efs
    EOF
    ```
14. PVC 상태 확인

    ```
    kubectl get pvc my-efs
    ```
15. 데모 애플리케이션 배포

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: data
              mountPath: /mnt
          volumes:
          - name: data
            persistentVolumeClaim:
              claimName: my-efs
    EOF
    ```
16. Pod 생성 확인

    ```
    kubectl get pod -l app=nginx
    ```
17. 첫번째 Pod에서 PV(EFS)가 마운트된 경로에 파일 생성

    ```
    kubectl exec -it \
    $(kubectl get pod -l app=nginx -o=jsonpath='{.items[0].metadata.name}') \
    -- bash -c "echo hello EFS > /mnt/message.txt"
    ```
18. 두번째 Pod에서 첫번째 Pod에서 생성한 파일 확인

    ```
    kubectl exec -it \
    $(kubectl get pod -l app=nginx -o=jsonpath='{.items[1].metadata.name}') \
    -- cat /mnt/message.txt
    ```
19. 두번째 Pod에서 파일 수정

    ```
    kubectl exec -it \
    $(kubectl get pod -l app=nginx -o=jsonpath='{.items[1].metadata.name}') \
    -- bash -c "echo greeting from 2nd pod > /mnt/message.txt"
    ```
20. 첫번째 Pod에서 파일 확인

    ```
    kubectl exec -it \
    $(kubectl get pod -l app=nginx -o=jsonpath='{.items[0].metadata.name}') \
    -- cat /mnt/message.txt
    ```
21. 리소스 삭제

    ```
    {
      kubectl delete deploy nginx
      kubectl delete pvc my-efs
      kubectl delete pv my-efs
    }
    ```
22. StorageClass 생성 - <https://github.com/kubernetes-sigs/aws-efs-csi-driver?tab=readme-ov-file#storage-class-parameters-for-dynamic-provisioning>

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: efs-ap
    provisioner: efs.csi.aws.com
    parameters:
      provisioningMode: efs-ap
      fileSystemId: $EFS_ID
      directoryPerms: "700"
    EOF
    ```
23. PVC 생성

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-efs-ap
    spec:
      accessModes:
      - ReadWriteMany
      storageClassName: efs-ap
      resources:
        requests:
          storage: 5Gi
    EOF
    ```
24. PVC 생성 확인

    ```
    kubectl get pvc my-efs-ap
    ```
25. PV 생성 확인

    ```
    kubectl get pv $(kubectl get pvc my-efs-ap -o=jsonpath='{.spec.volumeName}')
    ```
26. 생성된 EFS Access Point 확인&#x20;

    ```
    aws efs describe-access-points --file-system-id $EFS_ID --no-cli-pager
    ```
27. PVC 삭제&#x20;

    ```
    kubectl delete pvc my-efs-ap
    ```
28. PV가 삭제되었는지 확인 - *삭제 될때까지 시간이 걸릴수도 있음*

    ```
    kubectl get pv
    ```
29. PV에 연동된 EFS Access Point가 삭제되었는지 확인

    ```
    aws efs describe-access-points --file-system-id $EFS_ID --no-cli-pager
    ```
30. 리소스 삭제&#x20;

    ```
    {
      export EFS_MOUNT_TARGETS=$(aws efs describe-mount-targets --file-system-id $EFS_ID \
      --query 'MountTargets[*].MountTargetId' \
      --output text)
      
      for target in $EFS_MOUNT_TARGETS
      do
        aws efs delete-mount-target --mount-target-id $target --no-cli-pager
      done
      aws efs delete-file-system --file-system-id $EFS_ID --no-cli-pager
      aws ec2 delete-security-group --group-id $EFS_SG
    }
    ```

### Secret

1. Secrets Store CSI Driver 설치&#x20;

   ```
   {
     helm repo add secrets-store-csi-driver \
     https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
     helm install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver \
     --namespace kube-system \
     --set syncSecret.enabled=true \
     --set enableSecretRotation=true
   }
   ```
2. AWS provider for the Secrets Store CSI Driver 설치

   ```
   {
     helm repo add secrets-store-csi-driver-provider-aws \
     https://aws.github.io/secrets-store-csi-driver-provider-aws
     helm install secrets-store-csi-driver-provider-aws \
     secrets-store-csi-driver-provider-aws/secrets-store-csi-driver-provider-aws \
     --namespace kube-system
   }
   ```
3. AWS Secrets Manager에 저장할 파일 생성

   ```
   cat <<EOF | tee application.properties
   a=b
   c=d
   e=f
   g=h
   EOF
   ```
4. JSON 파일 생성&#x20;

   ```
   {
     jq -n --argjson application.properties \
     $(jq -Rs '.' application.properties) '$ARGS.named' > secret-file.json
     
     cat secret-file.json
   }
   ```
5. Secret 생성&#x20;

   ```
   aws secretsmanager create-secret --name secret-file \
   --secret-string file://secret-file.json
   ```
6. 생성된 Secret 확인&#x20;

   ```
   aws secretsmanager get-secret-value --secret-id secret-file
   ```
7. SecretProviderClass 생성&#x20;

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: secrets-store.csi.x-k8s.io/v1
   kind: SecretProviderClass
   metadata:
     name: secret-file
   spec:
     provider: aws
     parameters:
       objects: |
         - objectName: "secret-file"
           objectType: "secretsmanager"
   EOF
   ```
8. 데모 애플리케이션 배포

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     labels:
       app: nginx
     name: nginx
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: nginx
           ports:
           - containerPort: 80
           volumeMounts:
           - name: secret
             mountPath: /mnt
         volumes:
         - name: secret
           csi:
             driver: secrets-store.csi.k8s.io
             readOnly: true
             volumeAttributes:
               secretProviderClass: secret-file
   EOF
   ```
9. Pod 생성 확인

   ```
   kubectl get pod -l app=nginx
   ```
10. Pod에 발생한 이벤트 확인&#x20;

    ```
    kubectl describe pod -l app=nginx
    ```
11. ServiceAccount 생성&#x20;

    ```
    {
      export CLUSTER_NAME=$(kubectl get node \
      -o=jsonpath='{.items[0].metadata.labels.alpha\.eksctl\.io\/cluster-name}')
        
      eksctl create iamserviceaccount \
      --cluster $CLUSTER_NAME \
      --namespace=default \
      --name=nginx \
      --attach-policy-arn=arn:aws:iam::aws:policy/SecretsManagerReadWrite \
      --override-existing-serviceaccounts \
      --approve
    }
    ```
12. Pod에 위에서 생성한 ServiceAccount 부여&#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          serviceAccountName: nginx
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: secret
              mountPath: /mnt
          volumes:
          - name: secret
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: secret-file
    EOF
    ```
13. Pod 생성 확인

    ```
    kubectl get pod -l app=nginx
    ```
14. Pod에 발생한 이벤트 확인&#x20;

    ```
    kubectl describe pod -l app=nginx
    ```
15. Pod에 마운트된 파일 확인&#x20;

    ```
    kubectl exec -it deploy/nginx -- ls /mnt
    ```
16. 파일 내용 확인&#x20;

    ```
    kubectl exec -it deploy/nginx -- cat /mnt/secret-file
    ```
17. SecretProviderClass 수정&#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: secrets-store.csi.x-k8s.io/v1
    kind: SecretProviderClass
    metadata:
      name: secret-file
    spec:
      provider: aws
      parameters:
        objects: |
          - objectName: "secret-file"
            objectType: "secretsmanager"
            jmesPath:
            - path: '"application.properties"'
              objectAlias: application.properties
    EOF
    ```
18. Pod 재생성

    ```
    {
      kubectl scale deploy nginx --replicas=0
      kubectl scale deploy nginx --replicas=1
    }
    ```
19. Pod에 마운트된 파일 확인&#x20;

    ```
    kubectl exec -it deploy/nginx -- ls /mnt
    ```
20. 파일 내용 확인&#x20;

    ```
    kubectl exec -it deploy/nginx -- cat /mnt/application.properties
    ```
21. 파일 수정&#x20;

    ```
    cat <<EOF | tee application.properties
    a=b
    c=d
    e=f
    g=h
    additonal_key=additonal_value
    EOF
    ```
22. JSON 파일 생성&#x20;

    ```
    {
      jq -n --argjson application.properties \
      $(jq -Rs '.' application.properties) '$ARGS.named' > secret-file.json
      
      cat secret-file.json
    }
    ```
23. Secret 수정&#x20;

    ```
    aws secretsmanager put-secret-value --secret-id secret-file \
    --secret-string file://secret-file.json
    ```
24. 수정된 Secret 확인&#x20;

    ```
    aws secretsmanager get-secret-value --secret-id secret-file
    ```
25. 수정 사항이 반영되는지 확인 &#x20;

    ```
    kubectl exec -it deploy/nginx -- cat /mnt/application.properties
    ```
26. Pod에 발생한 이벤트 확인&#x20;

    ```
    kubectl describe pod -l app=nginx
    ```
27. AWS Secrets Manager에 저장할 파일 생성

    ```
    cat <<EOF | tee secret-env.json
    {
      "ACCESS_KEY": "ABCDEFG",
      "SECRET_ACCESS_KEY": "ASDF1234"
    }
    EOF
    ```
28. Secret 생성&#x20;

    ```
    aws secretsmanager create-secret --name secret-env \
    --secret-string file://secret-env.json
    ```
29. 생성된 Secret 확인&#x20;

    ```
    aws secretsmanager get-secret-value --secret-id secret-env
    ```
30. SecretProviderClass 생성&#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: secrets-store.csi.x-k8s.io/v1
    kind: SecretProviderClass
    metadata:
      name: secret-env
    spec:
      provider: aws
      parameters:
        objects: |
          - objectName: "secret-env"
            objectType: "secretsmanager"
            jmesPath:
            - path: ACCESS_KEY
              objectAlias: ACCESS_KEY
            - path: SECRET_ACCESS_KEY
              objectAlias: SECRET_ACCESS_KEY
      secretObjects:
      - secretName: secret-env
        type: Opaque
        data:
        - objectName: ACCESS_KEY
          key: ACCESS_KEY
        - objectName: SECRET_ACCESS_KEY
          key: SECRET_ACCESS_KEY
    EOF
    ```
31. 데모 애플리케이션 수정 &#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          serviceAccountName: nginx
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: secret
              mountPath: /mnt
            envFrom:
            - secretRef:
                name: secret-env
          volumes:
          - name: secret
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: secret-env
    EOF
    ```
32. Pod에 지정된 환경 변수 확인&#x20;

    ```
    kubectl exec -it deploy/nginx -- printenv
    ```
33. 생성된 Secret 확인&#x20;

    ```
    kubectl get secret
    ```
34. 생성된 Secret 데이터의 평문 확인

    ```
    kubectl get secret secret-env -o jsonpath={.data.ACCESS_KEY} \
    | base64 -d && echo
    ```
35. 파일 수정&#x20;

    ```
    cat <<EOF | tee secret-env.json
    {
      "ACCESS_KEY": "QWER1234",
      "SECRET_ACCESS_KEY": "ASDF1234"
    }
    EOF
    ```
36. Secret 수정&#x20;

    ```
    aws secretsmanager put-secret-value --secret-id secret-env \
    --secret-string file://secret-env.json
    ```
37. 수정된 Secret 확인&#x20;

    ```
    aws secretsmanager get-secret-value --secret-id secret-env
    ```
38. Pod에 수정 사항이 반영되는지 확인 &#x20;

    ```
    kubectl exec -it deploy/nginx -- printenv
    ```
39. Pod에 발생한 이벤트 확인&#x20;

    ```
    kubectl describe pod -l app=nginx
    ```
40. Secret에 수정 사항이 반영되었는지 확인&#x20;

    ```
    kubectl get secret secret-env -o jsonpath={.data.ACCESS_KEY} \
    | base64 -d && echo
    ```
41. Pod에 수정 사항이 반영되는지 확인 &#x20;

    ```
    kubectl exec -it deploy/nginx -- printenv
    ```
42. Reloader 설치 - <https://github.com/stakater/Reloader>

    ```
    {
      helm repo add stakater https://stakater.github.io/stakater-charts
      helm install reloader stakater/reloader \
      --set reloader.autoReloadAll=true \
      --namespace kube-system
    }
    ```
43. 파일 수정&#x20;

    ```
    cat <<EOF | tee secret-env.json
    {
      "ACCESS_KEY": "ZXCV1234",
      "SECRET_ACCESS_KEY": "ASDF1234"
    }
    EOF
    ```
44. Secret 수정&#x20;

    ```
    aws secretsmanager put-secret-value --secret-id secret-env \
    --secret-string file://secret-env.json
    ```
45. 수정된 Secret 확인&#x20;

    ```
    aws secretsmanager get-secret-value --secret-id secret-env
    ```
46. Pod에 발생한 이벤트 확인&#x20;

    ```
    kubectl describe pod -l app=nginx
    ```
47. Secret에 수정 사항이 반영되었는지 확인&#x20;

    ```
    kubectl get secret secret-env -o jsonpath={.data.ACCESS_KEY} \
    | base64 -d && echo
    ```
48. Pod에 수정 사항이 반영되는지 확인 &#x20;

    ```
    kubectl exec -it deploy/nginx -- printenv
    ```
49. 리소스 삭제&#x20;

    ```
    {
      kubectl delete deploy nginx
      
      helm uninstall secrets-store-csi-driver-provider-aws -n kube-system
      helm uninstall csi-secrets-store -n kube-system
      helm uninstall reloader -n kube-system
      
      aws secretsmanager delete-secret \
      --secret-id secret-file \
      --recovery-window-in-days 7
      
      aws secretsmanager delete-secret \
      --secret-id secret-env \
      --recovery-window-in-days 7
      
      eksctl delete iamserviceaccount \
      --cluster $CLUSTER_NAME \
      --namespace=default \
      --name=nginx
    }
    ```
