> For the complete documentation index, see [llms.txt](https://kubernetes.youngwjung.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://kubernetes.youngwjung.com/core-concepts/statefulset/lab.md).

# 실습

### StatefulSet

1. Service 생성 

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: Service
   metadata:
     name: web
     labels:
       app: httpd
   spec:
     ports:
     - port: 80
     clusterIP: None
     selector:
       app: httpd
   EOF
   ```
2. 생성된 Service 및 Endpoint 확인 &#x20;

   ```
   kubectl get svc,ep -l app=httpd
   ```
3. StatefulSet 생성&#x20;

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     name: httpd
     labels:
       app: httpd
   spec:
     serviceName: web
     replicas: 1
     selector:
       matchLabels:
         app: httpd
     template:
       metadata:
         labels:
           app: httpd
       spec:
         containers:
         - name: httpd
           image: httpd
   EOF
   ```
4. 생성된 StatefulSet 및 Pod 확인 &#x20;

   ```
   kubectl get sts,pod -l app=httpd
   ```
5. Endpoint 확인 &#x20;

   ```
   kubectl get ep web
   ```
6. Endpoint에 포함된 Pod의 상세정보 확인

   ```
   kubectl get ep web -o=jsonpath='{.subsets}' | jq
   ```
7. 생성된 Pod의 이름 및 IP 확인

   ```
   kubectl get pod -l app=httpd -o wide
   ```
8. Pod 생성&#x20;

   ```
   kubectl run curl --image=curlimages/curl --command -- sleep 3600
   ```
9. cURL 명령어로 기존에 생성한 Pod에 배포된 웹서버 호출&#x20;

   ```
   kubectl exec curl -- curl -s httpd-0.web
   ```
10. StatefulSet의 Replica 수를 3개로 수정&#x20;

    ```
    kubectl scale statefulset httpd --replicas=3
    ```
11. 생성된 Pod 확인&#x20;

    ```
    kubectl get pod -l app=httpd
    ```
12. cURL 명령어로 새로 생성된 Pod에 배포된 웹서버 호출

    ```
    {
        kubectl exec curl -- curl -s httpd-1.web
        kubectl exec curl -- curl -s httpd-2.web
    }
    ```
13. 첫번째로 생성된 Pod를 삭제&#x20;

    ```
    kubectl delete pod httpd-0
    ```
14. Pod가 다시 생성되는지 확인&#x20;

    ```
    kubectl get pod -l app=httpd
    ```
15. 생성된 리스소 삭제&#x20;

    ```
    {
        kubectl delete sts httpd
        kubectl delete svc web 
        kubectl delete pod curl
    }
    ```

### Persistent Volume

1. Persistent Volume 종류 확인 - <https://kubernetes.io/docs/concepts/storage/persistent-volumes/#types-of-persistent-volumes>
2. 현재 설정된 Storage Class 확인&#x20;

   ```
   kubectl get storageclass
   ```
3. NFS 서버 생성

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: nfs
     labels:
       app: nfs
   spec:
     containers:
     - name: nfs
       image: erichough/nfs-server
       ports:
       - containerPort: 2049
       volumeMounts:
       - name: data
         mountPath: /data
       - name: kernel-module
         mountPath: /lib/modules
         readOnly: true
       env:
       - name: NFS_EXPORT_0
         value: "/data/ *(rw,sync,no_root_squash,subtree_check)"
       securityContext:
         capabilities:
           add:
           - CAP_SYS_ADMIN
           - SYS_MODULE
     initContainers:
     - name: index
       image: busybox
       command: ["/bin/chmod","-R","1777", "/data"]
       volumeMounts:
       - name: data
         mountPath: /data
     volumes:
     - name: data
       emptyDir: {}
     - name: kernel-module
       hostPath:
         path: /lib/modules
   EOF
   ```
4. NFS 서버 생성 확인

   ```
   kubectl get pod -l app=nfs
   ```
5. NFS 서버에 HTML 파일 생성

   ```
   kubectl exec nfs -c nfs -- \
   bash -c 'echo "Greeting from NFS-powered NGINX" > /data/index.html'
   ```
6. HTML 파일이 정상적으로 생성되었는지 확인

   ```
   kubectl exec nfs -c nfs -- cat /data/index.html
   ```
7. PersistentVolume 생성&#x20;

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: my-pv
   spec:
     storageClassName: manual
     capacity:
       storage: 1Gi
     accessModes:
       - ReadWriteMany
     persistentVolumeReclaimPolicy: Retain
     nfs:
       path: /data
       server: $(kubectl get pod -l app=nfs -o=jsonpath='{.items[0].status.podIP}')
       readOnly: false
   EOF
   ```
8. 생성된 PersistentVolume 확인&#x20;

   ```
   kubectl get pv my-pv
   ```
9. 생성된 PVC가 있는지 확인&#x20;

   ```
   kubectl get pvc
   ```
10. PVC 생성 &#x20;

    ```
    cat <<EOF | kubectl create -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-pvc
    spec:
      accessModes:
      - ReadWriteMany
      resources:
         requests:
           storage: 200Mi
      storageClassName: manual
    EOF
    ```
11. 생성된 PVC 확인&#x20;

    ```
    kubectl get pvc my-pvc
    ```
12. PV 확인&#x20;

    ```
    kubectl get pv my-pv
    ```
13. 데모 애플리케이션 배포

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: data
              mountPath: /usr/share/nginx/html
          volumes:
          - name: data
            persistentVolumeClaim:
              claimName: my-pvc
    EOF
    ```
14. 위에서 배포한 애플리케이션의 Pod가 정상적으로 생성되었는지 확인

    ```
    kubectl get pod -l app=nginx
    ```
15. 배포된 NGINX 웹서버들이 동일한 페이지를 보여주는지 확인

    ```
    for pod in $(kubectl get pod -l app=nginx -o=jsonpath='{.items[*].metadata.name}')
    do
      kubectl exec $pod -- curl -s localhost
    done
    ```
16. &#x20;PVC 상태 확인&#x20;

    ```
    kubectl get pvc
    ```
17. 하나의 Pod에 접속해서 PV가 마운트된 경로에 파일 생성&#x20;

    ```
    kubectl exec deploy/nginx -- \
    bash -c  "echo helloworld > /usr/share/nginx/html/hello.txt"
    ```
18. NFS 서버에 접속해서 위에서 생성한 파일이 보이는지 확인&#x20;

    ```
    kubectl exec nfs -c nfs -- cat /data/hello.txt
    ```
19. 배포된 Pod들에서 위에서 생성한 파일이 보이는지 확인

    ```
    for pod in $(kubectl get pod -l app=nginx -o=jsonpath='{.items[*].metadata.name}')
    do
      kubectl exec $pod -- curl -s localhost/hello.txt
    done
    ```
20. 데모 애플리케이션 삭제 &#x20;

    ```
    kubectl delete deploy nginx
    ```
21. NFS 서버에 접속해서 파일이 삭제되었는지 확인&#x20;

    ```
    kubectl exec nfs -c nfs -- ls /data
    ```
22. PVC 상태 확인&#x20;

    ```
    kubectl get pvc my-pvc
    ```
23. 데모 애플리케이션 재배포&#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: data
              mountPath: /usr/share/nginx/html
          volumes:
          - name: data
            persistentVolumeClaim:
              claimName: my-pvc
    EOF
    ```
24. 이전에 생성했던 파일이 존재 하는지 확인&#x20;

    ```
    for pod in $(kubectl get pod -l app=nginx -o=jsonpath='{.items[*].metadata.name}')
    do
      kubectl exec $pod -- ls /usr/share/nginx/html
    done
    ```
25. PVC 삭제  시도

    ```
    kubectl delete pvc my-pvc
    ```
26. 위에서 실행한 명령어가 완료되지 않을 경우에 Ctrl+C를 입력해서 명령어 종료
27. PVC 상태 확인&#x20;

    ```
    kubectl get pvc my-pvc
    ```
28. PVC에 명시된 Finalizers 확인 - <https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storage-object-in-use-protection>

    ```
    kubectl get pvc my-pvc -o=jsonpath='{.metadata.finalizers}{"\n"}'
    ```
29. 데모 애플리케이션 삭제 &#x20;

    ```
    kubectl delete deploy nginx
    ```
30. PVC가 삭제되었는지 확인

    ```
    kubectl get pvc my-pvc
    ```
31. PV 상태 확인&#x20;

    ```
    kubectl get pv my-pv
    ```
32. PVC 생성 &#x20;

    ```
    cat <<EOF | kubectl create -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-pvc
    spec:
      accessModes:
      - ReadWriteMany
      resources:
         requests:
           storage: 200Mi
      storageClassName: manual
    EOF
    ```
33. PVC 상태 확인&#x20;

    ```
    kubectl get pvc my-pvc
    ```
34. PV에 명시된 PVC 정보 확인

    ```
    kubectl get pv my-pv -o=jsonpath='{.spec.claimRef}' | jq
    ```
35. 새로 생성한 PVC 상세정보 확인

    ```
    kubectl get pvc my-pvc -o yaml
    ```
36. PV 수정 - `spec.claimRef` 삭제&#x20;

    ```
    kubectl patch pv my-pv \
    --type=json -p='[{"op": "remove", "path": "/spec/claimRef"}]'
    ```
37. PVC 상태 확인&#x20;

    ```
    kubectl get pvc my-pvc
    ```
38. PV에 명시된 PVC 정보 확인

    ```
    kubectl get pv my-pv -o=jsonpath='{.spec.claimRef}' | jq
    ```
39. PVC의 uid 확인

    ```
    kubectl get pvc my-pvc -o=jsonpath='{.metadata.uid}{"\n"}'
    ```
40. PV 삭제 시도

    ```
    kubectl delete pv my-pv 
    ```
41. 위에서 실행한 명령어가 완료되지 않을 경우에 Ctrl+C를 입력해서 명령어 종료
42. PV 상태 확인&#x20;

    ```
    kubectl get pv my-pv
    ```
43. PV에 명시된 Finalizers 확인 - <https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storage-object-in-use-protection>

    ```
    kubectl get pv my-pv -o=jsonpath='{.metadata.finalizers}{"\n"}'
    ```
44. PVC 삭제

    ```
    kubectl delete pvc my-pvc 
    ```
45. PV가 삭제되었는지 확인

    ```
    kubectl get pv my-pv
    ```
46. NFS 서버에 접속해서 파일이 삭제되었는지 확인&#x20;

    ```
    kubectl exec nfs -c nfs -- ls /data
    ```
47. NFS 서버 삭제&#x20;

    ```
    kubectl delete pod nfs 
    ```

### Dynamic Volume Provisioning

1. 현재 설정된 StorageClass 확인&#x20;

   ```
   kubectl get storageclass
   ```
2. PVC 생성

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: my-pvc
   spec:
     accessModes:
     - ReadWriteMany
     resources:
        requests:
          storage: 200Mi
   EOF
   ```
3. PVC 상태 확인

   ```
   kubectl get pvc my-pvc
   ```
4. PVC 상태가 Pending인 이유 확인

   ```
   kubectl describe pvc my-pvc
   ```
5. 데모 애플리케이션 배포

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     labels:
       app: nginx
     name: nginx
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: nginx
           ports:
           - containerPort: 80
           volumeMounts:
           - name: data
             mountPath: /mnt
         volumes:
         - name: data
           persistentVolumeClaim:
             claimName: my-pvc
   EOF
   ```
6. 위에서 배포한 애플리케이션의 Pod가 정상적으로 생성되었는지 확인

   ```
   kubectl get pod -l app=nginx
   ```
7. Pod 상태가 Pending인 이유 확인

   ```
   kubectl describe pod -l app=nginx
   ```
8. PVC 상태가 Pending인 이유 확인

   ```
   kubectl describe pvc my-pvc
   ```
9. PVC 재생성

   ```
   cat <<EOF | kubectl replace --force -f -
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: my-pvc
   spec:
     accessModes:
     - ReadWriteOnce
     resources:
        requests:
          storage: 200Mi
   EOF
   ```
10. Pod 상태 확인 - *Running 상태로 될때까지 시간이 걸릴수도 있음*

    ```
    kubectl get pod -l app=nginx
    ```
11. PVC 상태 확인

    ```
    kubectl get pvc my-pvc
    ```
12. PVC에 명시된 PV가 생성되었는지 확인&#x20;

    ```
    kubectl get pv $(kubectl get pvc my-pvc -o=jsonpath='{.spec.volumeName}')
    ```
13. PVC에 명시한 볼륨 요청 크기 만큼 실제로 할당되었는지 확인

    ```
    diff <(kubectl get pvc my-pvc -o=jsonpath='{.spec}' | jq) \
    <(kubectl get pvc my-pvc -o=jsonpath='{.status}' | jq) -y
    ```
14. EBS (gp2)의 최소 볼륨 크기 확인 - <https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html#solid-state-drives>
15. 데모 애플리케이션 삭제

    ```
    kubectl delete deploy nginx
    ```
16. PVC 삭제

    ```
    kubectl delete pvc my-pvc 
    ```
17. PV가 삭제되었는지 확인 - *삭제 될때까지 시간이 걸릴수도 있음*

    ```
    kubectl get pv
    ```
18. LimitRange 생성

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: LimitRange
    metadata:
      name: storagelimits
    spec:
      limits:
      - type: PersistentVolumeClaim
        max:
          storage: 16Ti
        min:
          storage: 1Gi
    EOF
    ```
19. LimitRagne 확인&#x20;

    ```
    kubectl describe ns default
    ```
20. PVC 생성 시도

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-pvc
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
         requests:
           storage: 200Mi
    EOF
    ```
21. PVC 생성

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: my-pvc
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
         requests:
           storage: 2Gi
    EOF
    ```
22. PVC 상태 확인

    ```
    kubectl get pvc my-pvc
    ```
23. 데모 애플리케이션 배포&#x20;

    ```
    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: data
              mountPath: /mnt
          volumes:
          - name: data
            persistentVolumeClaim:
              claimName: my-pvc
    EOF
    ```
24. Pod 상태 확인

    ```
    kubectl get pod -l app=nginx
    ```
25. PVC 상태 확인

    ```
    kubectl get pvc my-pvc
    ```
26. PVC에 명시된 PV가 생성되었는지 확인&#x20;

    ```
    kubectl get pv $(kubectl get pvc my-pvc -o=jsonpath='{.spec.volumeName}')
    ```
27. PVC에 명시한 볼륨 요청 크기 만큼 실제로 할당되었는지 확인

    ```
    diff <(kubectl get pvc my-pvc -o=jsonpath='{.spec}' | jq) \
    <(kubectl get pvc my-pvc -o=jsonpath='{.status}' | jq) -y
    ```
28. Deployment의 Replica 수를 10개로 수정&#x20;

    ```
    kubectl scale deploy nginx --replicas=10
    ```
29. 추가로 생성되는 Pod 상태 확인

    ```
    kubectl get pod -l app=nginx
    ```
30. Pending 상태의 Pod가 있다면 그 이유 확인

    ```
    kubectl describe pod \
    $(kubectl get pod -l app=nginx \
    -o=jsonpath='{.items[?(@.status.phase=="Pending")].metadata.name}')
    ```
31. PV에 명시된 Node Affinity 확인 - <https://kubernetes.io/docs/concepts/storage/persistent-volumes/#node-affinity>

    ```
    kubectl get pv $(kubectl get pvc my-pvc -o=jsonpath='{.spec.volumeName}') \
    -o=jsonpath='{.spec.nodeAffinity}' | jq
    ```
32. 각 Node의 가용영역 확인&#x20;

    ```
    kubectl get nodes --label-columns topology.kubernetes.io/zone
    ```
33. Pod가 배포된 Node 확인&#x20;

    ```
    kubectl get pod -l app=nginx -o wide
    ```
34. Deployment의 Replica 수를 1개로 수정&#x20;

    ```
    kubectl scale deploy nginx --replicas=1
    ```
35. Pod가 배포된 Node의 이름 확인

    ```
    kubectl get pod -l app=nginx \
    -o=jsonpath='{.items[0].spec.nodeName}{"\n"}'
    ```
36. 해당 Node에 문제가 생겼다는 상황을 가정하고 Pod를 다른 Node로 이전하고 Node 상태를 SchedulingDisabled로 변경

    ```
    kubectl drain $(kubectl get pod -l app=nginx \
    -o=jsonpath='{.items[0].spec.nodeName}') \
    --ignore-daemonsets --delete-emptydir-data
    ```
37. Pod가 생성되는지 확인

    ```
    kubectl get pod -l app=nginx
    ```
38. Pod가 Pending 상태라면 그 이유를 확인

    ```
    kubectl describe pod \
    $(kubectl get pod -l app=nginx \
    -o=jsonpath='{.items[?(@.status.phase=="Pending")].metadata.name}')
    ```
39. Node 상태 확인

    ```
    kubectl get node
    ```
40. Node에 부여된 Taint 확인

    ```
    kubectl get nodes \
    -o=custom-columns=NodeName:.metadata.name,TaintKey:.spec.taints[*].key,TaintValue:.spec.taints[*].value,TaintEffect:.spec.taints[*].effect
    ```
41. PV 이름 확인

    ```
    kubectl get pvc my-pvc -o=jsonpath='{.spec.volumeName}{"\n"}'
    ```
42. PV에 연동된 EBS 볼륨를 확인하고 볼륨 ID를 환경변수로 지정

    ```
    {
        export VOL_ID=$(kubectl get pv $(kubectl get pvc my-pvc \
        -o=jsonpath='{.spec.volumeName}') \
        -o jsonpath='{.spec.awsElasticBlockStore.volumeID}' \
        | grep -oE "vol-[a-z0-9]+")

        echo $VOL_ID
    }
    ```
43. 위에서 확인한 EBS 볼륨 상태 확인

    ```
    aws ec2 describe-volumes --volume-ids $VOL_ID
    ```
44. Node가 정상으로 돌아왔다고 가정하고 Node에 부여된 Taint 삭제

    ```
    kubectl uncordon \
    $(kubectl get node -o=jsonpath='{.items[?(@.spec.unschedulable==true)].metadata.name}')
    ```
45. Node 상태 확인

    ```
    kubectl get node
    ```
46. Pod가 생성되는지 확인

    ```
    kubectl get pod -l app=nginx
    ```
47. PV에 연동된 EBS 볼륨 상태 확인

    ```
    aws ec2 describe-volumes --volume-ids $VOL_ID
    ```
48. 리소스 삭제

    ```
    {
        kubectl delete deploy nginx
        kubectl delete pvc my-pvc
        kubectl delete limitrange storagelimits
    }
    ```
49. PV가 삭제되었는지 확인 - *삭제 될때까지 시간이 걸릴수도 있음*

    ```
    kubectl get pv
    ```
50. PV에 연동된 EBS 볼륨이 삭제되었는지 확인

    ```
    aws ec2 describe-volumes --volume-ids $VOL_ID
    ```

### MySQL 클러스터 구성&#x20;

1. ConfigMap 생성&#x20;

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: mysql
     labels:
       app: mysql
   data:
     primary.cnf: |
       # Apply this config only on the primary.
       [mysqld]
       log-bin    
     replica.cnf: |
       # Apply this config only on replicas.
       [mysqld]
       super-read-only    
   EOF
   ```
2. Service 생성&#x20;

   ```
   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: Service
   metadata:
     name: mysql
     labels:
       app: mysql
   spec:
     ports:
     - name: mysql
       port: 3306
     clusterIP: None
     selector:
       app: mysql
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: mysql-read
     labels:
       app: mysql
   spec:
     ports:
     - name: mysql
       port: 3306
     selector:
       app: mysql
   EOF
   ```
3. StatefulSet 생성

   ```
   cat <<'EOF' | kubectl apply -f -
   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     name: mysql
   spec:
     selector:
       matchLabels:
         app: mysql
     serviceName: mysql
     replicas: 3
     template:
       metadata:
         labels:
           app: mysql
       spec:
         initContainers:
         - name: init-mysql
           image: mysql:5.7
           command:
           - bash
           - "-c"
           - |
             set -ex
             # Generate mysql server-id from pod ordinal index.
             [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
             ordinal=${BASH_REMATCH[1]}
             echo [mysqld] > /mnt/conf.d/server-id.cnf
             # Add an offset to avoid reserved server-id=0 value.
             echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf
             # Copy appropriate conf.d files from config-map to emptyDir.
             if [[ $ordinal -eq 0 ]]; then
               cp /mnt/config-map/primary.cnf /mnt/conf.d/
             else
               cp /mnt/config-map/replica.cnf /mnt/conf.d/
             fi          
           volumeMounts:
           - name: conf
             mountPath: /mnt/conf.d
           - name: config-map
             mountPath: /mnt/config-map
         - name: clone-mysql
           image: gcr.io/google-samples/xtrabackup:1.0
           command:
           - bash
           - "-c"
           - |
             set -ex
             # Skip the clone if data already exists.
             [[ -d /var/lib/mysql/mysql ]] && exit 0
             # Skip the clone on primary (ordinal index 0).
             [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
             ordinal=${BASH_REMATCH[1]}
             [[ $ordinal -eq 0 ]] && exit 0
             # Clone data from previous peer.
             ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql
             # Prepare the backup.
             xtrabackup --prepare --target-dir=/var/lib/mysql          
           volumeMounts:
           - name: data
             mountPath: /var/lib/mysql
             subPath: mysql
           - name: conf
             mountPath: /etc/mysql/conf.d
         containers:
         - name: mysql
           image: mysql:5.7
           env:
           - name: MYSQL_ALLOW_EMPTY_PASSWORD
             value: "1"
           ports:
           - name: mysql
             containerPort: 3306
           volumeMounts:
           - name: data
             mountPath: /var/lib/mysql
             subPath: mysql
           - name: conf
             mountPath: /etc/mysql/conf.d
           resources:
             requests:
               cpu: 500m
               memory: 500Mi
           livenessProbe:
             exec:
               command: ["mysqladmin", "ping"]
             initialDelaySeconds: 30
             periodSeconds: 10
             timeoutSeconds: 5
           readinessProbe:
             exec:
               # Check we can execute queries over TCP (skip-networking is off).
               command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]
             initialDelaySeconds: 5
             periodSeconds: 2
             timeoutSeconds: 1
         - name: xtrabackup
           image: gcr.io/google-samples/xtrabackup:1.0
           ports:
           - name: xtrabackup
             containerPort: 3307
           command:
           - bash
           - "-c"
           - |
             set -ex
             cd /var/lib/mysql

             # Determine binlog position of cloned data, if any.
             if [[ -f xtrabackup_slave_info && "x$(<xtrabackup_slave_info)" != "x" ]]; then
               # XtraBackup already generated a partial "CHANGE MASTER TO" query
               # because we're cloning from an existing replica. (Need to remove the tailing semicolon!)
               cat xtrabackup_slave_info | sed -E 's/;$//g' > change_master_to.sql.in
               # Ignore xtrabackup_binlog_info in this case (it's useless).
               rm -f xtrabackup_slave_info xtrabackup_binlog_info
             elif [[ -f xtrabackup_binlog_info ]]; then
               # We're cloning directly from primary. Parse binlog position.
               [[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1
               rm -f xtrabackup_binlog_info xtrabackup_slave_info
               echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\
                     MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in
             fi

             # Check if we need to complete a clone by starting replication.
             if [[ -f change_master_to.sql.in ]]; then
               echo "Waiting for mysqld to be ready (accepting connections)"
               until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done

               echo "Initializing replication from clone position"
               mysql -h 127.0.0.1 \
                     -e "$(<change_master_to.sql.in), \
                             MASTER_HOST='mysql-0.mysql', \
                             MASTER_USER='root', \
                             MASTER_PASSWORD='', \
                             MASTER_CONNECT_RETRY=10; \
                           START SLAVE;" || exit 1
               # In case of container restart, attempt this at-most-once.
               mv change_master_to.sql.in change_master_to.sql.orig
             fi

             # Start a server to send backups when requested by peers.
             exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \
               "xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"          
           volumeMounts:
           - name: data
             mountPath: /var/lib/mysql
             subPath: mysql
           - name: conf
             mountPath: /etc/mysql/conf.d
           resources:
             requests:
               cpu: 100m
               memory: 100Mi
         volumes:
         - name: conf
           emptyDir: {}
         - name: config-map
           configMap:
             name: mysql
     volumeClaimTemplates:
     - metadata:
         name: data
       spec:
         accessModes: ["ReadWriteOnce"]
         resources:
           requests:
             storage: 10Gi
   EOF
   ```
4. Pod 생성 확인&#x20;

   ```
   watch kubectl get pod -l app=mysql
   ```
5. 생성된 PVC 확인&#x20;

   ```
   kubectl get pvc -l app=mysql
   ```
6. PVC에 명시된 PV가 생성되었는지 확인&#x20;

   ```
   kubectl get pv \
   $(kubectl get pvc -l app=mysql -o=jsonpath='{.items[*].spec.volumeName}')
   ```
7. 각 PV에 맵핑된 EBS 볼륨 확인&#x20;

   ```
   for pv in $(kubectl get pvc -l app=mysql -o=jsonpath='{.items[*].spec.volumeName}')
   do
     aws ec2 describe-volumes --filters Name=tag:kubernetes.io/created-for/pv/name,Values=$pv
   done
   ```
8. Pod 생성&#x20;

   ```
   kubectl run mysql-client --image=mysql:5.7 -- sleep 3600
   ```
9. *mysql-0* Pod를 통해서 데이터베이스 및 레코드 생성&#x20;

   ```
   kubectl exec -i mysql-client -- \
   mysql -h mysql-0.mysql <<EOF
   CREATE DATABASE test;
   CREATE TABLE test.messages (message VARCHAR(250));
   INSERT INTO test.messages VALUES ('hello');
   EOF
   ```
10. 읽기 엔드포인트 *mysql-read* 통해서 위에서 생성한 데이터를 확인&#x20;

    ```
    kubectl exec -it mysql-client -- \
    mysql -h mysql-read -e "SELECT (SELECT message FROM test.messages) as message, (SELECT @@server_id) server_id;"
    ```
11. 읽기 엔드포인트 *mysql-read* 가 읽기에 대한 쿼리를 분산하는지 확인&#x20;

    ```
    kubectl exec -it mysql-client -- \
    bash -c "while sleep 1; do mysql -h mysql-read -e 'SELECT (SELECT message FROM test.messages) as message, (SELECT @@server_id) server_id;'; done"
    ```
12. Ctrl+C를 입력해서 위에서 실행한 프로세스 종료
13. *mysql-read* Endpoint에 타켓으로 포함되어 있는 Pod 목록 확인&#x20;

    ```
    kubectl get ep mysql-read \
    -o jsonpath='{.subsets[*].addresses[*].targetRef.name}{"\n"}'
    ```
14. *mysql-2* Pod의 ReadinessProbe가 실패하도록 수정&#x20;

    ```
    {
        kubectl exec mysql-2 -c mysql -- mv /usr/bin/mysql /usr/bin/mysql.off
        kubectl exec mysql-2 -c mysql -- cp /usr/bin/bash /usr/bin/mysql
    }
    ```
15. *mysql-2* Pod의 상태 확인&#x20;

    ```
    kubectl get pod mysql-2
    ```
16. *mysql-2* Pod의 상세 상태 확인&#x20;

    ```
    kubectl describe pod mysql-2
    ```
17. 읽기 엔드포인트 *mysql-read* 로 쿼리를 실행해서 *mysql-2* Pod로 트래픽이 전달되는지 확인&#x20;

    ```
    kubectl exec -it mysql-client -- \
    bash -c "while sleep 1; do mysql -h mysql-read -e 'SELECT (SELECT message FROM test.messages) as message, (SELECT @@server_id) server_id;'; done"
    ```
18. *mysql-read* Endpoint에 타켓으로 *mysql-2* Pod가 포함되어 있는지 확인&#x20;

    ```
    kubectl get ep mysql-read \
    -o jsonpath='{.subsets[*].addresses[*].targetRef.name}{"\n"}'
    ```
19. *mysql-2* Pod의 ReadinessProbe가 성공하도록 수정&#x20;

    ```
    kubectl exec mysql-2 -c mysql -- mv /usr/bin/mysql.off /usr/bin/mysql
    ```
20. *mysql-2* Pod의 상태 확인&#x20;

    ```
    kubectl get pod mysql-2
    ```
21. *mysql-read* Endpoint에 타켓으로 *mysql-2* Pod가 포함되어 있는지 확인&#x20;

    ```
    kubectl get ep mysql-read \
    -o jsonpath='{.subsets[*].addresses[*].targetRef.name}{"\n"}'
    ```
22. StatefulSet의 Replica 갯수를 0개로 조정&#x20;

    ```
    kubectl scale sts mysql --replicas=0
    ```
23. Pod 상태 확인&#x20;

    ```
    watch kubectl get pod -l app=mysql
    ```
24. PVC가 삭제되었는지 확인&#x20;

    ```
    kubectl get pvc -l app=mysql 
    ```
25. StatefulSet의 Replica 갯수를 3개로 조정&#x20;

    ```
    kubectl scale sts mysql --replicas=3
    ```
26. Pod 상태 확인&#x20;

    ```
    watch kubectl get pod -l app=mysql
    ```
27. 새로운 PVC가 생성되었는지 확인&#x20;

    ```
    kubectl get pvc -l app=mysql
    ```
28. 기존에 생성한 데이터베이스 및 레코드가 존재하는지 확인&#x20;

    ```
    kubectl exec -it mysql-client -- \
    bash -c "while sleep 1; do mysql -h mysql-read -e 'SELECT (SELECT message FROM test.messages) as message, (SELECT @@server_id) server_id;'; done"
    ```
29. Ctrl+C를 입력해서 위에서 실행한 프로세스 종료
30. StatefulSet 삭제&#x20;

    ```
    kubectl delete sts mysql
    ```
31. PVC가 삭제되었는지 확인&#x20;

    ```
    kubectl get pvc -l app=mysql
    ```
32. StatefulSet 재생성&#x20;

    ```
    cat <<'EOF' | kubectl apply -f -
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: mysql
    spec:
      selector:
        matchLabels:
          app: mysql
      serviceName: mysql
      replicas: 3
      template:
        metadata:
          labels:
            app: mysql
        spec:
          initContainers:
          - name: init-mysql
            image: mysql:5.7
            command:
            - bash
            - "-c"
            - |
              set -ex
              # Generate mysql server-id from pod ordinal index.
              [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
              ordinal=${BASH_REMATCH[1]}
              echo [mysqld] > /mnt/conf.d/server-id.cnf
              # Add an offset to avoid reserved server-id=0 value.
              echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf
              # Copy appropriate conf.d files from config-map to emptyDir.
              if [[ $ordinal -eq 0 ]]; then
                cp /mnt/config-map/primary.cnf /mnt/conf.d/
              else
                cp /mnt/config-map/replica.cnf /mnt/conf.d/
              fi          
            volumeMounts:
            - name: conf
              mountPath: /mnt/conf.d
            - name: config-map
              mountPath: /mnt/config-map
          - name: clone-mysql
            image: gcr.io/google-samples/xtrabackup:1.0
            command:
            - bash
            - "-c"
            - |
              set -ex
              # Skip the clone if data already exists.
              [[ -d /var/lib/mysql/mysql ]] && exit 0
              # Skip the clone on primary (ordinal index 0).
              [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
              ordinal=${BASH_REMATCH[1]}
              [[ $ordinal -eq 0 ]] && exit 0
              # Clone data from previous peer.
              ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql
              # Prepare the backup.
              xtrabackup --prepare --target-dir=/var/lib/mysql          
            volumeMounts:
            - name: data
              mountPath: /var/lib/mysql
              subPath: mysql
            - name: conf
              mountPath: /etc/mysql/conf.d
          containers:
          - name: mysql
            image: mysql:5.7
            env:
            - name: MYSQL_ALLOW_EMPTY_PASSWORD
              value: "1"
            ports:
            - name: mysql
              containerPort: 3306
            volumeMounts:
            - name: data
              mountPath: /var/lib/mysql
              subPath: mysql
            - name: conf
              mountPath: /etc/mysql/conf.d
            resources:
              requests:
                cpu: 500m
                memory: 500Mi
            livenessProbe:
              exec:
                command: ["mysqladmin", "ping"]
              initialDelaySeconds: 30
              periodSeconds: 10
              timeoutSeconds: 5
            readinessProbe:
              exec:
                # Check we can execute queries over TCP (skip-networking is off).
                command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]
              initialDelaySeconds: 5
              periodSeconds: 2
              timeoutSeconds: 1
          - name: xtrabackup
            image: gcr.io/google-samples/xtrabackup:1.0
            ports:
            - name: xtrabackup
              containerPort: 3307
            command:
            - bash
            - "-c"
            - |
              set -ex
              cd /var/lib/mysql

              # Determine binlog position of cloned data, if any.
              if [[ -f xtrabackup_slave_info && "x$(<xtrabackup_slave_info)" != "x" ]]; then
                # XtraBackup already generated a partial "CHANGE MASTER TO" query
                # because we're cloning from an existing replica. (Need to remove the tailing semicolon!)
                cat xtrabackup_slave_info | sed -E 's/;$//g' > change_master_to.sql.in
                # Ignore xtrabackup_binlog_info in this case (it's useless).
                rm -f xtrabackup_slave_info xtrabackup_binlog_info
              elif [[ -f xtrabackup_binlog_info ]]; then
                # We're cloning directly from primary. Parse binlog position.
                [[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1
                rm -f xtrabackup_binlog_info xtrabackup_slave_info
                echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\
                      MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in
              fi

              # Check if we need to complete a clone by starting replication.
              if [[ -f change_master_to.sql.in ]]; then
                echo "Waiting for mysqld to be ready (accepting connections)"
                until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done

                echo "Initializing replication from clone position"
                mysql -h 127.0.0.1 \
                      -e "$(<change_master_to.sql.in), \
                              MASTER_HOST='mysql-0.mysql', \
                              MASTER_USER='root', \
                              MASTER_PASSWORD='', \
                              MASTER_CONNECT_RETRY=10; \
                            START SLAVE;" || exit 1
                # In case of container restart, attempt this at-most-once.
                mv change_master_to.sql.in change_master_to.sql.orig
              fi

              # Start a server to send backups when requested by peers.
              exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \
                "xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"          
            volumeMounts:
            - name: data
              mountPath: /var/lib/mysql
              subPath: mysql
            - name: conf
              mountPath: /etc/mysql/conf.d
            resources:
              requests:
                cpu: 100m
                memory: 100Mi
          volumes:
          - name: conf
            emptyDir: {}
          - name: config-map
            configMap:
              name: mysql
      volumeClaimTemplates:
      - metadata:
          name: data
        spec:
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 10Gi
    EOF
    ```
33. Pod 생성 확인&#x20;

    ```
    watch kubectl get pod -l app=mysql
    ```
34. 새로운 PVC가 생성되었는지 확인&#x20;

    ```
    kubectl get pvc -l app=mysql
    ```
35. 기존에 생성한 데이터베이스 및 레코드가 존재하는지 확인&#x20;

    ```
    kubectl exec -it mysql-client -- \
    bash -c "while sleep 1; do mysql -h mysql-read -e 'SELECT (SELECT message FROM test.messages) as message, (SELECT @@server_id) server_id;'; done"
    ```
36. Ctrl+C를 입력해서 위에서 실행한 프로세스 종료
37. 리소스 삭제 &#x20;

    ```
    {
        kubectl delete sts mysql
        kubectl delete pvc -l app=mysql
        kubectl delete svc mysql mysql-read
        kubectl delete cm mysql
        kubectl delete pod mysql-client
    }
    ```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://kubernetes.youngwjung.com/core-concepts/statefulset/lab.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.