실습

Introduction

1. 데모 애플리케이션 배포

   kubectl run nginx --image=nginx

2. 생성된 Pod 확인

   kubectl get pod -l run=nginx

3. NGINX가 정상 동작하는지 확인

   kubectl exec nginx -- curl -s localhost

4. Envoy 설정파일 생성

   cat <<EOF | kubectl apply -f -
   kind: ConfigMap
   apiVersion: v1
   metadata:
     name: envoy
   data:
     envoy.yaml: |
       static_resources:
         listeners:
         - name: http-listener
           address:
             socket_address:
               address: 0.0.0.0
               port_value: 80
   EOF

5. Envoy 배포

   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: envoy
     labels:
       app: envoy
   spec:
     containers:
     - name: envoy
       image: envoyproxy/envoy:v1.22.2
       volumeMounts:
       - name: envoy-conf
         mountPath: /etc/envoy
     volumes:
     - name: envoy-conf
       configMap:
         name: envoy
   EOF

6. Envoy가 정상적으로 실행되었는지 확인

   kubectl get pod -l app=envoy

7. Envoy 로그 확인

   kubectl logs envoy

8. TCP Proxy 필터 추가 - https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/tcp_proxy/v3/tcp_proxy.proto

   cat <<EOF | kubectl apply -f -
   kind: ConfigMap
   apiVersion: v1
   metadata:
     name: envoy
   data:
     envoy.yaml: |
       static_resources:
         listeners:
         - name: http-listener
           address:
             socket_address:
               address: 0.0.0.0
               port_value: 80
           filter_chains:
           - filters:
             - name: envoy.filters.network.tcp_proxy
               typed_config:
                 "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                 stat_prefix: ingress_tcp
                 cluster: nginx
   EOF

9. 수정한 설정파일 반영

   kubectl get pod envoy -o yaml | kubectl replace --force -f -

10. Envoy가 정상적으로 실행되었는지 확인

    kubectl get pod -l app=envoy

11. Request을 보낼 Pod 생성

    kubectl run curl --image=curlimages/curl -- sleep infinity

12. NGINX로 Request 생성

    kubectl exec curl \
    -- curl -s $(kubectl get pod nginx -o=jsonpath="{.status.podIP}")

13. NGINX 서버 로그 확인

    kubectl logs nginx

14. Envoy로 Request 생성

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")

15. Envoy 로그 확인

    kubectl logs envoy

16. TCP Proxy 필터에 로그 활성화 - https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/accesslog/v3/accesslog.proto

    cat <<EOF | kubectl apply -f -
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: envoy
    data:
      envoy.yaml: |
        static_resources:
          listeners:
          - name: http-listener
            address:
              socket_address:
                address: 0.0.0.0
                port_value: 80
            filter_chains:
            - filters:
              - name: envoy.filters.network.tcp_proxy
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                  stat_prefix: ingress_tcp
                  cluster: nginx
                  access_log:
                  - name: envoy.access_loggers.stdout
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
    EOF

17. 수정한 설정파일 반영

    kubectl get pod envoy -o yaml | kubectl replace --force -f -

18. Envoy로 Request 생성

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")

19. Envoy 로그 확인 - https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage

    kubectl logs envoy

20. 클러스터 추가

    cat <<EOF | kubectl apply -f -
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: envoy
    data:
      envoy.yaml: |
        static_resources:
          listeners:
          - name: http-listener
            address:
              socket_address:
                address: 0.0.0.0
                port_value: 80
            filter_chains:
            - filters:
              - name: envoy.filters.network.tcp_proxy
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                  stat_prefix: ingress_tcp
                  access_log:
                  - name: envoy.access_loggers.stdout
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                  cluster: nginx
          clusters:
          - name: nginx
            type: STATIC
            load_assignment:
              cluster_name: nginx
              endpoints:
              - lb_endpoints:
                - endpoint:
                    address:
                      socket_address:
                        address: $(kubectl get pod nginx -o=jsonpath="{.status.podIP}")
                        port_value: 80
    EOF

21. 수정한 설정파일 반영

    kubectl get pod envoy -o yaml | kubectl replace --force -f -

22. Envoy로 Request 생성

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")

23. Envoy 로그 확인

    kubectl logs envoy

24. NGINX 서버 로그 확인

    kubectl logs nginx

25. Envoy가 내보내는 지표 확인 - https://www.envoyproxy.io/docs/envoy/latest/operations/stats_overview

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")/stats

26. Admin API 활성화

    cat <<EOF | kubectl apply -f -
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: envoy
    data:
      envoy.yaml: |
        admin:
          address:
            socket_address: 
              address: 0.0.0.0
              port_value: 15000
        static_resources:
          listeners:
          - name: http-listener
            address:
              socket_address:
                address: 0.0.0.0
                port_value: 80
            filter_chains:
            - filters:
              - name: envoy.filters.network.tcp_proxy
                typed_config:
                  "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
                  stat_prefix: ingress_tcp
                  access_log:
                  - name: envoy.access_loggers.stdout
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
                  cluster: nginx
          clusters:
          - name: nginx
            type: STATIC
            load_assignment:
              cluster_name: nginx
              endpoints:
              - lb_endpoints:
                - endpoint:
                    address:
                      socket_address:
                        address: $(kubectl get pod nginx -o=jsonpath="{.status.podIP}")
                        port_value: 80
    EOF

27. 수정한 설정파일 반영

    kubectl get pod envoy -o yaml | kubectl replace --force -f -

28. Envoy가 내보내는 지표 확인

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}"):15000/stats

29. Envoy가 내보내는 Prometheus 형식의 지표 확인

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}"):15000/stats/prometheus

30. Envoy로 Request 생성

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")

31. 지표 확인

    kubectl exec curl \
    -- curl -s $(kubectl get pod envoy -o=jsonpath="{.status.podIP}"):15000/stats/prometheus \
    | grep -i ingress_tcp

32. 리소스 삭제

    kubectl delete pod nginx envoy curl
    kubectl delete cm envoy

HTTP Connection Manager

1. Envoy 설정파일 생성 - https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto

   cat <<EOF | kubectl apply -f -
   kind: ConfigMap
   apiVersion: v1
   metadata:
     name: envoy
   data:
     envoy.yaml: |
       static_resources:
         listeners:
         - name: http-listener
           address:
             socket_address:
               address: 0.0.0.0
               port_value: 80
           filter_chains:
           - filters:
             - name: envoy.filters.network.http_connection_manager
               typed_config:
                 "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                 stat_prefix: helloworld
                 http_filters:
                 - name: envoy.filters.http.router
                   typed_config:
                     "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
                 route_config:
                   virtual_hosts:
                   - name: helloworld
                     domains: ["*"]
                     routes:
                     - match:
                         prefix: "/"
                       direct_response:
                         status: 200
                         body:
                           inline_string: "helloworld"   
   EOF

2. Envoy 배포

   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: envoy
     labels:
       app: envoy
   spec:
     containers:
     - name: envoy
       image: envoyproxy/envoy:v1.22.2
       volumeMounts:
       - name: envoy-conf
         mountPath: /etc/envoy
     volumes:
     - name: envoy-conf
       configMap:
         name: envoy
   EOF

3. Request을 보낼 Pod 생성

   kubectl run curl --image=curlimages/curl -- sleep infinity

4. Envoy로 Request 생성

   kubectl exec curl \
   -- curl -s -w "\n" $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")

5. Routing 규칙 추가

   cat <<EOF | kubectl apply -f -
   kind: ConfigMap
   apiVersion: v1
   metadata:
     name: envoy
   data:
     envoy.yaml: |
       static_resources:
         listeners:
         - name: http-listener
           address:
             socket_address:
               address: 0.0.0.0
               port_value: 80
           filter_chains:
           - filters:
             - name: envoy.filters.network.http_connection_manager
               typed_config:
                 "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                 stat_prefix: helloworld
                 http_filters:
                 - name: envoy.filters.http.router
                   typed_config:
                     "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
                 route_config:
                   virtual_hosts:
                   - name: helloworld
                     domains: ["*"]
                     routes:
                     - match:
                         prefix: "/"
                       direct_response:
                         status: 200
                         body:
                           inline_string: "helloworld"
                     - match:
                         path: "/api"
                       direct_response:
                         status: 200
                         body:
                           inline_string: "helloworld api"            
   EOF

6. 수정한 설정파일 반영

   kubectl get pod envoy -o yaml | kubectl replace --force -f -

7. Envoy로 Request 생성

   kubectl exec curl \
   -- curl -s -w "\n" $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")/api

8. VirtualHost 설정 리뷰 - https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#config-route-v3-virtualhost
9. Routing 규칙 순서 변경

   cat <<EOF | kubectl apply -f -
   kind: ConfigMap
   apiVersion: v1
   metadata:
     name: envoy
   data:
     envoy.yaml: |
       static_resources:
         listeners:
         - name: http-listener
           address:
             socket_address:
               address: 0.0.0.0
               port_value: 80
           filter_chains:
           - filters:
             - name: envoy.filters.network.http_connection_manager
               typed_config:
                 "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                 stat_prefix: helloworld
                 http_filters:
                 - name: envoy.filters.http.router
                   typed_config:
                     "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
                 route_config:
                   virtual_hosts:
                   - name: helloworld
                     domains: ["*"]
                     routes:
                     - match:
                         path: "/api"
                       direct_response:
                         status: 200
                         body:
                           inline_string: "helloworld api"          
                     - match:
                         prefix: "/"
                       direct_response:
                         status: 200
                         body:
                           inline_string: "helloworld"
   EOF

10. 수정한 설정파일 반영

    kubectl get pod envoy -o yaml | kubectl replace --force -f -

11. Envoy로 Request 생성

    kubectl exec curl \
    -- curl -s -w "\n" $(kubectl get pod envoy -o=jsonpath="{.status.podIP}")/api