실습

Prerequisite

  1. AWS Load Balancer Controller에 명시된 내용을 참고해서 AWS Load Balancer Controller 설치

Installation

Ingress Gateway

TLS

  1. Istio 설치

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: true
            k8s:
              serviceAnnotations:
                service.beta.kubernetes.io/aws-load-balancer-type: external
                service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
                service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
                service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
              service:
                externalTrafficPolicy: Cluster
                loadBalancerSourceRanges:
                  - 0.0.0.0/0
    EOF
  2. CA 인증서 생성

    openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
    -subj '/CN=example.com' -keyout ca-key.pem -out ca-cert.pem
  3. example.com 도메인에 인증서 서명 요청 및 키 생성

    openssl req -newkey rsa:2048 -nodes \
    -subj "/CN=*.example.com" -keyout example.com.key -out example.com.csr
  4. 인증서 서명

    openssl x509 -req -sha256 -days 365 -CA ca-cert.pem -CAkey ca-key.pem \
    -set_serial 0 -in example.com.csr -out example.com.crt
  5. Secret 생성

    kubectl create -n istio-system secret tls example-com \
    --key=example.com.key --cert=example.com.crt
  6. Gateway 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway 
      servers:
      - port:
          number: 443
          name: https
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: example-com
        hosts:
        - "*"
    EOF
  7. VirtualService 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: nginx
    spec:
      hosts:
      - "nginx.example.com"
      gateways:
      - istio-gateway
      http:
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: nginx
            port:
              number: 80
    EOF
  8. 데모 애플리케이션 배포

    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      ports:
      - port: 80
      selector:
        app: nginx
    EOF
  9. 데모 애플리케이션 호출 - HTTPS

    curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v \
    https://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  10. Gateway에 HTTPS Redirect 설정

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway 
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        tls:
          httpsRedirect: true
        hosts:
        - "*"
      - port:
          number: 443
          name: https
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: example-com
        hosts:
        - "*"
    EOF
  11. 데모 애플리케이션 호출 - HTTP

    curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v -L \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  12. Gateway에 설정한 HTTPS Redirect 삭제

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway 
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
      - port:
          number: 443
          name: https
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: example-com
        hosts:
        - "*"
    EOF
  13. VirtualService HTTPS Redirect 설정

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: nginx
    spec:
      hosts:
      - "nginx.example.com"
      gateways:
      - istio-gateway
      http:
      - match:
        - scheme:
            exact: http
        redirect:
          scheme: https
          redirectCode: 308
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: nginx
            port:
              number: 80
    EOF
  14. 데모 애플리케이션 호출 - HTTPS

    curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v \
    https://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  15. 데모 애플리케이션 호출 - HTTP

    curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v -L \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  16. 리소스 삭제

    kubectl delete deploy nginx
    kubectl delete svc nginx
    kubectl delete virtualservices.networking.istio.io nginx 
    kubectl delete gateways.networking.istio.io istio-gateway
    kubectl delete -n istio-system secret example-com
    istioctl x uninstall -y --purge

TLS with NLB

  1. AWS Certificate Manager를 통해서 SSL 인증서 발급

  2. 사용할 ACM 인증서의 ARN을 환경변수로 지정

    export SSL_CERT=<ACM_CERT_ARN>

    e.g.

    export SSL_CERT=arn:aws:acm:ap-northeast-2:111111111111:certificate/6a427716-ffde-4676-a4f1-02086398df7f
  3. Istio 설치

    cat <<EOF | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: true
            k8s:
              serviceAnnotations:
                service.beta.kubernetes.io/aws-load-balancer-type: external
                service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
                service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
                service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
                service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
                service.beta.kubernetes.io/aws-load-balancer-ssl-cert: $SSL_CERT
              service:
                externalTrafficPolicy: Cluster
                loadBalancerSourceRanges:
                  - 0.0.0.0/0
              service:
                ports:
                  - name: http2
                    port: 80
                    targetPort: 8080
                  - name: status-port
                    port: 15020
                    targetPort: 15020
              overlays:
                - apiVersion: v1
                  kind: Service
                  name: istio-ingressgateway
                  patches:
                    - path: spec.ports[2]
                      value:
                        name: https
                        protocol: TCP
                        port: 443
                        targetPort: 8080
    EOF
  4. 사용할 Top 레벨 도메인주소를 환경변수로 지정

    export DOMAIN_NAME=<TLD>

    e.g.

    export DOMAIN_NAME=example.com
  5. Gateway 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
    EOF
  6. VirtualService 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: nginx
    spec:
      hosts:
      - "nginx.$DOMAIN_NAME"
      gateways:
      - istio-gateway
      http:
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: nginx
            port:
              number: 80
    EOF
  7. 데모 애플리케이션 배포

    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      ports:
      - port: 80
      selector:
        app: nginx
    EOF
  8. 아래의 명령어를 실행해서 나온 결과값과 같은 DNS 레코드 생성

    echo nginx.$DOMAIN_NAME=$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  9. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME
  10. 데모 애플리케이션 호출 - HTTP

    curl http://nginx.$DOMAIN_NAME
  11. Gateway에 HTTPS Redirect 설정

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
        tls:
          httpsRedirect: true
    EOF
  12. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME -v
  13. 데모 애플리케이션 호출 - HTTP

    curl -L http://nginx.$DOMAIN_NAME -v
  14. NLB와 Istio Ingress Gateway는 HTTP 프로토콜로 통신하므로 모든 요청이 HTTPS로 전환됨

  15. Gateway에 설정한 HTTPS Redirect 삭제

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
    EOF
  16. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME -v
  17. 데모 애플리케이션 호출 - HTTP

    curl -L http://nginx.$DOMAIN_NAME -v
  18. VirtualService에 HTTPS Redirect 설정

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: nginx
    spec:
      hosts:
      - "nginx.$DOMAIN_NAME"
      gateways:
      - istio-gateway
      http:
      - match:
        - scheme:
            exact: http
        redirect:
          scheme: https
          redirectCode: 308
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: nginx
            port:
              number: 80
    EOF
  19. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME -v
  20. 데모 애플리케이션 호출 - HTTP

    curl -L http://nginx.$DOMAIN_NAME -v
  21. VirtualService 설정한 HTTPS Redirect 삭제

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: nginx
    spec:
      hosts:
      - "nginx.$DOMAIN_NAME"
      gateways:
      - istio-gateway
      http:
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: nginx
            port:
              number: 80
    EOF
  22. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME -v
  23. 데모 애플리케이션 호출 - HTTP

    curl -L http://nginx.$DOMAIN_NAME -v
  24. Istio 설정변경 - NLB(HTTP) -> Istio Ingress Gateway(HTTP), NLB(HTTPS) -> Istio Ingress Gateway(HTTPS)로 라우팅 되도록 구성

    cat <<EOF | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: true
            k8s:
              serviceAnnotations:
                service.beta.kubernetes.io/aws-load-balancer-type: external
                service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
                service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
                service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
                service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
                service.beta.kubernetes.io/aws-load-balancer-ssl-cert: $SSL_CERT
              service:
                externalTrafficPolicy: Cluster
                loadBalancerSourceRanges:
                  - 0.0.0.0/0
    EOF
  25. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME -v
  26. 데모 애플리케이션 호출 - HTTP

    curl -L http://nginx.$DOMAIN_NAME -v
  27. Gateway에 443 포트 리스너 추가

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
      - port:
          number: 443
          name: https
          protocol: HTTP
        hosts:
        - "*"
    EOF
  28. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME -v
  29. Gateway에 HTTPS Redirect 설정

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
        tls:
          httpsRedirect: true
      - port:
          number: 443
          name: https
          protocol: HTTP
        hosts:
        - "*"
    EOF
  30. 데모 애플리케이션 호출 - HTTPS

    curl https://nginx.$DOMAIN_NAME -v
  31. 데모 애플리케이션 호출 - HTTP

    curl -L http://nginx.$DOMAIN_NAME -v
  32. 데모 애플리케이션 배포 - https://github.com/youngwjung/flask-echo

    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: api
      name: api
    spec:
      selector:
        matchLabels:
          app: api
      template:
        metadata:
          labels:
            app: api
        spec:
          containers:
          - name: api
            image: youngwjung/flask-echo
            ports:
            - containerPort: 80
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: api
      labels:
        app: api
    spec:
      ports:
      - port: 80
      selector:
        app: api
    EOF
  33. VirtualService 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: api
    spec:
      hosts:
      - "api.$DOMAIN_NAME"
      gateways:
      - istio-gateway
      http:
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: api
            port:
              number: 80
    EOF
  34. 아래의 명령어를 실행해서 나온 결과값과 같은 DNS 레코드 생성

    echo api.$DOMAIN_NAME=$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  35. 데모 애플리케이션 호출 - HTTPS

    curl -v -X POST https://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'
  36. 데모 애플리케이션 호출 - HTTP

    curl -L -v -X POST http://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'
  37. Gateway에 설정한 HTTPS Redirect 삭제

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
      - port:
          number: 443
          name: https
          protocol: HTTP
        hosts:
        - "*"
    EOF
  38. 데모 애플리케이션 호출 - HTTP

    curl -v -X POST http://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'
  39. VirtualService에 HTTPS Redirect 설정

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: api
    spec:
      hosts:
      - "api.$DOMAIN_NAME"
      gateways:
      - istio-gateway
      http:
      - match:
        - port: 80
        redirect:
          scheme: https
          redirectCode: 308
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: api
            port:
              number: 80
    EOF
  40. 데모 애플리케이션 호출 - HTTPS

    curl -v -X POST https://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'
  41. 데모 애플리케이션 호출 - HTTP

    curl -v -L -X POST http://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'
  42. 리소스 삭제

    kubectl delete deploy nginx api
    kubectl delete svc nginx api
    kubectl delete virtualservices.networking.istio.io nginx api
    kubectl delete gateways.networking.istio.io istio-gateway
    istioctl x uninstall -y --purge

Authorization

  1. Istio 설치

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      components:
        ingressGateways:
        - name: istio-ingressgateway
          enabled: true
          k8s:
            serviceAnnotations:
              service.beta.kubernetes.io/aws-load-balancer-type: external
              service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
              service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
        pilot:
          k8s:
            resources:
              requests:
                cpu: 200m
                memory: 500Mi
    EOF
  2. Gateway 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: istio-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
    EOF
  3. 데모 애플리케이션 배포

    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx
    spec:
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            ports:
            - containerPort: 80
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      ports:
      - port: 80
      selector:
        app: nginx
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: nginx
    spec:
      hosts:
      - nginx.example.com
      gateways:
      - istio-gateway
      http:
      - match:
        - uri:
            prefix: /
        route:
        - destination:
            host: nginx
            port:
              number: 80
    EOF
  4. 데모 애플리케이션 호출

    curl -H "Host: nginx.example.com" \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  5. 내 컴퓨터(혹은 현재 접속된 Cloud9 인스턴스) IP 주소 확인

    curl -s ifconfig.io
  6. 내 컴퓨터에서 Ingress Gateway로 접근 차단

    cat <<EOF | kubectl apply -f -
    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: ingress-policy
      namespace: istio-system
    spec:
      selector:
        matchLabels:
          app: istio-ingressgateway
      action: DENY
      rules:
      - from:
        - source:
            ipBlocks: 
            - $(curl -s ifconfig.io)
    EOF
  7. 데모 애플리케이션 호출

    curl -H "Host: nginx.example.com" \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  8. Ingress Gateway 로그 확인

    kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20
  9. Envoy 로그 활성화

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      meshConfig:
        accessLogFile: /dev/stdout
      components:
        ingressGateways:
        - name: istio-ingressgateway
          enabled: true
          k8s:
            serviceAnnotations:
              service.beta.kubernetes.io/aws-load-balancer-type: external
              service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
              service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
        pilot:
          k8s:
            resources:
              requests:
                cpu: 200m
                memory: 500Mi
    EOF
  10. Ingress Gateway 재생성

    kubectl -n istio-system delete pod -l app=istio-ingressgateway
  11. 데모 애플리케이션 호출

    curl -H "Host: nginx.example.com" \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  12. Ingress Gateway 로그 확인

    kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20
  13. NLB 설정 변경

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      meshConfig:
        accessLogFile: /dev/stdout
      components:
        ingressGateways:
        - name: istio-ingressgateway
          enabled: true
          k8s:
            serviceAnnotations:
              service.beta.kubernetes.io/aws-load-balancer-type: external
              service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
              service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
        pilot:
          k8s:
            resources:
              requests:
                cpu: 200m
                memory: 500Mi
    EOF
  14. 데모 애플리케이션 호출

    curl -H "Host: nginx.example.com" \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  15. Ingress Gateway 로그 확인

    kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20
  16. NLB 설정 변경

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      meshConfig:
        accessLogFile: /dev/stdout
      components:
        ingressGateways:
        - name: istio-ingressgateway
          enabled: true
          k8s:
            serviceAnnotations:
              service.beta.kubernetes.io/aws-load-balancer-type: external
              service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
              service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
              service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
        pilot:
          k8s:
            resources:
              requests:
                cpu: 200m
                memory: 500Mi
    EOF
  17. Envoy Filter 생성 - Proxy Protocol

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: EnvoyFilter
    metadata:
      name: proxy-protocol
      namespace: istio-system
    spec:
      workloadSelector:
        labels:
          istio: ingressgateway
      configPatches:
      - applyTo: LISTENER
        patch:
          operation: MERGE
          value:
            listener_filters:
            - name: envoy.filters.listener.proxy_protocol
            - name: envoy.filters.listener.tls_inspector
    EOF
  18. Envoy Filter 생성 - X-Forwarded-For 헤더

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: EnvoyFilter
    metadata:
      name: ingressgateway-settings
      namespace: istio-system
    spec:
      configPatches:
      - applyTo: NETWORK_FILTER
        match:
          listener:
            filterChain:
              filter:
                name: envoy.filters.network.http_connection_manager
        patch:
          operation: MERGE
          value:
            name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              skip_xff_append: false
              use_remote_address: true
              xff_num_trusted_hops: 1
    EOF
  19. 데모 애플리케이션 호출

    curl -H "Host: nginx.example.com" \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  20. Ingress Gateway 로그 확인

    kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20
  21. AuthorizationPolicy 수정 - https://istio.io/latest/docs/tasks/security/authorization/authz-ingress/#ip-based-allow-list-and-deny-list

    cat <<EOF | kubectl apply -f -
    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: ingress-policy
      namespace: istio-system
    spec:
      selector:
        matchLabels:
          app: istio-ingressgateway
      action: DENY
      rules:
      - from:
        - source:
            remoteIpBlocks: 
            - $(curl -s ifconfig.io)
    EOF
  22. 데모 애플리케이션 호출

    curl -H "Host: nginx.example.com" \
    http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')
  23. Ingress Gateway 로그 확인

    kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20
  24. 리소스 삭제

    {
        kubectl delete deploy nginx
        kubectl delete svc nginx
        istioctl x uninstall -y --purge
    }

Service Entry

Introduction

  1. Istio 설치

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: false
    EOF
  2. default 네임스페이스에 생성되는 Pod에 프록시 주입 설정

    kubectl label namespace default istio-injection=enabled --overwrite
  3. Pod 생성

    kubectl run nginx --image=nginx
  4. Pod가 생성되고 프록시 컨테이너가 추가되었는지 확인

    kubectl get pod nginx
  5. 생성한 Pod에서 httpbin.org 접근 시도

    kubectl exec nginx -- curl -si httpbin.org/json
  6. Istio 아웃바운드 트래픽 정책 변경 - https://istio.io/latest/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig-OutboundTrafficPolicy-Mode

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      meshConfig:
        outboundTrafficPolicy:
          mode: REGISTRY_ONLY
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: false
    EOF
  7. Pod에서 httpbin.org 접근 시도

    kubectl exec nginx -- curl -si httpbin.org/json
  8. ServiceEntry 생성 - https://istio.io/latest/docs/reference/config/networking/service-entry/#ServiceEntry

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: ServiceEntry
    metadata:
      name: httpbin-org
    spec:
      hosts:
      - httpbin.org
      ports:
      - number: 80
        name: http
        protocol: HTTP
      resolution: DNS
      location: MESH_EXTERNAL
    EOF
  9. Pod에서 httpbin.org 접근 시도

    kubectl exec nginx -- curl -si httpbin.org/json
  10. 리소스 삭제

    kubectl delete serviceentries.networking.istio.io httpbin-org
    kubectl delete pod nginx
    kubectl label namespace default istio-injection-
    istioctl x uninstall -y --purge

Deep Dive

  1. Istio 설치

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: false
    EOF
  2. default 네임스페이스에 생성되는 Pod에 프록시 주입 설정

    kubectl label namespace default istio-injection=enabled --overwrite
  3. Pod 생성

    kubectl run nginx --image=nginx
  4. Pod가 생성되고 프록시 컨테이너가 추가되었는지 확인

    kubectl get pod nginx
  5. Sidecar Proxy의 리스너 설정 확인 - https://istio.io/latest/docs/ops/diagnostic-tools/proxy-cmd/#deep-dive-into-envoy-configuration

    istioctl proxy-config listeners nginx
  6. Istio Sidecar Proxy가 사용하는 포트 확인 - https://istio.io/latest/docs/ops/deployment/requirements/#ports-used-by-istio

  7. Sidecar Proxy의 아웃바운드 설정 확인 - https://faun.pub/understanding-how-envoy-sidecar-intercept-and-route-traffic-in-istio-service-mesh-20fea2a78833

    istioctl proxy-config listeners nginx --port 15001 -o json

    요약: 목적지 포트가 15001일 경우에는 BlackHoleCluster로 아닐 경우에는 PassthroughCluster로

  8. Sidecar Proxy의 클러스터 설정 확인

    istioctl proxy-config cluster nginx
  9. PassthroughCluster의 상세 설정 확인 - InboundPassthroughClusterIpv4 및 PassthroughCluster에 대한 정보가 보여지므로 name 값을 확인해서 PassthroughCluster에 대한 정보만 확인

    istioctl proxy-config cluster nginx --fqdn PassthroughCluster -o json
  10. BlackHoleCluster의 상세 설정 확인

    istioctl proxy-config cluster nginx --fqdn BlackHoleCluster -o json
  11. 생성한 Pod에서 httpbin.org 접근 시도

    kubectl exec nginx -- curl -si httpbin.org/json
  12. Istio 아웃바운드 트래픽 정책 변경

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      meshConfig:
        outboundTrafficPolicy:
          mode: REGISTRY_ONLY
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: false
    EOF
  13. Pod에서 httpbin.org 접근 시도

    kubectl exec nginx -- curl -si http://httpbin.org/json
  14. Sidecar Proxy의 리스너 설정 확인

    istioctl proxy-config listeners nginx
  15. Sidecar Proxy의 아웃바운드 설정 확인

    istioctl proxy-config listeners nginx --port 15001 -o json

    요약: 목적지 포트가 15001일 경우에는 BlackHoleCluster로 아닐 경우에도 BlackHoleCluster로

  16. ServiceEntry 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: ServiceEntry
    metadata:
      name: httpbin-org
    spec:
      hosts:
      - httpbin.org
      ports:
      - number: 80
        name: http
        protocol: HTTP
      resolution: DNS
      location: MESH_EXTERNAL
    EOF
  17. Pod에서 httpbin.org 접근 시도

    kubectl exec nginx -- curl -si httpbin.org/json
  18. Sidecar Proxy의 Route 설정 확인

    istioctl proxy-config route nginx
  19. Sidecar Proxy의 Route 설정 확인 - 80/httpbin.org

    istioctl proxy-config route nginx --name 80 -o json

    요약: 포트 80으로 나갈때 호스트가 httpbin.org을 경우 httpbin.org 클러스터로 아닐 경우 502 반환

  20. Pod에서 google.com 접근 시도

    kubectl exec nginx -- curl -si google.com
  21. Sidecar Proxy의 클러스터 설정 확인

    istioctl proxy-config cluster nginx
  22. httpbin.org 의 상세 설정 확인

    istioctl proxy-config cluster nginx --fqdn httpbin.org -o json
  23. 리소스 삭제

    kubectl delete serviceentries.networking.istio.io httpbin-org
    kubectl delete pod nginx
    kubectl label namespace default istio-injection-
    istioctl x uninstall -y --purge

External HTTPS Proxy

  1. Istio 설치

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      meshConfig:
        accessLogFile: /dev/stdout
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: false
    EOF
  2. default 네임스페이스에 생성되는 Pod에 프록시 주입 설정

    kubectl label namespace default istio-injection=enabled --overwrite
  3. HTTPS Proxy를 배포할 Namespace 생성

    kubectl create namespace external
  4. Squid proxy 설정파일 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: squid-proxy-conf
      namespace: external
    data:
      squid.conf: |
        http_port 3128
    
        acl SSL_ports port 443
        acl CONNECT method CONNECT
        
        http_access deny CONNECT !SSL_ports
        http_access allow localhost manager
        http_access deny manager
        http_access allow all
        
        coredump_dir /var/spool/squid
    EOF
  5. Squid proxy 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: Pod
    metadata:
      name: squid
      namespace: external
    spec:
      volumes:
      - name: proxy-config
        configMap:
          name: squid-proxy-conf
      containers:
      - name: squid
        image: datadog/squid
        ports:
        - containerPort: 3128
        volumeMounts:
        - name: proxy-config
          mountPath: /etc/squid
    EOF
  6. Squid proxy 생성 확인

    kubectl -n external get pod
  7. Sidecar Proxy가 없는 Pod 생성

    kubectl -n external run nginx --image=nginx
  8. Squid proxy를 통해서 외부 HTTPS 서비스 호출

    kubectl -n external exec nginx -- \
    sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"
  9. Squid proxy를 통해서 외부 HTTP 서비스 호출

    kubectl -n external exec nginx -- \
    sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"
  10. Squid proxy 로그 확인

    kubectl -n external exec squid -- tail /var/log/squid/access.log
  11. Sidecar Proxy를 포함하는 Pod 생성

    kubectl run nginx --image=nginx
  12. Squid proxy를 통해서 외부 HTTPS 서비스 호출

    kubectl exec nginx -- \
    sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"
  13. Squid proxy를 통해서 외부 HTTP 서비스 호출

    kubectl exec nginx -- \
    sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"
  14. Squid proxy 로그 확인

    kubectl -n external exec squid -- tail /var/log/squid/access.log
  15. Sidecar Proxy 로그 확인

    kubectl logs nginx -c istio-proxy
  16. Istio 아웃바운드 트래픽 정책 변경

    cat <<'EOF' | istioctl install -y -f -
    apiVersion: install.istio.io/v1alpha1
    kind: IstioOperator
    spec:
      meshConfig:
        accessLogFile: /dev/stdout
        outboundTrafficPolicy:
          mode: REGISTRY_ONLY
      components:
        ingressGateways:
          - name: istio-ingressgateway
            enabled: false
    EOF
  17. Squid proxy를 통해서 외부 HTTPS 서비스 호출

    kubectl exec nginx -- \
    sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"
  18. Squid proxy를 통해서 외부 HTTP 서비스 호출

    kubectl exec nginx -- \
    sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"
  19. Squid proxy 로그 확인

    kubectl -n external exec squid -- tail /var/log/squid/access.log
  20. Sidecar Proxy 로그 확인

    kubectl logs nginx -c istio-proxy
  21. ServiceEntry 생성

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: ServiceEntry
    metadata:
      name: external-proxy
    spec:
      hosts:
      - squid
      addresses:
      - $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}')/32
      ports:
      - number: 3128
        name: http
        protocol: HTTP
      location: MESH_EXTERNAL
    EOF
  22. Squid proxy를 통해서 외부 HTTPS 서비스 호출

    kubectl exec nginx -- \
    sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"
  23. Squid proxy를 통해서 외부 HTTP 서비스 호출

    kubectl exec nginx -- \
    sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"
  24. Squid proxy 로그 확인

    kubectl -n external exec squid -- tail /var/log/squid/access.log
  25. Sidecar Proxy 로그 확인

    kubectl logs nginx -c istio-proxy
  26. ServiceEntry 변경 - TCP 프로토콜 사용

    cat <<EOF | kubectl apply -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: ServiceEntry
    metadata:
      name: external-proxy
    spec:
      hosts:
      - squid
      addresses:
      - $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}')/32
      ports:
      - number: 3128
        name: tcp
        protocol: TCP
      location: MESH_EXTERNAL
    EOF
  27. Squid proxy를 통해서 외부 HTTPS 서비스 호출

    kubectl exec nginx -- \
    sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"
  28. Squid proxy를 통해서 외부 HTTP 서비스 호출

    kubectl exec nginx -- \
    sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"
  29. Squid proxy 로그 확인

    kubectl -n external exec squid -- tail /var/log/squid/access.log
  30. Sidecar Proxy 로그 확인

    kubectl logs nginx -c istio-proxy
  31. 리소스 삭제

    kubectl delete serviceentries.networking.istio.io external-proxy
    kubectl delete pod nginx
    kubectl delete namespace external
    kubectl label namespace default istio-injection-
    istioctl x uninstall -y --purge

Last updated