실습

Prerequisite

AWS Load Balancer Controller에 명시된 내용을 참고해서 AWS Load Balancer Controller 설치

Installation

https://istio.io/latest/docs/setup/getting-started/#download

Ingress Gateway

TLS

Istio 설치

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
        k8s:
          serviceAnnotations:
            service.beta.kubernetes.io/aws-load-balancer-type: external
            service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
            service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
            service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
          service:
            externalTrafficPolicy: Cluster
            loadBalancerSourceRanges:
              - 0.0.0.0/0
EOF

CA 인증서 생성

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
-subj '/CN=example.com' -keyout ca-key.pem -out ca-cert.pem

example.com 도메인에 인증서 서명 요청 및 키 생성

openssl req -newkey rsa:2048 -nodes \
-subj "/CN=*.example.com" -keyout example.com.key -out example.com.csr

인증서 서명

openssl x509 -req -sha256 -days 365 -CA ca-cert.pem -CAkey ca-key.pem \
-set_serial 0 -in example.com.csr -out example.com.crt

Secret 생성

kubectl create -n istio-system secret tls example-com \
--key=example.com.key --cert=example.com.crt

Gateway 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway 
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: example-com
    hosts:
    - "*"
EOF

VirtualService 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "nginx.example.com"
  gateways:
  - istio-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: nginx
        port:
          number: 80
EOF

데모 애플리케이션 배포

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
  selector:
    app: nginx
EOF

데모 애플리케이션 호출 - HTTPS

curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v \
https://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Gateway에 HTTPS Redirect 설정

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway 
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    tls:
      httpsRedirect: true
    hosts:
    - "*"
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: example-com
    hosts:
    - "*"
EOF

데모 애플리케이션 호출 - HTTP

curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v -L \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Gateway에 설정한 HTTPS Redirect 삭제

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway 
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: example-com
    hosts:
    - "*"
EOF

VirtualService HTTPS Redirect 설정

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "nginx.example.com"
  gateways:
  - istio-gateway
  http:
  - match:
    - scheme:
        exact: http
    redirect:
      scheme: https
      redirectCode: 308
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: nginx
        port:
          number: 80
EOF

데모 애플리케이션 호출 - HTTPS

curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v \
https://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

데모 애플리케이션 호출 - HTTP

curl -H "Host: nginx.example.com" --cacert ca-cert.pem --insecure -v -L \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

리소스 삭제

kubectl delete deploy nginx
kubectl delete svc nginx
kubectl delete virtualservices.networking.istio.io nginx 
kubectl delete gateways.networking.istio.io istio-gateway
kubectl delete -n istio-system secret example-com
istioctl x uninstall -y --purge

TLS with NLB

AWS Certificate Manager를 통해서 SSL 인증서 발급

사용할 ACM 인증서의 ARN을 환경변수로 지정

export SSL_CERT=<ACM_CERT_ARN>

e.g.

export SSL_CERT=arn:aws:acm:ap-northeast-2:111111111111:certificate/6a427716-ffde-4676-a4f1-02086398df7f

Istio 설치

cat <<EOF | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
        k8s:
          serviceAnnotations:
            service.beta.kubernetes.io/aws-load-balancer-type: external
            service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
            service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
            service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
            service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
            service.beta.kubernetes.io/aws-load-balancer-ssl-cert: $SSL_CERT
          service:
            externalTrafficPolicy: Cluster
            loadBalancerSourceRanges:
              - 0.0.0.0/0
          service:
            ports:
              - name: http2
                port: 80
                targetPort: 8080
              - name: status-port
                port: 15020
                targetPort: 15020
          overlays:
            - apiVersion: v1
              kind: Service
              name: istio-ingressgateway
              patches:
                - path: spec.ports[2]
                  value:
                    name: https
                    protocol: TCP
                    port: 443
                    targetPort: 8080
EOF

사용할 Top 레벨 도메인주소를 환경변수로 지정
```
export DOMAIN_NAME=<TLD>
```
e.g.
```
export DOMAIN_NAME=example.com
```

Gateway 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
EOF

VirtualService 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "nginx.$DOMAIN_NAME"
  gateways:
  - istio-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: nginx
        port:
          number: 80
EOF

데모 애플리케이션 배포

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
  selector:
    app: nginx
EOF

아래의 명령어를 실행해서 나온 결과값과 같은 DNS 레코드 생성

echo nginx.$DOMAIN_NAME=$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME
```
데모 애플리케이션 호출 - HTTP
```
curl http://nginx.$DOMAIN_NAME
```

Gateway에 HTTPS Redirect 설정

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
    tls:
      httpsRedirect: true
EOF

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME -v
```
데모 애플리케이션 호출 - HTTP
```
curl -L http://nginx.$DOMAIN_NAME -v
```
NLB와 Istio Ingress Gateway는 HTTP 프로토콜로 통신하므로 모든 요청이 HTTPS로 전환됨

Gateway에 설정한 HTTPS Redirect 삭제

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
EOF

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME -v
```
데모 애플리케이션 호출 - HTTP
```
curl -L http://nginx.$DOMAIN_NAME -v
```

VirtualService에 HTTPS Redirect 설정

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "nginx.$DOMAIN_NAME"
  gateways:
  - istio-gateway
  http:
  - match:
    - scheme:
        exact: http
    redirect:
      scheme: https
      redirectCode: 308
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: nginx
        port:
          number: 80
EOF

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME -v
```
데모 애플리케이션 호출 - HTTP
```
curl -L http://nginx.$DOMAIN_NAME -v
```

VirtualService 설정한 HTTPS Redirect 삭제

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "nginx.$DOMAIN_NAME"
  gateways:
  - istio-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: nginx
        port:
          number: 80
EOF

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME -v
```
데모 애플리케이션 호출 - HTTP
```
curl -L http://nginx.$DOMAIN_NAME -v
```

Istio 설정변경 - NLB(HTTP) -> Istio Ingress Gateway(HTTP), NLB(HTTPS) -> Istio Ingress Gateway(HTTPS)로 라우팅 되도록 구성

cat <<EOF | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
        k8s:
          serviceAnnotations:
            service.beta.kubernetes.io/aws-load-balancer-type: external
            service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
            service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
            service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
            service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
            service.beta.kubernetes.io/aws-load-balancer-ssl-cert: $SSL_CERT
          service:
            externalTrafficPolicy: Cluster
            loadBalancerSourceRanges:
              - 0.0.0.0/0
EOF

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME -v
```
데모 애플리케이션 호출 - HTTP
```
curl -L http://nginx.$DOMAIN_NAME -v
```

Gateway에 443 포트 리스너 추가

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
  - port:
      number: 443
      name: https
      protocol: HTTP
    hosts:
    - "*"
EOF

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME -v
```

Gateway에 HTTPS Redirect 설정

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
    tls:
      httpsRedirect: true
  - port:
      number: 443
      name: https
      protocol: HTTP
    hosts:
    - "*"
EOF

데모 애플리케이션 호출 - HTTPS
```
curl https://nginx.$DOMAIN_NAME -v
```
데모 애플리케이션 호출 - HTTP
```
curl -L http://nginx.$DOMAIN_NAME -v
```

데모 애플리케이션 배포 - https://github.com/youngwjung/flask-echo

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: api
  name: api
spec:
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: youngwjung/flask-echo
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: api
  labels:
    app: api
spec:
  ports:
  - port: 80
  selector:
    app: api
EOF

VirtualService 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: api
spec:
  hosts:
  - "api.$DOMAIN_NAME"
  gateways:
  - istio-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: api
        port:
          number: 80
EOF

아래의 명령어를 실행해서 나온 결과값과 같은 DNS 레코드 생성

echo api.$DOMAIN_NAME=$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

데모 애플리케이션 호출 - HTTPS

curl -v -X POST https://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'

데모 애플리케이션 호출 - HTTP

curl -L -v -X POST http://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'

Gateway에 설정한 HTTPS Redirect 삭제

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
  - port:
      number: 443
      name: https
      protocol: HTTP
    hosts:
    - "*"
EOF

데모 애플리케이션 호출 - HTTP

curl -v -X POST http://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'

VirtualService에 HTTPS Redirect 설정

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: api
spec:
  hosts:
  - "api.$DOMAIN_NAME"
  gateways:
  - istio-gateway
  http:
  - match:
    - port: 80
    redirect:
      scheme: https
      redirectCode: 308
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: api
        port:
          number: 80
EOF

데모 애플리케이션 호출 - HTTPS

curl -v -X POST https://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'

데모 애플리케이션 호출 - HTTP

curl -v -L -X POST http://api.$DOMAIN_NAME/echo -H 'Content-Type: application/json' -d '{"key": "value"}'

리소스 삭제

kubectl delete deploy nginx api
kubectl delete svc nginx api
kubectl delete virtualservices.networking.istio.io nginx api
kubectl delete gateways.networking.istio.io istio-gateway
istioctl x uninstall -y --purge

Authorization

Istio 설치

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        serviceAnnotations:
          service.beta.kubernetes.io/aws-load-balancer-type: external
          service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
          service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
    pilot:
      k8s:
        resources:
          requests:
            cpu: 200m
            memory: 500Mi
EOF

Gateway 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
EOF

데모 애플리케이션 배포

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
  selector:
    app: nginx
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - nginx.example.com
  gateways:
  - istio-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: nginx
        port:
          number: 80
EOF

데모 애플리케이션 호출

curl -H "Host: nginx.example.com" \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

내 컴퓨터(혹은 현재 접속된 Cloud9 인스턴스) IP 주소 확인
```
curl -s ifconfig.io
```

내 컴퓨터에서 Ingress Gateway로 접근 차단

cat <<EOF | kubectl apply -f -
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ingress-policy
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: DENY
  rules:
  - from:
    - source:
        ipBlocks: 
        - $(curl -s ifconfig.io)
EOF

데모 애플리케이션 호출

curl -H "Host: nginx.example.com" \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Ingress Gateway 로그 확인

kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20

Envoy 로그 활성화

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    accessLogFile: /dev/stdout
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        serviceAnnotations:
          service.beta.kubernetes.io/aws-load-balancer-type: external
          service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
          service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
    pilot:
      k8s:
        resources:
          requests:
            cpu: 200m
            memory: 500Mi
EOF

Ingress Gateway 재생성

kubectl -n istio-system delete pod -l app=istio-ingressgateway

데모 애플리케이션 호출

curl -H "Host: nginx.example.com" \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Ingress Gateway 로그 확인

kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20

NLB 설정 변경

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    accessLogFile: /dev/stdout
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        serviceAnnotations:
          service.beta.kubernetes.io/aws-load-balancer-type: external
          service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
          service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    pilot:
      k8s:
        resources:
          requests:
            cpu: 200m
            memory: 500Mi
EOF

데모 애플리케이션 호출

curl -H "Host: nginx.example.com" \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Ingress Gateway 로그 확인

kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20

NLB 설정 변경

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    accessLogFile: /dev/stdout
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        serviceAnnotations:
          service.beta.kubernetes.io/aws-load-balancer-type: external
          service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
          service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
          service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    pilot:
      k8s:
        resources:
          requests:
            cpu: 200m
            memory: 500Mi
EOF

Envoy Filter 생성 - Proxy Protocol

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: proxy-protocol
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: LISTENER
    patch:
      operation: MERGE
      value:
        listener_filters:
        - name: envoy.filters.listener.proxy_protocol
        - name: envoy.filters.listener.tls_inspector
EOF

Envoy Filter 생성 - X-Forwarded-For 헤더

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: ingressgateway-settings
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
    patch:
      operation: MERGE
      value:
        name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          skip_xff_append: false
          use_remote_address: true
          xff_num_trusted_hops: 1
EOF

데모 애플리케이션 호출

curl -H "Host: nginx.example.com" \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Ingress Gateway 로그 확인

kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20

AuthorizationPolicy 수정 - https://istio.io/latest/docs/tasks/security/authorization/authz-ingress/#ip-based-allow-list-and-deny-list

cat <<EOF | kubectl apply -f -
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ingress-policy
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: DENY
  rules:
  - from:
    - source:
        remoteIpBlocks: 
        - $(curl -s ifconfig.io)
EOF

데모 애플리케이션 호출

curl -H "Host: nginx.example.com" \
http://$(kubectl -n istio-system get svc istio-ingressgateway -o=jsonpath='{.status.loadBalancer.ingress[0].hostname}')

Ingress Gateway 로그 확인

kubectl -n istio-system logs deploy/istio-ingressgateway --tail 20

리소스 삭제

{
    kubectl delete deploy nginx
    kubectl delete svc nginx
    istioctl x uninstall -y --purge
}

Service Entry

Introduction

Istio 설치

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: false
EOF

default 네임스페이스에 생성되는 Pod에 프록시 주입 설정
```
kubectl label namespace default istio-injection=enabled --overwrite
```
Pod 생성
```
kubectl run nginx --image=nginx
```
Pod가 생성되고 프록시 컨테이너가 추가되었는지 확인
```
kubectl get pod nginx
```
생성한 Pod에서 httpbin.org 접근 시도
```
kubectl exec nginx -- curl -si httpbin.org/json
```

Istio 아웃바운드 트래픽 정책 변경 - https://istio.io/latest/docs/reference/config/istio.mesh.v1alpha1/#MeshConfig-OutboundTrafficPolicy-Mode

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    outboundTrafficPolicy:
      mode: REGISTRY_ONLY
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: false
EOF

Pod에서 httpbin.org 접근 시도

kubectl exec nginx -- curl -si httpbin.org/json

ServiceEntry 생성 - https://istio.io/latest/docs/reference/config/networking/service-entry/#ServiceEntry

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: httpbin-org
spec:
  hosts:
  - httpbin.org
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: DNS
  location: MESH_EXTERNAL
EOF

Pod에서 httpbin.org 접근 시도

kubectl exec nginx -- curl -si httpbin.org/json

리소스 삭제

kubectl delete serviceentries.networking.istio.io httpbin-org
kubectl delete pod nginx
kubectl label namespace default istio-injection-
istioctl x uninstall -y --purge

Deep Dive

Istio 설치

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: false
EOF

default 네임스페이스에 생성되는 Pod에 프록시 주입 설정
```
kubectl label namespace default istio-injection=enabled --overwrite
```
Pod 생성
```
kubectl run nginx --image=nginx
```
Pod가 생성되고 프록시 컨테이너가 추가되었는지 확인
```
kubectl get pod nginx
```
Sidecar Proxy의 리스너 설정 확인 - https://istio.io/latest/docs/ops/diagnostic-tools/proxy-cmd/#deep-dive-into-envoy-configuration
```
istioctl proxy-config listeners nginx
```
Istio Sidecar Proxy가 사용하는 포트 확인 - https://istio.io/latest/docs/ops/deployment/requirements/#ports-used-by-istio
Sidecar Proxy의 아웃바운드 설정 확인 - https://faun.pub/understanding-how-envoy-sidecar-intercept-and-route-traffic-in-istio-service-mesh-20fea2a78833
```
istioctl proxy-config listeners nginx --port 15001 -o json
```
요약: 목적지 포트가 15001일 경우에는 BlackHoleCluster로 아닐 경우에는 PassthroughCluster로
Sidecar Proxy의 클러스터 설정 확인
```
istioctl proxy-config cluster nginx
```
PassthroughCluster의 상세 설정 확인 - InboundPassthroughClusterIpv4 및 PassthroughCluster에 대한 정보가 보여지므로 name 값을 확인해서 PassthroughCluster에 대한 정보만 확인
```
istioctl proxy-config cluster nginx --fqdn PassthroughCluster -o json
```

BlackHoleCluster의 상세 설정 확인

istioctl proxy-config cluster nginx --fqdn BlackHoleCluster -o json

생성한 Pod에서 httpbin.org 접근 시도
```
kubectl exec nginx -- curl -si httpbin.org/json
```

Istio 아웃바운드 트래픽 정책 변경

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    outboundTrafficPolicy:
      mode: REGISTRY_ONLY
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: false
EOF

Pod에서 httpbin.org 접근 시도

kubectl exec nginx -- curl -si http://httpbin.org/json

Sidecar Proxy의 리스너 설정 확인
```
istioctl proxy-config listeners nginx
```
Sidecar Proxy의 아웃바운드 설정 확인
```
istioctl proxy-config listeners nginx --port 15001 -o json
```
요약: 목적지 포트가 15001일 경우에는 BlackHoleCluster로 아닐 경우에도 BlackHoleCluster로
BlackHoleCluster의 동작방식 확인 - https://istio.io/latest/blog/2019/monitoring-external-service-traffic/#what-are-blackhole-and-passthrough-clusters

ServiceEntry 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: httpbin-org
spec:
  hosts:
  - httpbin.org
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: DNS
  location: MESH_EXTERNAL
EOF

Pod에서 httpbin.org 접근 시도

kubectl exec nginx -- curl -si httpbin.org/json

Sidecar Proxy의 Route 설정 확인
```
istioctl proxy-config route nginx
```
Sidecar Proxy의 Route 설정 확인 - 80/httpbin.org
```
istioctl proxy-config route nginx --name 80 -o json
```
요약: 포트 80으로 나갈때 호스트가 httpbin.org을 경우 httpbin.org 클러스터로 아닐 경우 502 반환

Pod에서 google.com 접근 시도

kubectl exec nginx -- curl -si google.com

Sidecar Proxy의 클러스터 설정 확인
```
istioctl proxy-config cluster nginx
```

httpbin.org 의 상세 설정 확인

istioctl proxy-config cluster nginx --fqdn httpbin.org -o json

리소스 삭제

kubectl delete serviceentries.networking.istio.io httpbin-org
kubectl delete pod nginx
kubectl label namespace default istio-injection-
istioctl x uninstall -y --purge

External HTTPS Proxy

Istio 설치

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    accessLogFile: /dev/stdout
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: false
EOF

default 네임스페이스에 생성되는 Pod에 프록시 주입 설정
```
kubectl label namespace default istio-injection=enabled --overwrite
```
HTTPS Proxy를 배포할 Namespace 생성
```
kubectl create namespace external
```

Squid proxy 설정파일 생성

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: squid-proxy-conf
  namespace: external
data:
  squid.conf: |
    http_port 3128

    acl SSL_ports port 443
    acl CONNECT method CONNECT
    
    http_access deny CONNECT !SSL_ports
    http_access allow localhost manager
    http_access deny manager
    http_access allow all
    
    coredump_dir /var/spool/squid
EOF

Squid proxy 생성

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: squid
  namespace: external
spec:
  volumes:
  - name: proxy-config
    configMap:
      name: squid-proxy-conf
  containers:
  - name: squid
    image: datadog/squid
    ports:
    - containerPort: 3128
    volumeMounts:
    - name: proxy-config
      mountPath: /etc/squid
EOF

Squid proxy 생성 확인
```
kubectl -n external get pod
```

Sidecar Proxy가 없는 Pod 생성

kubectl -n external run nginx --image=nginx

Squid proxy를 통해서 외부 HTTPS 서비스 호출

kubectl -n external exec nginx -- \
sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"

Squid proxy를 통해서 외부 HTTP 서비스 호출

kubectl -n external exec nginx -- \
sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"

Squid proxy 로그 확인

kubectl -n external exec squid -- tail /var/log/squid/access.log

Sidecar Proxy를 포함하는 Pod 생성
```
kubectl run nginx --image=nginx
```

Squid proxy를 통해서 외부 HTTPS 서비스 호출

kubectl exec nginx -- \
sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"

Squid proxy를 통해서 외부 HTTP 서비스 호출

kubectl exec nginx -- \
sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"

Squid proxy 로그 확인

kubectl -n external exec squid -- tail /var/log/squid/access.log

Sidecar Proxy 로그 확인
```
kubectl logs nginx -c istio-proxy
```

Istio 아웃바운드 트래픽 정책 변경

cat <<'EOF' | istioctl install -y -f -
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    accessLogFile: /dev/stdout
    outboundTrafficPolicy:
      mode: REGISTRY_ONLY
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: false
EOF

Squid proxy를 통해서 외부 HTTPS 서비스 호출

kubectl exec nginx -- \
sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"

Squid proxy를 통해서 외부 HTTP 서비스 호출

kubectl exec nginx -- \
sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"

Squid proxy 로그 확인

kubectl -n external exec squid -- tail /var/log/squid/access.log

Sidecar Proxy 로그 확인
```
kubectl logs nginx -c istio-proxy
```

ServiceEntry 생성

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-proxy
spec:
  hosts:
  - squid
  addresses:
  - $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}')/32
  ports:
  - number: 3128
    name: http
    protocol: HTTP
  location: MESH_EXTERNAL
EOF

Squid proxy를 통해서 외부 HTTPS 서비스 호출

kubectl exec nginx -- \
sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"

Squid proxy를 통해서 외부 HTTP 서비스 호출

kubectl exec nginx -- \
sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"

Squid proxy 로그 확인

kubectl -n external exec squid -- tail /var/log/squid/access.log

Sidecar Proxy 로그 확인
```
kubectl logs nginx -c istio-proxy
```
에러 로그 내용 리뷰

ServiceEntry 변경 - TCP 프로토콜 사용

cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: external-proxy
spec:
  hosts:
  - squid
  addresses:
  - $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}')/32
  ports:
  - number: 3128
    name: tcp
    protocol: TCP
  location: MESH_EXTERNAL
EOF

Squid proxy를 통해서 외부 HTTPS 서비스 호출

kubectl exec nginx -- \
sh -c "HTTPS_PROXY=$(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 curl -si https://httpbin.org/json"

Squid proxy를 통해서 외부 HTTP 서비스 호출

kubectl exec nginx -- \
sh -c "curl -x $(kubectl get pod -n external squid -o=jsonpath='{.status.podIP}'):3128 -si http://httpbin.org/json"

Squid proxy 로그 확인

kubectl -n external exec squid -- tail /var/log/squid/access.log

Sidecar Proxy 로그 확인
```
kubectl logs nginx -c istio-proxy
```

리소스 삭제

kubectl delete serviceentries.networking.istio.io external-proxy
kubectl delete pod nginx
kubectl delete namespace external
kubectl label namespace default istio-injection-
istioctl x uninstall -y --purge

PreviousIstio (WIP)NextEnvoy (WIP)

Last updated 3 years ago

hashtagPrerequisite

hashtagInstallation

hashtagIngress Gateway

hashtagTLS

hashtagTLS with NLB

hashtagAuthorization

hashtagService Entry

hashtagIntroduction

hashtagDeep Dive

hashtagExternal HTTPS Proxy

Prerequisite

Installation

Ingress Gateway

TLS

TLS with NLB

Authorization

Service Entry

Introduction

Deep Dive

External HTTPS Proxy