从零搭建基于Istio的ServiceMesh-02Ingress&Egress

发表于 | 更新于: | 分类于 ， | 阅读次数:
字数统计: 4.4k

说明

  • istio版本号1.4.2
  • k8s集群版本v1.14.8
  • istio在1.4提供了基于istioctl命令直接部署的功能，这里使用istioctl部署istio。
    • 自带配置校验、和丰富的自定义配置选项
    • API版本号还在Alpha阶段，install.istio.io/v1alpha2，请自行判断是否适用
    • 部署要求
      • 至少得有Kubernetes集群
      • Istio-1.4版本在1.13、1.14、1.15的k8s集群上是做过测试通过的
      • 最新的1.16没在官方文档里注明，应该也是可以用的。官方说明在此
  • 这里通过官方示例熟悉一下istio的ServiceMesh特性

Ingress

说明

  • 在Kubernetes环境中，Kubernetes的Ingress资源用于配置集群外部访问集群内部的服务。

  • 在Istio中，使用Istio Gateway进行替代。

Istio通过Gateway实现对进入集群的流量进行路由和监控。

环境准备

  • 部署httpbin
1
kubectl apply -f samples/httpbin/httpbin.yaml

确定Ingress的IP地址端口

1
kubectl get svc istio-ingressgateway -n istio-system

输出示例，家境贫寒，这里用的是NodePort

1
2
NAME                   TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)                                                                                                                                      AGE
istio-ingressgateway   NodePort   10.96.33.206   <none>        15020:31226/TCP,80:31355/TCP,443:30885/TCP,15029:31094/TCP,15030:31128/TCP,15031:30163/TCP,31400:31400/TCP,15032:31966/TCP,15443:30659/TCP   7d3h

LoadBalancer

对于使用LoadBalancer的Service，可以通过EXTERNAL-IP加端口的形式访问Istio-Gateway

1
2
3
export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')

NodePort

我的实验环境没有LoadBalancer，因此用了NodePort

1
2
3
export INGRESS_HOST=$(kubectl get po -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].status.hostIP}')
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].nodePort}')

基础的Ingress

使用Gateway配置ingress

与Kubernetes自带的Ingress对象不同

Istio Gateway会使用istio自己的路由规则来转发流量，转发方式跟集群内部路由方式相同。

创建Gateway

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "httpbin.example.com"
EOF

创建VirtualService

指定了httpbin-gateway作为Istio Gateway

只允许/status和/delay两个请求，其他外部请求会404

主机名指定httpbin.example.com

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.com"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

集群外部访问

命令行

  • 正确访问方式
1
curl -I -H 'Host:httpbin.example.com' http://$INGRESS_HOST:$INGRESS_PORT/status/200

输出示例

1
2
3
4
5
6
7
8
HTTP/1.1 200 OK
server: istio-envoy
date: Mon, 30 Dec 2019 06:21:10 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 3
  • 错误访问方式
1
2
3
4
# 不带Host字段或者Host字段不匹配
curl -I  http://$INGRESS_HOST:$INGRESS_PORT/status/200
# 访问其他URL
curl -I -H 'Host:httpbin.example.com' http://$INGRESS_HOST:$INGRESS_PORT/index.html

输出示例

1
2
3
4
HTTP/1.1 404 Not Found
date: Mon, 30 Dec 2019 06:21:46 GMT
server: istio-envoy
transfer-encoding: chunked

浏览器

修改hosts文件

添加以下内容，注意替换Ingress_HOST的值

1
<Ingress_HOST> httpbin.example.com
打开网页

注意替换Ingress_Port的值

http://httpbin.example.com:<Ingress_PORT>/status/200

清理环境

按需清理，后面做HTTPS还要用到httpbin

1
2
3
kubectl delete gateway httpbin-gateway
kubectl delete virtualservice httpbin
kubectl delete --ignore-not-found=true -f samples/httpbin/httpbin.yaml

基于SSL/TLS的Ingress (File Mount文件挂载)

说明

  • Istio-Gateway同样可以处理HTTPS请求
  • TLS类型的Secret会被挂载到/etc/istio/ingressgateway-certs
  • Secret对象的名字必须是istio-ingressgateway-certs并且命名空间是在istio-system，否则istio-Gateway无法识别

创建TLS类型的Secret

已有证书

如果购买了公网的可信任证书，可以跳过自建证书的步骤。

自建证书

创建CA证书
1
2
3
4
5
6
7
8
openssl req -x509 \
            -sha256 \
            -nodes \
            -days 365 \
            -newkey rsa:2048 \
            -subj '/O=example Inc./CN=example.com' \
            -keyout example.com.key \
            -out example.com.crt
创建证书
1
2
3
4
5
6
7
8
9
10
11
12
13
openssl req -out \
            httpbin.example.com.csr \
            -newkey rsa:2048 \
            -nodes \
            -keyout httpbin.example.com.key \
            -subj "/CN=httpbin.example.com/O=httpbin organization"
openssl x509 -req \
             -days 365 \
             -CA example.com.crt \
             -CAkey example.com.key \
             -set_serial 0 \
             -in httpbin.example.com.csr \
             -out httpbin.example.com.crt

创建Secret

1
2
3
4
kubectl -n istio-system create secret tls \
        istio-ingressgateway-certs \
        --key httpbin.example.com.key \
        --cert httpbin.example.com.crt

检查证书是否挂载

1
kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/istio/ingressgateway-certs

输出示例

1
2
3
4
5
6
7
total 0
drwxrwxrwt 3 root root 120 Dec 30 06:54 .
drwxr-xr-x 1 root root  78 Dec 23 02:51 ..
drwxr-xr-x 2 root root  80 Dec 30 06:54 ..2019_12_30_06_54_28.616938615
lrwxrwxrwx 1 root root  31 Dec 30 06:54 ..data -> ..2019_12_30_06_54_28.616938615
lrwxrwxrwx 1 root root  14 Dec 30 06:54 tls.crt -> ..data/tls.crt
lrwxrwxrwx 1 root root  14 Dec 30 06:54 tls.key -> ..data/tls.key

使用Gateway配置Ingress

创建Gateway

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "httpbin.example.com"
EOF

创建VirtualService

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.com"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

访问HTTPS服务

1
2
3
4
5
6
curl -v \
     -k \
     -H 'Host:httpbin.example.com' \
     --resolve httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST \
     --cacert example.com.crt \
     https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418

输出示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
* Added httpbin.example.com:30885:192.168.48.37 to DNS cache
* About to connect() to httpbin.example.com port 30885 (#0)
*   Trying 192.168.48.37...
* Connected to httpbin.example.com (192.168.48.37) port 30885 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: O=httpbin organization,CN=httpbin.example.com
* 	start date: Dec 30 06:53:12 2019 GMT
* 	expire date: Dec 29 06:53:12 2020 GMT
* 	common name: httpbin.example.com
* 	issuer: CN=example.com,O=example Inc.
> GET /status/418 HTTP/1.1
> User-Agent: curl/7.29.0
> Accept: */*
> Host:httpbin.example.com
> 
< HTTP/1.1 418 Unknown
< server: istio-envoy
< date: Mon, 30 Dec 2019 07:09:01 GMT
< x-more-info: http://tools.ietf.org/html/rfc2324
< access-control-allow-origin: *
< access-control-allow-credentials: true
< content-length: 135
< x-envoy-upstream-service-time: 1
< 

    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`
* Connection #0 to host httpbin.example.com left intact

配置多Host的TLS Ingress

创建TLS类型的Secret

这里以bookinfo.com作为示例

创建证书
1
2
3
4
5
6
7
8
9
10
11
12
13
openssl req \
        -out bookinfo.com.csr \
        -newkey rsa:2048 \
        -nodes \
        -keyout bookinfo.com.key \
        -subj "/CN=bookinfo.com/O=bookinfo organization"
openssl x509 -req \
        -days 365 \
        -CA example.com.crt \
        -CAkey example.com.key \
        -set_serial 0 \
        -in bookinfo.com.csr \
        -out bookinfo.com.crt
创建Secret
1
2
3
4
kubectl -n istio-system create secret tls \
        istio-ingressgateway-bookinfo-certs \
        --key bookinfo.com.key \
        --cert bookinfo.com.crt

更新istio-ingressgateway

这里需要重新配置istio-ingressgateway的deployment以挂载新的证书

创建Patch file
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
cat > gateway-patch.json <<EOF
[{
  "op": "add",
  "path": "/spec/template/spec/containers/0/volumeMounts/0",
  "value": {
    "mountPath": "/etc/istio/ingressgateway-bookinfo-certs",
    "name": "ingressgateway-bookinfo-certs",
    "readOnly": true
  }
},
{
  "op": "add",
  "path": "/spec/template/spec/volumes/0",
  "value": {
  "name": "ingressgateway-bookinfo-certs",
    "secret": {
      "secretName": "istio-ingressgateway-bookinfo-certs",
      "optional": true
    }
  }
}]
EOF
部署Patch
1
kubectl -n istio-system patch --type=json deploy istio-ingressgateway -p "$(cat gateway-patch.json)"
检查证书挂载
1
kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/istio/ingressgateway-bookinfo-certs

输出示例

1
2
3
4
5
6
7
total 0
drwxrwxrwt 3 root root 120 Dec 30 07:18 .
drwxr-xr-x 1 root root 115 Dec 30 07:18 ..
drwxr-xr-x 2 root root  80 Dec 30 07:18 ..2019_12_30_07_18_56.019679131
lrwxrwxrwx 1 root root  31 Dec 30 07:18 ..data -> ..2019_12_30_07_18_56.019679131
lrwxrwxrwx 1 root root  14 Dec 30 07:18 tls.crt -> ..data/tls.crt
lrwxrwxrwx 1 root root  14 Dec 30 07:18 tls.key -> ..data/tls.key

配置bookinfo.com路由

部署bookinfo
1
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
定义Gateway
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https-bookinfo
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-bookinfo-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-bookinfo-certs/tls.key
    hosts:
    - "bookinfo.com"
EOF
定义VirtualService
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "bookinfo.com"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080
EOF

访问服务

bookinfo
1
2
3
4
5
6
7
8
curl -o /dev/null \
     -s \
     -v \
     -w "%{http_code}\n" \
     -H 'Host:bookinfo.com' \
     --resolve bookinfo.com:$SECURE_INGRESS_PORT:$INGRESS_HOST \
     --cacert example.com.crt \
     https://bookinfo.com:$SECURE_INGRESS_PORT/productpage

输出示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
* Added bookinfo.com:30885:192.168.48.37 to DNS cache
* About to connect() to bookinfo.com port 30885 (#0)
*   Trying 192.168.48.37...
* Connected to bookinfo.com (192.168.48.37) port 30885 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: example.com.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: O=bookinfo organization,CN=bookinfo.com
* 	start date: Dec 30 07:13:53 2019 GMT
* 	expire date: Dec 29 07:13:53 2020 GMT
* 	common name: bookinfo.com
* 	issuer: CN=example.com,O=example Inc.
> GET /productpage HTTP/1.1
> User-Agent: curl/7.29.0
> Accept: */*
> Host:bookinfo.com
> 
< HTTP/1.1 200 OK
< content-type: text/html; charset=utf-8
< content-length: 3889
< server: istio-envoy
< date: Mon, 30 Dec 2019 07:25:10 GMT
< x-envoy-upstream-service-time: 1015
< 
{ [data not shown]
* Connection #0 to host bookinfo.com left intact
200
httpbin

httpbin服务依然能正常访问

1
2
3
4
5
6
curl -H 'Host:httpbin.example.com' \
     --resolve httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST \
     --cacert example.com.crt \
     --cert httpbin-client.example.com.crt \
     --key httpbin-client.example.com.key \
     https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418

输出示例

1
2
3
4
5
6
7
8
9
-=[ teapot ]=-

   _...._
 .'  _ _ `.
| ."` ^ `". _,
\_;`"---"`|//
  |       ;/
  \_     _/
    `"""`

清理现场

说明

请按需清理！后面做SSL/TLS实验还会用到！

清理istio资源对象
1
2
3
kubectl delete gateway --ignore-not-found=true httpbin-gateway bookinfo-gateway
kubectl delete virtualservice httpbin
kubectl delete --ignore-not-found=true virtualservice bookinfo
清理证书
1
2
3
rm -rf example.com.crt example.com.key
rm -rf bookinfo.com.crt bookinfo.com.csr bookinfo.com.key
rm -rf httpbin.example.com.crt httpbin.example.com.csr httpbin.example.com.key
清理Patch文件
1
rm -rf gateway-patch.json
清理httpbin服务
1
kubectl delete --ignore-not-found=true -f samples/httpbin/httpbin.yaml

基于SSL/TLS的Ingress(SDS密钥发现服务)

说明

  • 通过Secret Discovery Service可以监视TLS Credential的生成

  • IngressGateway可以动态增删改查证书，而不是通过Patch的方式重新部署IngressGateway

  • 不需要配置IngressGateway挂载Secret卷

  • IngressGateway可以监控多个密钥对，因此只需要更新Gateway定义即可

  • Demo默认不开启SDS功能，需要通过istioctl重新配置istio，配置如下

    1
    2
    --set values.gateways.istio-egressgateway.enabled=false
    --set values.gateways.istio-ingressgateway.sds.enabled=true

环境准备

Istio特性部署要求

  • 启用SDS功能

Istio部署参数

1
2
3
istioctl manifest apply \
         --set values.gateways.istio-egressgateway.enabled=false \
         --set values.gateways.istio-ingressgateway.sds.enabled=true

为了方便测试，这里额外添加了部署参数

1
2
3
4
5
6
7
8
9
10
istioctl manifest apply \
    --set profile='demo' \
    --set cni.enabled=true \
    --set cni.components.cni.namespace='kube-system' \
    --set hub='dockerhub.azk8s.cn/istio' \
    --set values.gateways.istio-ingressgateway.type='NodePort' \
    --set values.gateways.istio-egressgateway.enabled=false \
    --set values.gateways.istio-ingressgateway.sds.enabled=true \
    --set values.global.controlPlaneSecurityEnabled=true \
    --set values.global.mtls.enabled=false

获取Ingress环境变量

1
2
3
export INGRESS_HOST=$(kubectl get po -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].status.hostIP}')
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].nodePort}')

部署httpbin

1
kubectl apply -f samples/httpbin/httpbin.yaml

创建Secret

说明

  • Secret对象的名字不能以istio或者prometheus开头
  • Secret对象不能包含token字段

已有证书

如果购买了公网的可信任证书，可以跳过自建证书的步骤。

自建证书

创建CA证书
1
2
3
4
5
6
7
8
openssl req -x509 \
            -sha256 \
            -nodes \
            -days 365 \
            -newkey rsa:2048 \
            -subj '/O=example Inc./CN=example.com' \
            -keyout example.com.key \
            -out example.com.crt
创建证书
1
2
3
4
5
6
7
8
9
10
11
12
13
openssl req -out \
            httpbin.example.com.csr \
            -newkey rsa:2048 \
            -nodes \
            -keyout httpbin.example.com.key \
            -subj "/CN=httpbin.example.com/O=httpbin organization"
openssl x509 -req \
             -days 365 \
             -CA example.com.crt \
             -CAkey example.com.key \
             -set_serial 0 \
             -in httpbin.example.com.csr \
             -out httpbin.example.com.crt
创建Secret
1
2
3
4
5
6
7
kubectl -n istio-system \
        create \
        secret \
        generic \
        httpbin-credential \
        --from-file=key=httpbin.example.com.key \
        --from-file=cert=httpbin.example.com.crt

使用Gateway配置Ingress

创建Gateway

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: sds-gateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: "httpbin-credential" # must be the same as secret
    hosts:
    - "httpbin.example.com"
EOF

创建VirtualService

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.com"
  gateways:
  - sds-gateway
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

访问HTTPS服务

1
2
3
4
5
curl -v \
     -H 'Host:httpbin.example.com' \
     --resolve httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST \
     --cacert example.com.crt \
     https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418

输出示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
* Added httpbin.example.com:31944:192.168.48.37 to DNS cache
* About to connect() to httpbin.example.com port 31944 (#0)
*   Trying 192.168.48.37...
* Connected to httpbin.example.com (192.168.48.37) port 31944 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: example.com.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: O=httpbin organization,CN=httpbin.example.com
* 	start date: Dec 30 06:53:12 2019 GMT
* 	expire date: Dec 29 06:53:12 2020 GMT
* 	common name: httpbin.example.com
* 	issuer: CN=example.com,O=example Inc.
> GET /status/418 HTTP/1.1
> User-Agent: curl/7.29.0
> Accept: */*
> Host:httpbin.example.com
> 
< HTTP/1.1 418 Unknown
< server: istio-envoy
< date: Mon, 30 Dec 2019 08:53:33 GMT
< x-more-info: http://tools.ietf.org/html/rfc2324
< access-control-allow-origin: *
< access-control-allow-credentials: true
< content-length: 135
< x-envoy-upstream-service-time: 1
< 

    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`

配置多Host的TLS Ingress

创建TLS类型的Secret

这里以bookinfo.com作为示例

创建证书
1
2
3
4
5
6
7
8
9
10
11
12
13
openssl req \
        -out bookinfo.com.csr \
        -newkey rsa:2048 \
        -nodes \
        -keyout bookinfo.com.key \
        -subj "/CN=bookinfo.com/O=bookinfo organization"
openssl x509 -req \
        -days 365 \
        -CA example.com.crt \
        -CAkey example.com.key \
        -set_serial 0 \
        -in bookinfo.com.csr \
        -out bookinfo.com.crt
创建Secret
1
2
3
4
5
6
7
kubectl -n istio-system \
        create \
        secret \
        generic \
        bookinfo-credential \
        --from-file=key=bookinfo.com.key \
        --from-file=cert=bookinfo.com.crt

配置Gateway

添加bookinfo的host

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: sds-gateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https-httpbin
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: "httpbin-credential"
    hosts:
    - "httpbin.example.com"
  - port:
      number: 443
      name: https-bookinfo
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: "bookinfo-credential"
    hosts:
    - "bookinfo.com"
EOF

配置VirtualService

配置bookinfo的VirtualService

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "bookinfo.com"
  gateways:
  - sds-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080
EOF

访问HTTPS服务

bookinfo
1
2
3
4
5
6
7
8
curl -o /dev/null \
     -s \
     -v \
     -w "%{http_code}\n" \
     -H 'Host:bookinfo.com' \
     --resolve bookinfo.com:$SECURE_INGRESS_PORT:$INGRESS_HOST \
     --cacert example.com.crt \
     https://bookinfo.com:$SECURE_INGRESS_PORT/productpage
httpbin
1
2
3
4
5
6
curl -H 'Host:httpbin.example.com' \
     --resolve httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST \
     --cacert example.com.crt \
     --cert httpbin-client.example.com.crt \
     --key httpbin-client.example.com.key \
     https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418

清理现场

清理istio资源对象

1
2
3
4
5
kubectl delete sds-gateway mygateway
kubectl delete virtualservice httpbin bookinfo
kubectl delete --ignore-not-found=true -n istio-system secret httpbin-credential \
bookinfo-credential
kubectl delete --ignore-not-found=true virtualservice bookinfo httpbin

清理证书

1
2
rm -rf httpbin.example.com.crt httpbin.example.com.csr httpbin.example.com.key
rm -rf bookinfo.com.crt bookinfo.com.csr bookinfo.com.key

清理Kubernetes资源对象

1
2
kubectl delete service --ignore-not-found=true bookinfo
kubectl delete service --ignore-not-found=true httpbin

Ingress Gateway without TLS Termination

说明

  • 这个TLS Termination不知道怎么翻译，简单的说就是Gateway不处理TLS/HTTPS，直接透传后端的TLS
  • 这里简单部署一个提供HTTPS的Nginx服务作为实验样例

环境准备

创建证书

沿用上面的证书生成方式

1
2
3
4
5
6
7
8
9
10
11
12
13
openssl req \
        -out nginx.example.com.csr \
        -newkey rsa:2048 \
        -nodes \
        -keyout nginx.example.com.key \
        -subj "/CN=nginx.example.com/O=Nginx organization"
openssl x509 -req \
        -days 365 \
        -CA example.com.crt \
        -CAkey example.com.key \
        -set_serial 0 \
        -in nginx.example.com.csr \
        -out nginx.example.com.crt

创建Secret

1
2
3
4
5
6
kubectl create \
        secret \
        tls \
        nginx-server-certs \
        --key nginx.example.com.key \
        --cert nginx.example.com.crt

创建Nginx配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
events {
}

http {
  log_format main '$remote_addr - $remote_user [$time_local]  $status '
  '"$request" $body_bytes_sent "$http_referer" '
  '"$http_user_agent" "$http_x_forwarded_for"';
  access_log /var/log/nginx/access.log main;
  error_log  /var/log/nginx/error.log;

  server {
    listen 443 ssl;

    root /usr/share/nginx/html;
    index index.html;

    server_name nginx.example.com;
    ssl_certificate /etc/nginx-server-certs/tls.crt;
    ssl_certificate_key /etc/nginx-server-certs/tls.key;
  }
}

创建ConfigMap

1
kubectl create configmap nginx-configmap --from-file=nginx.conf=./nginx.conf

部署Nginx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
cat <<EOF | istioctl kube-inject -f - | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  labels:
    run: my-nginx
spec:
  ports:
  - port: 443
    protocol: TCP
  selector:
    run: my-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 443
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx
          readOnly: true
        - name: nginx-server-certs
          mountPath: /etc/nginx-server-certs
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-configmap
      - name: nginx-server-certs
        secret:
          secretName: nginx-server-certs
EOF

测试Nginx服务

1
2
3
4
5
6
NGINX_POD=$(kubectl get pod  -l run=my-nginx -o jsonpath={.items..metadata.name})
kubectl exec -it $NGINX_POD -c istio-proxy \
        -- \
        curl -I -k \
        --resolve nginx.example.com:443:127.0.0.1 \
        https://nginx.example.com

输出示例

1
2
3
4
5
6
7
8
9
HTTP/1.1 200 OK
Server: nginx/1.17.6
Date: Mon, 30 Dec 2019 14:26:15 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 19 Nov 2019 12:50:08 GMT
Connection: keep-alive
ETag: "5dd3e500-264"
Accept-Ranges: bytes

使用Gateway配置Ingress

配置Gateway

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: PASSTHROUGH
    hosts:
    - nginx.example.com
EOF

配置VirtualService

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - nginx.example.com
  gateways:
  - mygateway
  tls:
  - match:
    - port: 443
      sni_hosts:
      - nginx.example.com
    route:
    - destination:
        host: my-nginx
        port:
          number: 443
EOF

测试TLS直通

从集群外部访问Nginx服务

1
2
3
4
curl -v \
     --resolve nginx.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST \
     --cacert example.com.crt \
     https://nginx.example.com:$SECURE_INGRESS_PORT

输出示例

这里可以看到证书信息是nginx.example.com，并且证书能顺利通过验证

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
* Added nginx.example.com:31944:192.168.48.37 to DNS cache
* About to connect() to nginx.example.com port 31944 (#0)
*   Trying 192.168.48.37...
* Connected to nginx.example.com (192.168.48.37) port 31944 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: example.com.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* 	subject: O=Nginx organization,CN=nginx.example.com
* 	start date: Dec 30 13:55:24 2019 GMT
* 	expire date: Dec 29 13:55:24 2020 GMT
* 	common name: nginx.example.com
* 	issuer: CN=example.com,O=example Inc.
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: nginx.example.com:31944
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.17.6
< Date: Mon, 30 Dec 2019 14:30:45 GMT
< Content-Type: text/html
< Content-Length: 612
< Last-Modified: Tue, 19 Nov 2019 12:50:08 GMT
< Connection: keep-alive
< ETag: "5dd3e500-264"
< Accept-Ranges: bytes
< 
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
</body>
</html>

清理现场

清理Kubernetes资源对象

1
2
3
4
5
6
kubectl delete secret nginx-server-certs
kubectl delete configmap nginx-configmap
kubectl delete service my-nginx
kubectl delete deployment my-nginx
kubectl delete gateway mygateway
kubectl delete virtualservice nginx

清理证书和配置文件

1
rm -rf nginx.conf nginx.example.com.crt nginx.example.com.csr nginx.example.com.key

Ingress With Cert-Manager

[暂时没这块需求就不弄了]

说明

  • Cert-Manager是一个社区项目，这里是文档
  • 项目状态是pre-1.0，还没GA，API可能还会变更，所以谨慎使用！
  • 可以通过这个项目申请Let's Encrypt证书
  • 自动管理和颁发证书
  • 定期更新轮换证书

环境准备

Istio特性部署要求

  • SDS(Secrect Discovery Service)
  • Ingress

Istio部署参数

1
2
3
4
5
istioctl manifest apply \
  --set values.gateways.istio-ingressgateway.sds.enabled=true \
  --set values.global.k8sIngress.enabled=true \
  --set values.global.k8sIngress.enableHttps=true \
  --set values.global.k8sIngress.gatewayName=ingressgateway

为了测试方便，这里额外添加参数

1
2
3
4
5
6
7
8
9
10
11
12
13
istioctl manifest apply \
    --set profile='demo' \
    --set cni.enabled=true \
    --set cni.components.cni.namespace='kube-system' \
    --set hub='dockerhub.azk8s.cn/istio' \
    --set values.gateways.istio-ingressgateway.type='NodePort' \
    --set values.gateways.istio-egressgateway.enabled=false \
    --set values.gateways.istio-ingressgateway.sds.enabled=true \
    --set values.global.controlPlaneSecurityEnabled=true \
    --set values.global.k8sIngress.enabled=true \
    --set values.global.k8sIngress.enableHttps=true \
    --set values.global.k8sIngress.gatewayName=ingressgateway \
    --set values.global.mtls.enabled=false

部署Cert-Manager

参考这里的Cert-Manager安装部署文档

创建命名空间
1
kubectl create namespace cert-manager
部署CRD和Cert-Manager
1
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.12.0/cert-manager.yaml
验证安装
1
kubectl get pods --namespace cert-manager

Egress

[暂时没这块需求就不弄了]

说明

  • Egress是用来管控Istio服务网格的出口流量