Day9 告別網站危機 Google Trust Services 證書及External DNS

2024 iThome 鐵人賽

DAY 9

Kubernetes

異世界生存戰記：30天煉成GKE大師系列第 9 篇

16th鐵人賽 google trust services external dns gke gcp

andrewchen930511

2024-09-23 11:23:55

128 瀏覽

分享至

前言

SSL 證書的維護是所有 SRE 的必修課程，有 DV, OV, EV, Wildcard, Multi Domain, Self-Signed, 免費證書等，如果公司規模不大，為了節省成本，通常會使用免費證書，常見的免費證書像是 Let's Encrypt SSL, ZeroSSL，今天就來介紹 GCP 代管，由 GTS 簽發的免費證書。

Google Trust Services 介紹

Google 於2017年初設立 Google Trust Services（GTS），這是一家根憑證機構（Root CA），將負責發行 Google 與 Alphabet 旗下各式服務所使用的數位憑證，像是旗下的 Youtube 也是由其簽發。

通過提供透明、可信且可靠的證書頒發機構，幫助構建更安全的互聯網，只要擁有網域的 DNS 控制權通過 GTS ACME API 即可免費提供公眾信任的 TLS 證書。

示範

Ingress 當中的kubernetes.io/ingress.class: gceAnnotions 已經棄用，不填寫的話預設就是產生 gce 的 Ingress

service.yaml, gce-ingress.yaml, ManagedCertificate.yaml, NodePort-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: httpd-nodeport-service
  namespace: httpd
spec:
  type: NodePort
  selector:
    app: httpd
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gts-httpd-ingress
  namespace: httpd
  annotations:
    # If the class annotation is not specified it defaults to "gce".
    kubernetes.io/ingress.class: gce
    # 綁定的 ManagedCertificate 證書名稱
    networking.gke.io/managed-certificates: httpd-managed-certificate
    # 綁定的 FrontendConfig 名稱
    networking.gke.io/v1beta1.FrontendConfig: httpd-frontend-config
spec:
  rules:
    - host: gts-httpd.demoit.shop
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: httpd-httpd
                port:
                  number: 80
---
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
  name: httpd-frontend-config
  namespace: httpd
spec:
  redirectToHttps:
    enabled: true
    responseCodeName: PERMANENT_REDIRECT
---
# ManagedCertificate 證書申請的網域
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
  name: httpd-managed-certificate
  namespace: httpd
spec:
  domains:
    - gts-httpd.demoit.shop

kubectl apply 以上的 Yaml 後會發現 gts-httpd-ingress call 了很多 GCP 的 API

$ kubectl describe ingress gts-httpd-ingress -n httpd  
Name:             gts-httpd-ingress
Labels:           <none>
Namespace:        httpd
Address:          34.54.255.56
Ingress Class:    <none>
Default backend:  <default>
Rules:
  Host                   Path  Backends
  ----                   ----  --------
  gts-httpd.demoit.shop  
                         /   httpd-nodeport-service:80 (10.120.81.12:80)
Annotations:             ingress.gcp.kubernetes.io/pre-shared-cert: mcrt-00c8c601-f509-4024-bd67-55615cf227e5
                         ingress.kubernetes.io/backends: {"k8s-be-30703--7fea8163c4312bae":"HEALTHY","k8s-be-31365--7fea8163c4312bae":"HEALTHY"}
                         ingress.kubernetes.io/forwarding-rule: k8s2-fr-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy
                         ingress.kubernetes.io/https-forwarding-rule: k8s2-fs-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy
                         ingress.kubernetes.io/https-target-proxy: k8s2-ts-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy
                         ingress.kubernetes.io/redirect-url-map: k8s2-rm-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy
                         ingress.kubernetes.io/ssl-cert: mcrt-00c8c601-f509-4024-bd67-55615cf227e5
                         ingress.kubernetes.io/static-ip: k8s2-fr-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy
                         ingress.kubernetes.io/target-proxy: k8s2-tp-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy
                         ingress.kubernetes.io/url-map: k8s2-um-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy
                         kubernetes.io/ingress.class: gce
                         networking.gke.io/managed-certificates: httpd-managed-certificate
                         networking.gke.io/v1beta1.FrontendConfig: httpd-frontend-config
Events:
  Type     Reason     Age                  From                     Message
  ----     ------     ----                 ----                     -------
  Warning  Translate  20m (x15 over 21m)   loadbalancer-controller  Translation failed: invalid ingress spec: could not find service "httpd/httpd-httpd"
  Normal   Sync       18m                  loadbalancer-controller  UrlMap "k8s2-um-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy" created
  Normal   Sync       18m                  loadbalancer-controller  TargetProxy "k8s2-tp-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy" created
  Normal   Sync       17m                  loadbalancer-controller  ForwardingRule "k8s2-fr-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy" created
  Normal   Sync       17m                  loadbalancer-controller  TargetProxy "k8s2-ts-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy" created
  Normal   Sync       17m                  loadbalancer-controller  ForwardingRule "k8s2-fs-cs4rb6lw-httpd-gts-httpd-ingress-1dvm7kqy" created
  Normal   Sync       9m44s (x8 over 21m)  loadbalancer-controller  Scheduled for sync

將 Ingress 產生的 Address(34.54.255.56) 綁定到 Cloud DNS，大約等待 30 分鐘～1 小時，之後到 Certificate Manager 頁面查看

看到如下頁面，就代表 GTS 證書已經簽發下來

打開網頁檢查是否帶有證書，為安全訪問

每次在 K8s 創建新的對外服務時，都要手動去 Cloud DNS 建立新的一筆紀錄，那有沒有自動化的方式可以省下這個手動的過程呢？

當然有，那就是 External DNS，讓我們開始介紹 External DNS 吧！

External DNS 介紹

ExternalDNS 為 Kubernetes 的入口和服務配置外部 DNS 伺服器，幫您使用公用 DNS 伺服器來發現Kubernetes 資源。ExternalDNS 是從 Kubernetes API 中檢索資源清單（如 Service、Ingress 等資源），以確定所需的 DNS 記錄資訊。本文介紹如何在 GKE 叢集中部署和使用 ExternalDNS 服務，並通過樣本驗證其功能特性。

安裝 External DNS

$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo update
$ helm search repo bitnami/external-dns                                                     
NAME                	CHART VERSION	APP VERSION	DESCRIPTION                                       
bitnami/external-dns	8.3.7        	0.15.0     	ExternalDNS is a Kubernetes addon that configur...

$ helm pull bitnami/external-dns --version 8.3.7
$ tar zxvf external-dns-8.3.7.tgz

在 values.yaml 更改以下設定

provider: google
google:
#改成管理Cloud DNS的專案ID 
  project: "ithome-202409-demo"
crd:
  create: true

helm upgrade --install external-dns -f values.yaml . -n external-dns --create-namespace

接下來，需要授予 External DNS 修改 DNS 紀錄的權限。
如果還不會在 GKE 內建立過 External DNS 的 SA Workload Identity，請回到 Day6 Workload Identity 教學，建立 External DNS 的 SA Workload Identity，因為 External DNS 必須擁有更改 Cloud DNS 紀錄的權限。

建立 External DNS 的 SA Workload Identity

    external-dns = {
      gcp_service_account           = "external-dns"
      k8s_service_account           = "external-dns"
      k8s_service_account_namespace = "external-dns"
      use_existing_gcp_sa           = false
      use_existing_k8s_sa           = true
      roles = [
        "roles/dns.admin"
      ],
      # 填入管理 Cloud DNS 所在的專案
      additional_projects = {
        "ithome-202409-demo" = [
          "roles/dns.admin"
        ]
      }
    }

查看 external-dns Pod Logs

level=info msg="Created Kubernetes client https://10.120.96.1:443"
level=info msg="Instantiating new Kubernetes client"
level=info msg="Using inCluster-config based on serviceaccount-token"
level=info msg="Created Kubernetes client https://10.120.96.1:443"
level=info msg="All records are already up to date"
level=info msg="All records are already up to date"
level=info msg="All records are already up to date"

會發現它需要去 call dns.googleapis.com/dns/v1/projects/ithome-202409-demo 這隻 API，如同上面講述的需要修改 DNS 紀錄的權限，如果不具備權限會持續報錯。從 Log 紀錄可知 External DNS 每分鐘會定時去查詢且更新 DNS 紀錄。

可建立以下測試用的 Ingress.yaml 了解 External DNS 的作用原理

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpd-external-dns-test-ingress
  namespace: httpd
spec:
  ingressClassName: external-nginx
  rules:
    - host: httpd-external-dns-test.demoit.shop
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: httpd-service
                port:
                  number: 80

查看 External DNS Logs 及 Cloud DNS 會發現它建立了三筆紀錄，一筆 A 紀錄及兩筆 TXT

level=info msg="Add records: a-httpd-external-dns-test.demoit.shop. TXT [\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/httpd/httpd-external-dns-test-ingress\"] 300"

level=info msg="Add records: httpd-external-dns-test.demoit.shop. A [35.238.126.135] 300"

level=info msg="Add records: httpd-external-dns-test.demoit.shop. TXT [\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/httpd/httpd-external-dns-test-ingress\"] 300"