iT邦幫忙

2022 iThome 鐵人賽
DAY 8
0
DevOps

那些文件沒告訴你的 AWS EKS系列 第 8 篇

[08] 為什麼 EKS 可以整合 IAM roles for service accounts（IRSA）（一）

14th鐵人賽
1407 瀏覽

  • 分享至 

  • xImage
    •  

根據 EKS IAM roles for service accounts[1]，我們可以透過於在 Kubernetes 上的 Service Account 及 Pod annotation 設置 IAM role ARN 後，則 Pod 就可以取得此 IAM role 對應的權限。

因此本文希望探討了解「為什麼 EKS 是如何根據原生 Kubernetes 功能，整合 IAM 權限最終使得 Pod 內環境可以獲取此 IAM role」。

建置環境

  1. 建立以下 IAM policy 並儲存為 iam-policy.json JSON。
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetObjectVersion"
            ],
            "Resource": "arn:aws:s3:::*"
        }
    ]
}
  1. 透過 AWS CLI 建立 Policy
$ aws iam create-policy --policy-name my-s3 --policy-document file://iam-policy.json

{
    "Policy": {
        "PolicyName": "my-s3",
        "PolicyId": "ANPAYFMQSNSE2LL6JJT3Q",
        "Arn": "arn:aws:iam::111111111111:policy/my-s3",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "CreateDate": "2022-09-23T10:56:00+00:00",
        "UpdateDate": "2022-09-23T10:56:00+00:00"
    }
}
  1. 透過 eksctl 命令建立 Kubernetes Service Account 及 IAM role。此會透過 CloudFormation 建立對應的 role。
$ eksctl create iamserviceaccount \
  --name my-s3-sa \
  --namespace default \
  --cluster ironman-2022 \
  --attach-policy-arn arn:aws:iam::111111111111:policy/my-s3 \
  --approve
2022-09-23 10:56:52 [ℹ]  1 existing iamserviceaccount(s) (kube-system/aws-node) will be excluded
2022-09-23 10:56:52 [ℹ]  1 iamserviceaccount (default/my-s3-sa) was included (based on the include/exclude rules)
2022-09-23 10:56:52 [!]  serviceaccounts that exist in Kubernetes will be excluded, use --override-existing-serviceaccounts to override
2022-09-23 10:56:52 [ℹ]  1 task: {
    2 sequential sub-tasks: {
        create IAM role for serviceaccount "default/my-s3-sa",
        create serviceaccount "default/my-s3-sa",
    } }2022-09-23 10:56:52 [ℹ]  building iamserviceaccount stack "eksctl-ironman-2022-addon-iamserviceaccount-default-my-s3-sa"
2022-09-23 10:56:52 [ℹ]  deploying stack "eksctl-ironman-2022-addon-iamserviceaccount-default-my-s3-sa"
2022-09-23 10:56:52 [ℹ]  waiting for CloudFormation stack "eksctl-ironman-2022-addon-iamserviceaccount-default-my-s3-sa"
2022-09-23 10:57:22 [ℹ]  waiting for CloudFormation stack "eksctl-ironman-2022-addon-iamserviceaccount-default-my-s3-sa"
2022-09-23 10:57:22 [ℹ]  created serviceaccount "default/my-s3-sa"
  1. 建立 Pod 使用 AWS CLI image 作為測試，儲存為 aws-cli.yaml YAML。
$ cat ./aws-cli.yaml
apiVersion: v1
kind: Pod
metadata:
  name: aws-cli
  namespace: default
spec:
  serviceAccountName: my-s3-sa
  containers:
  - name: aws-cli
    image: amazon/aws-cli:latest
    command:
      - sleep
      - infinity
    imagePullPolicy: IfNotPresent
  restartPolicy: Always
  1. 部署此 Pod。
$ kubectl apply -f ./aws-cli.yaml
pod/aws-cli created

分析

透過 kubectl describe 命令查看 Pod，以下列出部分資訊：

$ kubectl describe po aws-cli
Name:         aws-cli
Namespace:    default
...
...
Containers:
  aws-cli:
    ...
    Command:
      sleep
      infinity
    State:          Running
      Started:      Fri, 23 Sep 2022 10:59:13 +0000
    Ready:          True
    Restart Count:  0
    Environment:
      AWS_STS_REGIONAL_ENDPOINTS:   regional
      AWS_DEFAULT_REGION:           eu-west-1
      AWS_REGION:                   eu-west-1
      AWS_ROLE_ARN:                 arn:aws:iam::111111111111:role/eksctl-ironman-2022-addon-iamserviceaccount-defau-Role1-KXGCZM9EIHAA
      AWS_WEB_IDENTITY_TOKEN_FILE:  /var/run/secrets/eks.amazonaws.com/serviceaccount/token
    Mounts:
      /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cvtwf (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  aws-iam-token:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  86400
  kube-api-access-cvtwf:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
...

可以觀察多了一些設定：

  • 環境變數 Environment： AWS_STS_REGIONAL_ENDPOINTS、AWS_DEFAULT_REGION、AWS_REGION、AWS_ROLE_ARN、AWS_WEB_IDENTITY_TOKEN_FILE
  • 掛載目錄 Mount：
    • /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro)
    • /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cvtwf (ro)

根據 Day6 我們所探討過的 Mutating Webhook，是有可能在 Pod 部署過程中更新 Pod Spec。

filter @logStream like /^kube-apiserver-audit/
 | fields @timestamp, @message
 | sort @timestamp asc
 | filter objectRef.name == 'aws-cli' AND verb != 'get' AND verb != 'list'
 | limit 10000

從 audit logs 中，我們可以觀察到 mutating webhook pod-identity-webhook 更新了對應 annotation、environment variable、Mount 等資訊：

  • annotations.patch.webhook.admission.k8s.io/round_0_index_2: {"configuration":"pod-identity-webhook","webhook":"iam-for-pods.amazonaws.com","patch":[{"op":"add","path":"/spec/volumes/0","value":{"name":"aws-iam-token","projected":{"sources":[{"serviceAccountToken":{"audience":"sts.amazonaws.com","expirationSeconds":86400,"path":"token"}}]}}},{"op":"add","path":"/spec/containers","value":[{"name":"aws-cli","image":"amazon/aws-cli:latest","command":["sleep","infinity"],"env":[{"name":"AWS_STS_REGIONAL_ENDPOINTS","value":"regional"},{"name":"AWS_DEFAULT_REGION","value":"eu-west-1"},{"name":"AWS_REGION","value":"eu-west-1"},{"name":"AWS_ROLE_ARN","value":"arn:aws:iam::111111111111:role/eksctl-ironman-2022-addon-iamserviceaccount-defau-Role1-KXGCZM9EIHAA"},{"name":"AWS_WEB_IDENTITY_TOKEN_FILE","value":"/var/run/secrets/eks.amazonaws.com/serviceaccount/token"}],"resources":{},"volumeMounts":[{"name":"kube-api-access-cvtwf","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"},{"name":"aws-iam-token","readOnly":true,"mountPath":"/var/run/secrets/eks.amazonaws.com/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}]}],"patchType":"JSONPatch"}

近一步查看這個 Service Account Token 是什麼，其格式似乎是 JSON Web Tokens（JWT）[2] token 格式：

$ kubectl exec -it aws-cli -- cat /var/run/secrets/eks.amazonaws.com/serviceaccount/token
eyJhbGciOiJSUzI1NiIsImtpZCI6IjFlMWQzNzMyZGE0ZjY3OGQyOTYyOWFhODkwNDUzNWMzNTRhNTBjYzkifQ.eyJhdWQiOlsic3RzLmFtYXpvbmF3cy5jb20iXSwiZXhwIjoxNjY0MDE3MTQyLCJpYXQiOjE2NjM5MzA3NDIsImlzcyI6Imh0dHBzOi8vb2lkYy5la3MuZXUtd2VzdC0xLmFtYXpvbmF3cy5jb20vaWQvQThFN0EzOUNBRUJFRjZBQTkyNTBERkE5MzY2RkRGQTIiLCJrdWJlcm5ldGVzLmlvIjp7Im5hbWVzcGFjZSI6ImRlZmF1bHQiLCJwb2QiOnsibmFtZSI6ImF3cy1jbGkiLCJ1aWQiOiJlNjIzY2U4Yi0zOGEyLTQxNmQtYTgzOS05ZmU3ZDMzYTA0ZWIifSwic2VydmljZWFjY291bnQiOnsibmFtZSI6Im15LXMzLXNhIiwidWlkIjoiMTFmNWFhMjAtMDIxYi00MmI5LWI0N2YtOWU0MDVlYTdkOTMyIn19LCJuYmYiOjE2NjM5MzA3NDIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0Om15LXMzLXNhIn0.kQMRT47oEs4hO62Zv61qFMllnxjt6stsqtRpaIPypZ69-TaasCsE-LzME1aEqh3AeGjJxCqVMWqq4_AP5z1lL_bwSpfB4TJDcezGCVf1cp-Ko4jQ61W5GvJzPxsg_u_xdXKXppFw5vsDHsR0xoGG-3z8D_9VJRcnTOEm9azT_l6LkdqRfuRcG9Vys-KbBAtI68DNAYDnn6P2voddltvaBytS8FgYfHrAapf0UwKN059MpH5icHbhJAvV45oXrZsVs6_gAaYVPZaSRateQ-Z6d6JZVb9KDJl7T1RvX77ers4l8-lLGLeA4ZpyIYbljckOSmi39O-UJ2lVw1dFJziMKg

透過上述 JWT 頁面 decode 後，我們可以關注 payload 資料部分，其中我們關注到 issuer (iss) 正是 EKS OIDC provider URL：

{
  "aud": [
    "sts.amazonaws.com"
  ],
  "exp": 1664017142,
  "iat": 1663930742,
  "iss": "https://oidc.eks.eu-west-1.amazonaws.com/id/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "kubernetes.io": {
    "namespace": "default",
    "pod": {
      "name": "aws-cli",
      "uid": "e623ce8b-38a2-416d-a839-9fe7d33a04eb"
    },
    "serviceaccount": {
      "name": "my-s3-sa",
      "uid": "11f5aa20-021b-42b9-b47f-9e405ea7d932"
    }
  },
  "nbf": 1663930742,
  "sub": "system:serviceaccount:default:my-s3-sa"
}

我們也可以透過 aws eks describe-cluster 命令來查看 OIDC provider URL：

$ aws eks describe-cluster --name=ironman-2022 --output=text --query 'cluster.identity.oidc'
https://oidc.eks.eu-west-1.amazonaws.com/id/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

一般來說，若要使用 IRSA 功能，首先第一步驟是建立 IAM OIDC Provider[3] 於 EKS cluster 上。然而此步驟於先前定義 eksctl ClusterConfig 時則設定：

iam:
  withOIDC: true

總結

由上述初步查看 Pod 及 audit log 可以了解：

  • pod-identity-webhook 會替我們注入對應環境變數及掛載 service account token
  • 其中 token 格式為 JWT 並且由 EKS IAM OIDC endpoint 所發放

下一篇我們將會探討 pod-identity-webhook 如何實現掛載 Service Account Token 及 Kubernetes 為什麼可以整合 IAM 及此 token 用處為何。

以上資訊透過 EKS 所提供 Logs 來驗證上游 Kubernetes 運作原理，倘若上述內文有所錯誤，隨時可以留言或是私訊我。

參考文件

  1. IAM roles for service accounts - https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
  2. JWT - https://jwt.io/
  3. Creating an IAM OIDC provider for your cluster - https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html

上一篇
[07] 為什麼 CDK 部署 EKS cluster 會比較慢
下一篇
[09] 為什麼 EKS 可以整合 IAM roles for service accounts（IRSA）（二）
系列文
那些文件沒告訴你的 AWS EKS 共 30 篇
  1. 26
    [26] 為什麼 EKS 在更新 Kubernetes deployment 時會有 HTTP 502 的 error（一）
  2. 27
    [27] 為什麼 EKS 在更新 Kubernetes deployment 時會有 HTTP 502 的 error（二）
  3. 28
    [28] 為什麼 EKS 在更新 Kubernetes deployment 時會有 HTTP 502 的 error（三）
  4. 29
    [29] 為什麼 EKS 在更新 Kubernetes deployment 時會有 HTTP 502 的 error（四）
  5. 30
    [30] Summary
完整目錄
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言
iThome鐵人賽
參賽組數
1123 組
團體組數
52 組
累計文章數
23097 篇
完賽人數
656 人
iT+看影片追技術 看影片追技術 看更多
熱門tag 看更多
15th鐵人賽 13th鐵人賽 14th鐵人賽 12th鐵人賽 11th鐵人賽 鐵人賽 2019鐵人賽 2018鐵人賽 javascript 2017鐵人賽 python windows php c# windows server linux css react 程式設計 vue.js
熱門問題
熱門回答
熱門文章
IT邦幫忙