AWS S3 模組實作

本篇是實作常用的 AWS IAM 服務之 Terraform 模組，並且會使用到 YAML 資料結構來定義模組的內容，完整的專案程式碼分享在我的 Github 上。

1. 先定義整個專案檔案結構設定檔 ./configs/s3/s3.yaml 與放置 bucket policy JSON 檔案位置的目錄:
   • ./configs/s3/policies: 放置 bucket policy 的目錄
2. 模組 my_s3 的放置位置 modules/my_s3:

├── configs
│   ├── iam
│   │   ├── assume_role_policies
│   │   ├── policies
│   │   ├── role_policies
│   │   ├── user_policies
│   │   └── iam.yaml
│   ├── s3
│   │   ├── policies
│   │   └── s3.yaml
│   ├── subnet
│   │   └── my-subnets.yaml
│   └── vpc
│       └── my-vpcs.yaml
├── example.tfvars
├── locals.tf
├── main.tf
├── modules
│   ├── my_eips
│   ├── my_iam
│   ├── my_igw
│   ├── my_instances
│   ├── my_nacls
│   ├── my_route_tables
│   ├── my_s3
│   │   ├── outputs.tf
│   │   ├── provider.tf
│   │   ├── s3_bucket.tf
│   │   ├── s3_bucket_cors_configuration.tf
│   │   ├── s3_bucket_lifecycle_configuration.tf
│   │   ├── s3_bucket_server_side_encryption_configuration.tf
│   │   ├── s3_bucket_versioning.tf
│   │   ├── s3_bucket_website_configuration.tf
│   │   ├── s3_notification.tf
│   │   ├── s3_policy.tf
│   │   └── variables.tf
│   ├── my_subnets
│   └── my_vpc
└── variables.tf

3. 撰寫 ./configs/s3/s3.yaml 內容來定義 S3 需要用建立的資源：

buckets: []
# Example
# - bucket: nxd-dev-elecgame
#   bucket_policy_file: "./configs/s3/policies/????????.json"
#   department: "The department of bucket"
#   project: "The project of bucket"
#   versioning: "Enabled" or "Disabled"
#   index_document: "index.html"
#   server_site_encryption: "true"
#   lifecycle_rule:
#     - enabled: true or false
#       id: "delete-index"
#       prefix: "some-prefix/"
#       expiration:
#         days: 3
#         expired_object_delete_marker: false
#   cors_rule:
#     allowed_headers:
#       - "*"
#     allowed_methods:
#       - "GET"
#     allowed_origins:
#       - "http://localhost:8080"
#     expose_headers:
#       - "x-amz-server-side-encryption"
#       - "x-amz-request-id"
#       - "x-amz-id-2"
#     max_age_seconds: 3000

4. 撰寫 my_s3 模組

• ./modules/my_s3/outputs.tf:

output "s3_bucket" {
  value = aws_s3_bucket.s3_bucket
}

• ./modules/my_s3/provider.tf:

provider "aws" {
    region  = var.aws_region
    profile = var.aws_profile
}

• ./modules/my_s3/variables.tf:

variable "aws_region" {
  description = "AWS region"
  default     = "ap-northeast-1"
}

variable "aws_profile" {
  description = "AWS profile"
  default     = ""
}

variable "project_name" {
  type    = string
  description = "Project name"
  default = ""
}

variable "department_name" {
  type        = string
  description = "Department name"
  default     = "SRE"
}

variable "s3_path" {
  type        = string
  description = "s3 path"
  default     = ""
}

• ./modules/my_s3/s3_bucket.tf:
  利用 for_each 來迭代 locals.s3_buckets 以 bucket 為 key 值建立 map 物件

data "aws_canonical_user_id" "current_user" {}

resource "aws_s3_bucket" "s3_bucket" {
  for_each = { for r in local.s3_buckets : r.bucket => r }

  bucket = each.value.bucket

  tags = {
    Name       = each.value.bucket
    Department = each.value.department
    Project    = each.value.project
  }

  tags_all = {
    Name       = each.value.bucket
    Department = each.value.department
    Project    = each.value.project
  }
}

• ./modules/my_s3/s3_bucket_cors_configuration.tf:

resource "aws_s3_bucket_cors_configuration" "configurations" {

  for_each = { for r in local.s3_buckets : r.bucket => r if lookup(r, "cors_rule", "") != "" }
  
  bucket = aws_s3_bucket.s3_bucket[each.value.bucket].id

  cors_rule {
    allowed_headers = each.value.cors_rule.allowed_headers == "" ? null : each.value.cors_rule.allowed_headers
    allowed_methods = each.value.cors_rule.allowed_methods == "" ? null : each.value.cors_rule.allowed_methods
    allowed_origins = each.value.cors_rule.allowed_origins == "" ? null : each.value.cors_rule.allowed_origins
    expose_headers  = each.value.cors_rule.expose_headers == "" ? null : each.value.cors_rule.expose_headers
    max_age_seconds = each.value.cors_rule.max_age_seconds
  }

  depends_on = [
    aws_s3_bucket.s3_bucket
  ]
}

• ./modules/my_s3/s3_bucket_lifecycle_configuration.tf:

resource "aws_s3_bucket_lifecycle_configuration" "configurations" {

  for_each = { for r in local.s3_buckets : r.bucket => r if length(r.lifecycle_rule) > 0 }

  bucket = aws_s3_bucket.s3_bucket[each.value.bucket].id

  dynamic "rule" {

    for_each = lookup(each.value, "lifecycle_rule", [])
    
    content {
      id = rule.value.id

      dynamic "abort_incomplete_multipart_upload" {
        for_each = lookup(rule.value, "abort_incomplete_multipart_upload_days", 0) == 0 ? [] : [1]
        
        content {
          days_after_initiation = rule.value.abort_incomplete_multipart_upload_days
        }
      }

      filter {
        prefix = lookup(rule.value, "prefix", "")  
      }
      
      expiration {
        days                         = lookup(rule.value.expiration, "days", null)
        expired_object_delete_marker = lookup(rule.value.expiration, "expired_object_delete_marker", null)
      }
      
      status = (lookup(rule.value, "enabled", false)) ? "Enabled" : "Disabled"
    }
  }

  depends_on = [
    aws_s3_bucket.s3_bucket,
    aws_s3_bucket_versioning.versionings
  ]
}

• ./modules/my_s3/s3_bucket_server_side_encryption_configuration.tf:

resource "aws_s3_bucket_server_side_encryption_configuration" "configurations" {

  for_each = { for r in local.s3_buckets : r.bucket => r if lookup(r, "server_site_encryption", "") != "" }

  bucket = aws_s3_bucket.s3_bucket[each.value.bucket].id

  rule {    
    bucket_key_enabled = false
    
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  } 

  depends_on = [
    aws_s3_bucket.s3_bucket
  ]
}

• ./modules/my_s3/s3_bucket_versioning.tf:

resource "aws_s3_bucket_versioning" "versionings" {
  
  for_each = { for r in local.s3_buckets : r.bucket => r }

  bucket = aws_s3_bucket.s3_bucket[each.value.bucket].id

  versioning_configuration {
    status = each.value.versioning
  }

  depends_on = [
    aws_s3_bucket.s3_bucket
  ]
}

• ./modules/my_s3/s3_bucket_website_configuration.tf:

resource "aws_s3_bucket_website_configuration" "configurations" {
  
  for_each = { for r in local.s3_buckets : r.bucket => r if lookup(r, "index_document", "") != "" }

  bucket = aws_s3_bucket.s3_bucket[each.value.bucket].id

  index_document {
    suffix = each.value.index_document
  }

  depends_on = [
    aws_s3_bucket.s3_bucket
  ]
}

• ./modules/my_s3/s3_notification.tf:

resource "aws_s3_bucket_notification" "bucket_notifications" {
  for_each = { for r in local.s3_buckets : r.bucket => r if lookup(r, "notification_eventbridge", false) == true }

  bucket = aws_s3_bucket.s3_bucket["${each.value.bucket}"].id

  eventbridge = true
}

• ./modules/my_s3/s3_policy.tf:

resource "aws_s3_bucket_policy" "s3_bucket_policy" {
  for_each = { for r in local.s3_buckets : r.bucket => r if r.bucket_policy_file != "" }

  bucket = each.value.bucket
  policy = file("${each.value.bucket_policy_file}")

  depends_on = [
    aws_s3_bucket.s3_bucket
  ]
}

4. 撰寫專案相關程式

• example.tfvars:

aws_region="ap-northeast-1"
aws_profile="<YOUR_PROFILE>"
project_name="example"
department_name="SRE"

• main.tf:

terraform {
  required_providers {
    aws = {
      version = "5.15.0"
    }
  }

  backend "s3" {
    bucket                  = "<YOUR_S3_BUCKET_NAME>"
    dynamodb_table          = "<YOUR_DYNAMODB_TABLE_NAME>"
    key                     = "terraform.tfstate"
    region                  = "ap-northeast-1"
    shared_credentials_file = "~/.aws/config"
    profile                 = "<YOUR_PROFILE>"
  }
}

其他模組省略...

# s3
module "s3" {
  aws_profile = var.aws_profile
  aws_region  = var.aws_region
  environment = var.environment
  s3_path     = "./configs/s3/s3.yaml"

  source = "./modules/my_s3"
}

Terraform 執行計畫

1. 嘗試建立一個 Bucket 與 Bucket Policy 測試一下模組：

buckets:
  - bucket: my-bucket
    bucket_policy_file: "./configs/s3/policies/my-bucket.json"
    department: "The department of bucket"
    project: ""
    versioning: "Enabled"
    index_document: ""
    server_site_encryption: "true"
    lifecycle_rule:
      - enabled: true or false
        id: "delete-index"
        prefix: "some-prefix/"
        expiration:
          days: 3
          expired_object_delete_marker: false
    cors_rule: {}

2. 於專案目錄下執行 terraform init && terraform plan --out .plan -var-file=example.tfvars 來確認一下結果:

  ...
  # module.s3.aws_s3_bucket.s3_bucket["my-bucket"] will be created
  + resource "aws_s3_bucket" "s3_bucket" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      + arn                         = (known after apply)
      + bucket                      = "my-bucket"
      + bucket_domain_name          = (known after apply)
      + bucket_prefix               = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags                        = {
          + "Department" = "The department of bucket"
          + "Name"       = "my-bucket"
          + "Project"    = ""
        }
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
    }

  # module.s3.aws_s3_bucket_lifecycle_configuration.configurations["my-bucket"] will be created
  + resource "aws_s3_bucket_lifecycle_configuration" "configurations" {
      + bucket = (known after apply)
      + id     = (known after apply)

      + rule {
          + id     = "delete-index"
          + status = "Disabled"

          + expiration {
              + days                         = 3
              + expired_object_delete_marker = false
            }

          + filter {
              + prefix = "some-prefix/"
            }
        }
    }

  # module.s3.aws_s3_bucket_policy.s3_bucket_policy["my-bucket"] will be created
  + resource "aws_s3_bucket_policy" "s3_bucket_policy" {
      + bucket = "my-bucket"
      + id     = (known after apply)
      + policy = jsonencode(
            {
              + Id        = "PolicyForMyBucket"
              + Statement = []
              + Version   = "2008-10-17"
            }
        )
    }

  # module.s3.aws_s3_bucket_server_side_encryption_configuration.configurations["my-bucket"] will be created
  + resource "aws_s3_bucket_server_side_encryption_configuration" "configurations" {
      + bucket = (known after apply)
      + id     = (known after apply)

      + rule {
          + bucket_key_enabled = false

          + apply_server_side_encryption_by_default {
              + sse_algorithm = "AES256"
            }
        }
    }

  # module.s3.aws_s3_bucket_versioning.versionings["my-bucket"] will be created
  + resource "aws_s3_bucket_versioning" "versionings" {
      + bucket = (known after apply)
      + id     = (known after apply)

      + versioning_configuration {
          + mfa_delete = (known after apply)
          + status     = "Enabled"
        }
    }

Plan: 52 to add, 0 to change, 0 to destroy.
─────────────────────────────────────────────────────────────────────────────

Saved the plan to: .plan

To perform exactly these actions, run the following command to apply:
    terraform apply ".plan"

下一篇文章將會展示實作 AWS CloudFront 之 Terraform 模組。