iT邦幫忙

2021 iThome 鐵人賽

DAY 23
0
自我挑戰組

HomeLab 30天,胡搞瞎搞亂弄一通。系列 第 23

Day23,替你的Gitlab pipeline 添加點搞事

正文

在前面介紹gitlab-ci的pipeline中我僅僅只用到了build stage作為container image build的動作。

然而在實務上,我們可能還會有許多需要整合的stages單元測試、整合測試、各種佈署細節、各種通報細節以及與gitlab issue、dashboard整合的動作。

這邊我就準備來搞三個官方有在主導的安全項目,不過不是完全以官方的方式去進行,因為部分的功能在介面上需要enterpirse版本才能夠支援,但不代表無法使用他,只是說我們無法在官方的介面上看到對應的功能畫面。

三個動作則分別是 SAST、Container Scanning、DAST

  • SAST - Static Application Security Testing

    • 靜態程式碼掃描,這個部分是完全開放的,也就是說無論是在CE/EE的gitlab版本中皆可以從介面上直接套用他,而他支援的語言掃描也非常多種,可以直接參考連結,而寫法上大概就像這樣,修改.gitlab-ci.yaml
    stages:
    - sast
    sast:
      variables:
        SAST_EXCLUDED_ANALYZERS: bandit, brakeman, eslint, flawfinder, gosec, kubesec,
      nodejs-scan, phpcs-security-audit, pmd-apex, semgrep, sobelow, spotbugs
      stage: sast    
    include:
    - template: Security/SAST.gitlab-ci.yml
    

    相關的variables都可以再去參考官方

    • 輸出的json則是像這樣(特地找一個有vulnerability的),最終都可以搭配jq做操作。
    {
      "version": "14.0.0",
      "vulnerabilities": [
        {
          "id": "65f3cc30cdd0fea1d39c9f7b3300112aa84194829d19c8288038473877b1e549",
          "category": "sast",
          "name": "Weak random generator",
          "message": "Weak random generator",
          "cve": "sast-test/Controllers/WeatherForecastController.cs:33:SCS0005",
          "scanner": {
            "id": "security_code_scan",
            "name": "Security Code Scan"
          },
          "location": {
            "file": "sast-test/Controllers/WeatherForecastController.cs",
            "start_line": 33
          },
          "identifiers": [
            {
              "type": "security_code_scan_rule_id",
              "name": "SCS0005",
              "value": "SCS0005",
              "url": "https://security-code-scan.github.io/#SCS0005"
            }
          ]
        },
        {
          "id": "1a69e74a7f4a5242b1ff87276f31f151f8a1a7aacc27106a2991de3a0a7ca28e",
          "category": "sast",
          "name": "Weak random generator",
          "message": "Weak random generator",
          "cve": "sast-test/Controllers/WeatherForecastController.cs:34:SCS0005",
          "scanner": {
            "id": "security_code_scan",
            "name": "Security Code Scan"
          },
          "location": {
            "file": "sast-test/Controllers/WeatherForecastController.cs",
            "start_line": 34
          },
          "identifiers": [
            {
              "type": "security_code_scan_rule_id",
              "name": "SCS0005",
              "value": "SCS0005",
              "url": "https://security-code-scan.github.io/#SCS0005"
            }
          ]
        }
      ],
      "remediations": [],
      "scan": {
        "scanner": {
          "id": "security_code_scan",
          "name": "Security Code Scan",
          "url": "https://security-code-scan.github.io",
          "vendor": {
            "name": "GitLab"
          },
          "version": "3.5.3"
        },
        "type": "sast",
        "start_time": "2021-09-08T07:09:20",
        "end_time": "2021-09-08T07:09:38",
        "status": "success"
      }
    }
    
    
  • Container Scanning

    • 這邊就是針對container的掃瞄,gitlab官方則是基於trivy、grype來使用,而因為我日常在使用的gitlab為CE版本這個部分就必須自己填上,會直接使用trivy的source image來做掃描,舉例如下
    stages:
    - container_scan
    image-scanner:
      image: 
        name: aquasec/trivy:latest
        entrypoint: [""]
      stage: container_scan
      before_script:
      - apk add --no-cache docker openrc 
      - rc-update add docker boot
      - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
      script:
      - |
        if [[ "$CI_COMMIT_BRANCH" == "master" ]]; then
          tag=":dev"
        else
          tag=":$CI_COMMIT_REF_NAME"
        fi
      - trivy image -f json -o report.json $CI_REGISTRY_IMAGE${tag}
      artifacts:
        paths: 
        - report.json    
      tags:
      - docker
    

    trivy還有相關多的指令能使用,如也能夠搭配 trivy image --exit-code 1 之類的來讓你的pipeline fail,也能夠做針對部分語言做package scanner的動作,有興趣可以去閱讀官方文件

    • 一樣會是json的輸出,也可以搭配jq操作
    [
      {
        "Target": "registry.gitlab.com/gurubear-ithome-13th/homelabapi:dev (debian 10.10)",
        "Class": "os-pkgs",
        "Type": "debian",
        "Vulnerabilities": [
          {
            "VulnerabilityID": "CVE-2011-3374",
            "PkgName": "apt",
            "InstalledVersion": "1.8.2.3",
            "Layer": {
              "Digest": "sha256:a330b6cecb98cd2425fd25fce36669073f593b3176b4ee14731e48c05d678cdd",
              "DiffID": "sha256:d000633a56813933cb0ac5ee3246cf7a4c0205db6290018a169d7cb096581046"
            },
            "SeveritySource": "nvd",
            "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2011-3374",
            "Description": "It was found that apt-key in apt, all versions, do not correctly validate gpg keys with the master keyring, leading to a potential man-in-the-middle attack.",
            "Severity": "LOW",
            "CweIDs": [
              "CWE-347"
            ],
            "CVSS": {
              "nvd": {
                "V2Vector": "AV:N/AC:M/Au:N/C:N/I:P/A:N",
                "V3Vector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:L/A:N",
                "V2Score": 4.3,
                "V3Score": 3.7
              }
            },
            "References": [
              "https://access.redhat.com/security/cve/cve-2011-3374",
              "https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=642480",
              "https://people.canonical.com/~ubuntu-security/cve/2011/CVE-2011-3374.html",
              "https://seclists.org/fulldisclosure/2011/Sep/221",
              "https://security-tracker.debian.org/tracker/CVE-2011-3374",
              "https://snyk.io/vuln/SNYK-LINUX-APT-116518",
              "https://ubuntu.com/security/CVE-2011-3374"
            ],
            "PublishedDate": "2019-11-26T00:15:00Z",
            "LastModifiedDate": "2021-02-09T16:08:00Z"
          },
          .
          .
          . 大量略過
          .
          {
            "VulnerabilityID": "CVE-2021-37600",
            "PkgName": "util-linux",
            "InstalledVersion": "2.33.1-0.1",
            "Layer": {
              "Digest": "sha256:a330b6cecb98cd2425fd25fce36669073f593b3176b4ee14731e48c05d678cdd",
              "DiffID": "sha256:d000633a56813933cb0ac5ee3246cf7a4c0205db6290018a169d7cb096581046"
            },
            "SeveritySource": "nvd",
            "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2021-37600",
            "Title": "util-linux: integer overflow can lead to buffer overflow in get_sem_elements() in sys-utils/ipcutils.c",
            "Description": "An integer overflow in util-linux through 2.37.1 can potentially cause a buffer overflow if an attacker were able to use system resources in a way that leads to a large number in the /proc/sysvipc/sem file.",
            "Severity": "MEDIUM",
            "CweIDs": [
              "CWE-190"
            ],
            "CVSS": {
              "nvd": {
                "V2Vector": "AV:L/AC:H/Au:N/C:N/I:N/A:P",
                "V3Vector": "CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H",
                "V2Score": 1.2,
                "V3Score": 5.5
              },
              "redhat": {
                "V3Vector": "CVSS:3.1/AV:L/AC:H/PR:N/UI:R/S:U/C:N/I:N/A:H",
                "V3Score": 4.7
              }
            },
            "References": [
              "https://github.com/karelzak/util-linux/commit/1c9143d0c1f979c3daf10e1c37b5b1e916c22a1c",
              "https://github.com/karelzak/util-linux/issues/1395",
              "https://security.netapp.com/advisory/ntap-20210902-0002/"
            ],
            "PublishedDate": "2021-07-30T14:15:00Z",
            "LastModifiedDate": "2021-09-02T09:15:00Z"
          }
        ]
      }
    ]
    

    實務上使用可能會要加上 --ignore-unfixed,不然可能就會像我這樣多到炸裂~

  • DAST - Dynamic Application Security Testing

    • 這邊也是使用gitlab官方包好的image,不過非EE版本不支援從介面直接新增,所以我們需要去看他的open source並將加入到自己的pipeline當中,而這邊的DAST則是基於ZAP這套開源掃描工具,我們撰寫的方式也很簡單
    stages:
    - dast
    dast:
      stage: dast
      image:
        name: "$SECURE_ANALYZERS_PREFIX/dast:$DAST_VERSION"
      variables:
        GIT_STRATEGY: none
        DAST_VERSION: 1
        DAST_MARKDOWN_REPORT: report.md
        SECURE_ANALYZERS_PREFIX: "registry.gitlab.com/gitlab-org/security-products/analyzers"
      allow_failure: true
      script:
        - export DAST_WEBSITE="https://homelab.gurubear.info/"
        - /analyze
      artifacts:
        name: "dast-report"
        paths:
          - gl-dast-report.json
          - $DAST_MARKDOWN_REPORT
    

    這邊一樣有大量變數可以參考官方設定,可以輸出的有json、html、markdown

    • 這邊我就不輸出json改輸出markdown看看了~以下截圖

本次使用的repo pipeline

以上就是不透過官方正規管道產出的3種掃描方式,如果有EE版本授權、Ultimate subscription的話,還是建議照著官方來,也能夠配合搭配dashboard使用,相信能夠更有效率。

閒聊

這一兩年來資安的議題非常熱絡,市面也是充斥著各種原碼掃描、端點掃描軟體。自己是覺得在流程確立的情況下用什麼工具都好(具公信力的),重點還是在有沒有人能解決、有沒有人要處理,不過現實總會有各種無奈~


上一篇
Day22,Cert-Manager
下一篇
Day24,試著用rancher交差Dashboard
系列文
HomeLab 30天,胡搞瞎搞亂弄一通。30
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言