YAML範本設計終於來到了尾聲,前一篇連變數都把它範本化了,還有什麼地方可以優化的嗎?
有的!如果回頭檢視之前設計的YAML內容,肯定還是可以發現有什麼地方可以改得更好,只是一開始沒發現或是暫時沒時間弄而已。然而實際在我的工作上其實已經針對使用的YAML重構到了第三個版本,雖然沒有完全寫在鐵人賽的文章內容中,但是大致上的精髓也是十之八九,就來看看還有什麼東西可以調整的吧!
之前在認識Pipeline的參數(Parameters)與變數(Variables)這篇文章中有提到參數的型態,除了常用的string、boolean這種常值類型的以外,還有對應stage、job、step與它們的集合類型的參數,而stepList就是可以加入許多額外的step的參數類型。
如果你是寫過程式的開發者,我想應該都有事件驅動的概念,如果你曾經寫過.Net WinForm的程式,應該更能理解我在這裡要加入的概念。
還記得我們的Job有分成三個吧?分別是BuildCode、BuildImage、DeployCloudRun,在這邊我希望能夠在這些Job實際執行動作「之前」與「之後」加入一些能夠讓外部動態增加一些step的彈性。這裡的外部不一定指的是範本以外的專案,也可以是範本中不同層級的YAML範本。
在BuildCode的範本裡面我打算加入codePreBuildSteps、codePostBuildSteps這兩個參數,分別代表程式碼在編譯之前與編譯之後可以額外加入step的區段。
為了這麼做,首先必須在BuildCode.yaml中加入對應的參數設定如下:
# jobs/buildCode.yaml
#======= Extensibility ========
- name: codePreBuildSteps
type: stepList
default: []
- name: codePostBuildSteps
type: stepList
default: []
接著在使用dotnet-sdk-in-linux-container.yaml之前加入codePreBuildSteps的參數,在結尾的部份加入codePostBuildSteps參數,YAML內容部份如下:
# jobs/buildCode.yaml
- ${{ parameters.codePreBuildSteps }}
- template: ../steps/dotnet-sdk-in-linux-container.yaml
parameters:
srcPath: ${{ parameters.sourcePath }}
slnOrCsprojName: ${{ parameters.slnOrCsprojName }}
dotnetCommand: publish
dotnetCommandArgs: '-c ${{ parameters.buildConfiguration }}'
- ${{ parameters.codePostBuildSteps }}
因為buildCode.yaml是job層級,上面還有個stage template,所以在stages/dotnet-build-stage.yaml中也必須同樣加上上面的參數設定,然後在使用buildCode.yaml的部份加上對應的參數值。
# stages/dotnet-build-stage.yaml
- template: ../jobs/buildCode.yaml
parameters:
sourcePath: ${{ parameters.sourcePath }}
slnOrCsprojName: ${{ parameters.slnOrCsprojName }}
codePreBuildSteps: ${{ parameters.codePreBuildSteps }}
codePostBuildSteps: ${{ parameters.codePostBuildSteps }}
以相同的概念在buildImage的部份則是加上imagePreBuildSteps、imagePostBuildSteps,不過這邊要加入的檔案有三個,分別是stages/dotnet-build-stage.yaml、jobs/buildImage.yaml、steps/docker-build.yaml。
參數設定如下:
#======= Extensibility ========
- name: imagePreBuildSteps
type: stepList
default: []
- name: imagePostBuildSteps
type: stepList
default: []
加了參數設定之後,接下來稍微有點不太一樣,在buildImage.yaml加入了上面的參數之後,是在使用docker-build.yaml的地方設定參數值,真正使用到參數值的地方則是在docker-build.yaml裡面,會不一樣主要還是裡面的steps差異,如果不影響就可以放外面,會影響就放裡面。
# steps/docker-build.yaml
- ${{ parameters.imagePreBuildSteps }}
- task: Docker@2
displayName: Build image
inputs:
repository: '${{ parameters.imgRepository }}'
command: 'build'
Dockerfile: ${{ parameters.buildDockerfile }}
buildContext: ${{ parameters.buildContext }}
${{ if eq(parameters.buildArgs, '') }}:
arguments: '--no-cache'
${{ else }}:
arguments: '--no-cache --build-arg ${{ parameters.buildArgs }}'
tags: ${{ parameters.imgTags }}
- ${{ parameters.imagePostBuildSteps }}
在deployCloudRun的部份因為沒有細到step template,所以直接就是在deployCloudRun.yaml中加入參數設定與使用參數值了(同一份檔案就直接全貼內容了)。不過參數設定的部份還是要另外複製一份到stages/deploy-cloudrun-stage.yaml檔案中。
# jobs/deployCloudRun.yaml
parameters:
- name: cloudRunProjectId
type: string
- name: imgRepository
type: string
- name: cloudRunRegion
type: string
- name: cloudRunServiceName
type: string
- name: templateResourceName
type: string
default: templates
- name: pipelineResourceName
type: string
default: pipelines
#======= Extensibility ========
- name: preDeploySteps
type: stepList
default: []
- name: postDeploySteps
type: stepList
default: []
- name: jobName
type: string
default: DeployCloudRun
jobs:
- job: ${{ parameters.jobName }}
steps:
- script: |
echo '${{ convertToJson(parameters) }}' >> parameters.json
cat parameters.json
displayName: Print template parameters
- checkout: ${{ parameters.templateResourceName }}
path: ${{ parameters.templateResourceName }}
clean: true
- checkout: ${{ parameters.pipelineResourceName }}
path: ${{ parameters.pipelineResourceName }}
clean: true
- ${{ parameters.preDeploySteps }}
- template: ../steps/merge-cloudRun-envVars.yaml
parameters:
templateResourceName: ${{ parameters.templateResourceName }}
pipelineResourceName: ${{ parameters.pipelineResourceName }}
projectEnvVarsFile: variables/cloudrun-envVars.yaml
sharedEnvVarsFile: variables/cloudrun-envVars.yaml
mergedNewFileFullPath: $(Agent.BuildDirectory)/mergedEnvVars.yaml
- task: Bash@3
displayName: Deploy docker image to cloudrun
inputs:
targetType: 'inline'
script: |
docker run --rm \
-v $(Agent.BuildDirectory)/${{ parameters.templateResourceName }}/ironman2022-gcp-key.json:/gcp/cloudKey.json \
-v $(Agent.BuildDirectory)/mergedEnvVars.yaml:/gcp/env-vars.yaml \
asia.gcr.io/google.com/cloudsdktool/google-cloud-cli:latest \
bash -c "gcloud auth login --cred-file=/gcp/cloudKey.json && gcloud run deploy ${{ parameters.cloudRunServiceName }} --env-vars-file /gcp/env-vars.yaml --image ${{ parameters.imgRepository }} --region ${{ parameters.cloudRunRegion }} --project ${{ parameters.cloudRunProjectId }} --allow-unauthenticated"
- ${{ parameters.postDeploySteps }}
stages/deploy-cloudrun-stage.yaml檔案中的內容則是下面這段:
# stages/deploy-cloudrun-stage.yaml
parameters:
# deployCloudRun.yaml parameters
- name: cloudRunProjectId
type: string
- name: imgRepository
type: string
- name: cloudRunRegion
type: string
- name: cloudRunServiceName
type: string
- name: templateResourceName
type: string
default: templates
- name: pipelineResourceName
type: string
default: pipelines
#======= Extensibility ========
- name: preDeploySteps
type: stepList
default: []
- name: postDeploySteps
type: stepList
default: []
- name: stageName
type: string
default: DeployCloudRun
stages:
- stage: ${{ parameters.stageName }}
displayName: 佈署Cloud Run
jobs:
- template: ../jobs/deployCloudRun.yaml
parameters:
cloudRunProjectId: ${{ parameters.cloudRunProjectId }}
imgRepository: ${{ parameters.imgRepository }}
cloudRunRegion: ${{ parameters.cloudRunRegion }}
cloudRunServiceName: ${{ parameters.cloudRunServiceName }}
templateResourceName: ${{ parameters.templateResourceName }}
pipelineResourceName: ${{ parameters.pipelineResourceName }}
preDeploySteps: ${{ parameters.preDeploySteps }}
postDeploySteps: ${{ parameters.postDeploySteps }}
如果你有仔細的看deployCloudRun.yaml和deploy-cloudrun-stage.yaml這兩個檔案的內容的話,應該會發現除了前面提到的stepList類型的參數之外,還另外增加了jobName或stageName參數。
這個修改就是讓外部可以重新設定job或stage的名稱(也就是識別ID),如果沒有特別設定的話則是使用預設值,也就是原本的名稱。這麼做的用意在哪呢?
主要是為了dependsOn屬性,也就是Job的依賴關係。
其實在鐵人賽的文章中,我只有以.Net的程式為範例來舉例,所以編譯程式碼的Job YAML直接就叫作buildCode.yaml,但是實際上若不是完全只有使用一種程式語言的環境,那麼應該會分成好幾種不同程式語言的YAML檔,例如:dotnet-build.yaml、ios-build.yaml、node-build.yaml之類的,也就是在buildImage的部份如果dependsOn固定設定一個名稱,可能在範本使用上不太妥當,所以我將jobName、stageName和dependsOn都設為參數。
要使用到參數設定的stepList,就是在引用範本的parameters底下對應的參數直接使用task類型的YAML語法就可以了,如果是在範本中使用到之後又要開放給外部(上一層)使用,則是繼續使用參數值即可。
例如在buildImage之前要做一些事,像是額外下載需要的檔案,或是更改檔名或設定值之類的…,用stages/dotnet-build-stage.yaml的部份內容來舉例如下:
- template: ../jobs/buildImage.yaml
parameters:
templateResourceName: ${{ parameters.templateResourceName }}
dependsOn: BuildCode
artifactName: ${{ parameters.artifactName }}
unzip: ${{ parameters.unzip }}
zipFileName: ${{ parameters.zipFileName }}
unzipToFolderPath: ${{ parameters.unzipToFolderPath }}
imgRepository: ${{ parameters.imgRepository }}
imgTags: ${{ parameters.imgTags }}
buildDockerfile: ${{ parameters.buildDockerfile }}
buildContext: ${{ parameters.buildContext }}
containerRegistry: ${{ parameters.containerRegistry }}
buildArgs: "TARGET_FILE=${{ parameters.startFileName }}"
imagePreBuildSteps:
- script: |
echo "這是使用的範例1"
displayName: 範例啦
- task: Bash@3
displayName: 還是範例啦
inputs:
targetType: 'inline'
script: |
echo "這是使用的範例2"
- ${{ parameters.imagePreBuildSteps }}
imagePostBuildSteps: ${{ parameters.imagePostBuildSteps }}
細部來看的話,就只有下面這段:
imagePreBuildSteps:
- script: |
echo "這是使用的範例1"
displayName: 範例啦
- task: Bash@3
displayName: 還是範例啦
inputs:
targetType: 'inline'
script: |
echo "這是使用的範例2"
- ${{ parameters.imagePreBuildSteps }}
最後的 - ${{ parameters.imagePreBuildSteps }}是讓範本設定同名的參數能延續使用,也就是不會讓外部設定的內容失效,簡單來說就是接下去的意思。
下面就是這篇文章修改過的完整YAML檔案內容:
# steps/docker-build.yaml
parameters:
- name: imgRepository
type: string
- name: imgTags
type: object
- name: buildDockerfile
type: string
default: Build.Dockerfile
- name: buildContext
type: string
- name: containerRegistry
type: string
- name: buildArgs
type: string
default: ''
#======= Extensibility ========
- name: imagePreBuildSteps
type: stepList
default: []
- name: imagePostBuildSteps
type: stepList
default: []
steps:
- ${{ parameters.imagePreBuildSteps }}
- task: Docker@2
displayName: Build image
inputs:
repository: '${{ parameters.imgRepository }}'
command: 'build'
Dockerfile: ${{ parameters.buildDockerfile }}
buildContext: ${{ parameters.buildContext }}
${{ if eq(parameters.buildArgs, '') }}:
arguments: '--no-cache'
${{ else }}:
arguments: '--no-cache --build-arg ${{ parameters.buildArgs }}'
tags: ${{ parameters.imgTags }}
- ${{ parameters.imagePostBuildSteps }}
- task: Docker@2
displayName: "Login to Container Registry"
inputs:
command: login
containerRegistry: ${{ parameters.containerRegistry }}
- task: Bash@3
displayName: Push docker image
inputs:
targetType: 'inline'
script: |
docker push -a ${{ parameters.imgRepository }}
# jobs/buildCode.yaml
parameters:
- name: sourcePath
type: string
default: $(Build.SourcesDirectory)
- name: slnOrCsprojName
type: string
default: Pipeline.sln
- name: buildConfiguration
type: string
default: release
values:
- release
- debug
- name: jobName
type: string
default: BuildCode
#======= Extensibility ========
- name: codePreBuildSteps
type: stepList
default: []
- name: codePostBuildSteps
type: stepList
default: []
jobs:
- job: ${{ parameters.jobName }}
steps:
- checkout: sources
clean: true
- template: ../steps/publish-commit-sha.yaml
parameters:
sourcePath: ${{ parameters.sourcePath }}
- ${{ parameters.codePreBuildSteps }}
- template: ../steps/dotnet-sdk-in-linux-container.yaml
parameters:
srcPath: ${{ parameters.sourcePath }}
slnOrCsprojName: ${{ parameters.slnOrCsprojName }}
dotnetCommand: publish
dotnetCommandArgs: '-c ${{ parameters.buildConfiguration }}'
- ${{ parameters.codePostBuildSteps }}
- template: ../steps/publish-pipeline-artifacts.yaml
parameters:
publishPath: '$(Build.ArtifactStagingDirectory)/zipFiles'
artifactName: '$(pipelineArtifact)'
# jobs/buildImage.yaml
parameters:
- name: artifactName
type: string
default: OutputFiles
- name: unzip
type: boolean
default: false
- name: zipFileName
type: string
default: ''
- name: unzipToFolderPath
type: string
default: ''
# - name: pipelineResourceName
# type: string
# default: pipelines
- name: templateResourceName
type: string
default: templates
- name: imgRepository
type: string
- name: imgTags
type: object
- name: buildDockerfile
type: string
default: Build.Dockerfile
- name: buildContext
type: string
- name: containerRegistry
type: string
- name: buildArgs
type: string
default: ''
#======= Extensibility ========
- name: imagePreBuildSteps
type: stepList
default: []
- name: imagePostBuildSteps
type: stepList
default: []
- name: dependsOn
type: object
default: []
- name: jobName
type: string
default: BuildImage
jobs:
- job: ${{ parameters.jobName }}
dependsOn: ${{ parameters.dependsOn }}
steps:
- checkout: ${{ parameters.templateResourceName }}
path: ${{ parameters.templateResourceName }}
clean: true
- template: ../steps/download-pipeline-artifacts.yaml
parameters:
artifactName: ${{ parameters.artifactName }}
unzip: ${{ parameters.unzip }}
zipFileName: ${{ parameters.zipFileName }}
unzipToFolderPath: ${{ parameters.unzipToFolderPath }}
- template: ../steps/docker-build.yaml
parameters:
imgRepository: ${{ parameters.imgRepository }}
imgTags: ${{ parameters.imgTags }}
buildDockerfile: $(Agent.BuildDirectory)/${{ parameters.templateResourceName }}/Dockerfile
buildContext: ${{ parameters.buildContext }}
containerRegistry: ${{ parameters.containerRegistry }}
${{ if ne(parameters.buildArgs, '') }}:
buildArgs: ${{ parameters.buildArgs }}
imagePreBuildSteps: ${{ parameters.imagePreBuildSteps }}
imagePostBuildSteps: ${{ parameters.imagePostBuildSteps }}
# jobs/deployCloudRun.yaml
parameters:
- name: cloudRunProjectId
type: string
- name: imgRepository
type: string
- name: cloudRunRegion
type: string
- name: cloudRunServiceName
type: string
- name: templateResourceName
type: string
default: templates
- name: pipelineResourceName
type: string
default: pipelines
#======= Extensibility ========
- name: preDeploySteps
type: stepList
default: []
- name: postDeploySteps
type: stepList
default: []
- name: jobName
type: string
default: DeployCloudRun
jobs:
- job: ${{ parameters.jobName }}
steps:
- script: |
echo '${{ convertToJson(parameters) }}' >> parameters.json
cat parameters.json
displayName: Print template parameters
- checkout: ${{ parameters.templateResourceName }}
path: ${{ parameters.templateResourceName }}
clean: true
- checkout: ${{ parameters.pipelineResourceName }}
path: ${{ parameters.pipelineResourceName }}
clean: true
- ${{ parameters.preDeploySteps }}
- template: ../steps/merge-cloudRun-envVars.yaml
parameters:
templateResourceName: ${{ parameters.templateResourceName }}
pipelineResourceName: ${{ parameters.pipelineResourceName }}
projectEnvVarsFile: variables/cloudrun-envVars.yaml
sharedEnvVarsFile: variables/cloudrun-envVars.yaml
mergedNewFileFullPath: $(Agent.BuildDirectory)/mergedEnvVars.yaml
- task: Bash@3
displayName: Deploy docker image to cloudrun
inputs:
targetType: 'inline'
script: |
docker run --rm \
-v $(Agent.BuildDirectory)/${{ parameters.templateResourceName }}/ironman2022-gcp-key.json:/gcp/cloudKey.json \
-v $(Agent.BuildDirectory)/mergedEnvVars.yaml:/gcp/env-vars.yaml \
asia.gcr.io/google.com/cloudsdktool/google-cloud-cli:latest \
bash -c "gcloud auth login --cred-file=/gcp/cloudKey.json && gcloud run deploy ${{ parameters.cloudRunServiceName }} --env-vars-file /gcp/env-vars.yaml --image ${{ parameters.imgRepository }} --region ${{ parameters.cloudRunRegion }} --project ${{ parameters.cloudRunProjectId }} --allow-unauthenticated"
- ${{ parameters.postDeploySteps }}
# stages/deploy-cloudrun-stage.yaml
parameters:
# deployCloudRun.yaml parameters
- name: cloudRunProjectId
type: string
- name: imgRepository
type: string
- name: cloudRunRegion
type: string
- name: cloudRunServiceName
type: string
- name: templateResourceName
type: string
default: templates
- name: pipelineResourceName
type: string
default: pipelines
#======= Extensibility ========
- name: preDeploySteps
type: stepList
default: []
- name: postDeploySteps
type: stepList
default: []
- name: stageName
type: string
default: DeployCloudRun
stages:
- stage: ${{ parameters.stageName }}
displayName: 佈署Cloud Run
jobs:
- template: ../jobs/deployCloudRun.yaml
parameters:
cloudRunProjectId: ${{ parameters.cloudRunProjectId }}
imgRepository: ${{ parameters.imgRepository }}
cloudRunRegion: ${{ parameters.cloudRunRegion }}
cloudRunServiceName: ${{ parameters.cloudRunServiceName }}
templateResourceName: ${{ parameters.templateResourceName }}
pipelineResourceName: ${{ parameters.pipelineResourceName }}
preDeploySteps: ${{ parameters.preDeploySteps }}
postDeploySteps: ${{ parameters.postDeploySteps }}
# stages/dotnet-build-stage.yaml
parameters:
# buildCode.yaml parameters
- name: sourcePath
type: string
default: $(Build.SourcesDirectory)
- name: slnOrCsprojName
type: string
default: Pipeline.sln
# buildImage.yaml parameters
- name: artifactName
type: string
default: OutputFiles
- name: unzip
type: boolean
default: false
- name: zipFileName
type: string
default: ''
- name: unzipToFolderPath
type: string
default: ''
- name: imgRepository
type: string
- name: imgTags
type: object
- name: buildDockerfile
type: string
default: Build.Dockerfile
- name: buildContext
type: string
- name: containerRegistry
type: string
- name: startFileName
type: string
- name: templateResourceName
type: string
default: templates
#======= Extensibility ========
- name: imagePreBuildSteps
type: stepList
default: []
- name: imagePostBuildSteps
type: stepList
default: []
- name: codePreBuildSteps
type: stepList
default: []
- name: codePostBuildSteps
type: stepList
default: []
- name: stageName
type: string
default: BuildSourceAndImage
stages:
- stage: ${{ parameters.stageName }}
displayName: 編譯程式碼和建立Docker Image
jobs:
- template: ../jobs/buildCode.yaml
parameters:
sourcePath: ${{ parameters.sourcePath }}
slnOrCsprojName: ${{ parameters.slnOrCsprojName }}
codePreBuildSteps: ${{ parameters.codePreBuildSteps }}
codePostBuildSteps: ${{ parameters.codePostBuildSteps }}
- template: ../jobs/buildImage.yaml
parameters:
templateResourceName: ${{ parameters.templateResourceName }}
dependsOn: BuildCode
artifactName: ${{ parameters.artifactName }}
unzip: ${{ parameters.unzip }}
zipFileName: ${{ parameters.zipFileName }}
unzipToFolderPath: ${{ parameters.unzipToFolderPath }}
imgRepository: ${{ parameters.imgRepository }}
imgTags: ${{ parameters.imgTags }}
buildDockerfile: ${{ parameters.buildDockerfile }}
buildContext: ${{ parameters.buildContext }}
containerRegistry: ${{ parameters.containerRegistry }}
buildArgs: "TARGET_FILE=${{ parameters.startFileName }}"
imagePreBuildSteps: ${{ parameters.imagePreBuildSteps }}
# - script: |
# echo "這是使用的範例1"
# displayName: 範例啦
# - task: Bash@3
# displayName: 還是範例啦
# inputs:
# targetType: 'inline'
# script: |
# echo "這是使用的範例2"
# - ${{ parameters.imagePreBuildSteps }}
imagePostBuildSteps: ${{ parameters.imagePostBuildSteps }}