過去的環境是開發人員在 Juypter notebook 上開發,但他們有時候會需要安裝依賴 Ubuntu 的 liberies,而安裝方法就是直接在 Terminal apt install。過去,我們為了讓他們方便開發,會直接用 Docker commit 的方式將開發環境打包成 image,但這段也讓後端協助人員多了跳進去洗頭的工夫。現在為了統一由 Git 管理,我們要開始想辦法將 “Ubuntu 的環境也納入管理”
我原本的想法是新增一個 install.sh ,請開發人員有需要調整就進去修改,但這樣還是有人工的疑慮。有人工就會出錯。
後來想到如果我直接將現有環境的 liberies 找出來,打包時部署進去是否就可以解決了呢?於是找到了 dpkg 指令。但這個指令包含了超多預設的 liberies,抓出來竟有 600 多個。
$ dpkg --get-selections
那會不會能夠找出有哪些環境是額外被建立出來的,是否就可以解決了呢。剛好 apt 就是套件管理工具,他有提供你透過 user 來安裝的套件。這可以比我們原本用 dpkg-query 來抓的 package 從 600 多降到 100 多。
apt-mark showmanual
但這個還是很多,而且當我們必須要從 docker 基本的環境 apt install 的時候,會發現很多套件無法直接安裝
Package docker-ce is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
Package docker-ce-cli is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
Package netcat is a virtual package provided by:
netcat-openbsd 1.219-1
netcat-traditional 1.10-47
E: Unable to locate package containerd.io
E: Couldn't find any package by glob 'containerd.io'
E: Couldn't find any package by regex 'containerd.io'
E: Package 'docker-ce' has no installation candidate
E: Package 'docker-ce-cli' has no installation candidate
E: Unable to locate package gcc-10-base
這時突然臨機一想,官方應該有針對這個狀況給基本的 image,所以查到了 vertex ai 的 workbench 官方套件
gcr.io/deeplearning-platform-release/workbench-container:latest
但我們遇到了另一個問題是有太多相依的 libraries 了,導致安裝一直不成功,於是我決定直接打包現有的 environment 當作環境。所以明天會來進行環境打包。
Reading package lists...
Building dependency tree...
Reading state information...
Package grub-cloud-amd64 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
grub2-common
Package docker-ce is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Unable to locate package containerd.io
E: Couldn't find any package by glob 'containerd.io'
E: Couldn't find any package by regex 'containerd.io'
E: Package 'docker-ce' has no installation candidate
E: Unable to locate package docker-ce-cli
E: Unable to locate package gce-disk-expand
E: Unable to locate package google-cloud-packages-archive-keyring
E: Package 'grub-cloud-amd64' has no installation candidate
E: Unable to locate package libsemanage1
E: Unable to locate package libsepol1
E: Unable to locate package libssl1.1
E: Couldn't find any package by glob 'libssl1.1'
E: Couldn't find any package by regex 'libssl1.1'
E: Unable to locate package linux-headers-5.10.0-32-cloud-amd64
E: Couldn't find any package by glob 'linux-headers-5.10.0-32-cloud-amd64'
E: Couldn't find any package by regex 'linux-headers-5.10.0-32-cloud-amd64'
E: Unable to locate package linux-headers-cloud-amd64
E: Unable to locate package linux-image-cloud-amd64