Day15 開源專案-kube5gnfvo安裝

第 12 屆 iThome 鐵人賽

DAY 15

自我挑戰組

Cloud/SDN/SFC...菜鳥的學習筆記系列第 15 篇

12th鐵人賽

noobUser

團隊NUTC_IMAC_ZERO

2020-09-30 00:26:27

1742 瀏覽

分享至

這是學長研究模擬5g元件可以在VM與容器混合佈署中可實驗，這屬於開源的專案，因此透過這次it鐵人賽來推廣如何使用，有興趣的可以看看。

開始佈署的前置

Kubernetes version為:v1.15.3
git的branch為v2.0.0
最少要兩台機器，一台當master一台當node。
node最低要五core

git clone https://github.com/free5gmano/kube5gnfvo.git

安裝套件

Multus

本身並不提供網路配置，是透過滿足其他 CNI 規範的 Plug-in 進行 container 的網路配置。

cd kube5gnfvo/example/
kubectl apply -f multus-daemonset-pre-1.16.yml

OpenvSwitch

apt install openvswitch-switch -y #安裝ovs
ovs-vsctl add-br br1 #新增bridge

OVS-CNI

cd kube5gnfvo/example/
kubectl apply -f ovs-cni.yaml

用CRD將CNI加入

(與上面啟用ovs的CNI無關)

cat <<EOF >./ovs-net-crd.yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: ovs-net
  annotations:
    k8s.v1.cni.cncf.io/resourceName: ovs-cni.network.kubevirt.io/br1
spec:
  config: '{
      "cniVersion": "0.3.1",
      "type": "ovs",
      "bridge": "br1"
    }'
EOF
kubectl apply -f ovs-net-crd.yaml

Etcd Operator

注意，這部分如果master沒被汙染會Pending。

cd kube5gnfvo/example/etcd-cluster/rbac/
./create_role.sh
cd ..
kubectl apply -f deployment.yaml
(Please make sure that etcdclusters.etcd.database.coreos.com CRD in Kubernetes has been created)
kubectl apply -f ./

Metrics Server

cd kube5gnfvo/example/metrics-server/
kubectl apply -f ./

Node Exporter

可以取得node內的資訊

cd kube5gnfvo/example/
kubectl apply -f prom-node-exporter.yaml

KubeVirt

KubeVirt取得命令的方式

cd kube5gnfvo/example/kubevirt/
kubectl apply -f ./

遇到底下問題

error: unable to recognize "kubevirt-cr.yaml": no matches for kind "KubeVirt" in version "kubevirt.io/v1alpha3"

原因為kubevrit有沒還沒跑起來的在執行一次kubectl apply -f .即可

Quick Start

cat <<EOF >./kube5gnfvo-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: kube5gnfvo-config
data:
  config: |
    ...
      'Here!'
    ...

EOF

kubectl apply -f kube5gnfvo-configmap.yaml

...
'Here!'
...
這部分改將 cat ~/.kube/config 取得的全部貼上去如底下

cat <<EOF >./kube5gnfvo-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: kube5gnfvo-config
data:
  config: |
    apiVersion: v1
    clusters:
      - cluster:
          certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01EZ3lOekV6TlRVME5Gb1hEVE13TURneU5URXpOVFUwTkZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBUHkvClIzY2s0d3NBMlJQbEwzTTdyeTBqTzJhUEpQUWlKazRpVWFoMG5DcEdsS01zeUlnOHZ6bGs5eVRDNWx3QXZyRUwKeVhWUlNwbjQyNGVFVFZUS3M1VEZ5NHlSNlpFeEdsRzZ3TmVHc3BSZkNrelA3RDZNL3RMVmNUZFJqMFp2Q0owawpEZTRvcCtGMlNIVllKVm1Cb3A2cjUwTkkwWDBiKzRvY0V4WTREUEZpSGZ1cDR5c1pWc05XdGhFK205UDJXSGdjCittNS94VjJ0K0NLU21rUG1mZDFNMVh3ZXNjYW5rQlgwKy9PQ1hKbHRTZHNHRUdKY1ZoUE1SOXFlNEFrNU85MkcKSC9qT29RK21NcnBPeWFGbFdiWURoeGZLZVJNakNBTDlhU3dMa3VhWmZ2dTV6Mm44YTF6R0pJNUpaQWNoVGRZRwpWS3NhV2ZkRUFYdmF3bDRIVUE4Q0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFd29heWhzYW56NTR3OFRxT2UwanZuQytLcFMKUkdXL2huQTJnbTU3ZXBmN2JMSU15UHN5WTlIOXhkVDJ4VjM1VXVqb2hpL3NTb3NmajcyU2N1QXZoL1Rkd2ZuQQoxNTU2ZFR4MFd6RUF4ekI4NGtxWlZWcTdFQXExbno1alFBRTN5eS9LMG1OMlRlelNIL3FHd2xaMUhqRnh2bUEyCmE2cmxvdkx3YXpWNWpORkwxRElGcjhONitHQ25sblZMU0VXM3ZWVHc4MzhTRnBNdXhXUE94OHdEbTA4bDlqYTYKdzdyemVDQTQ1eUpjMkxtQ0lUVkVYYlJUMTJLcHluaUlhWEVMa1cySmJ4eVFZa2ZGeUdKSVREOFZCSVo5ZFY1eApJLzBnS3d0Q0l6UFo2eG56UWx1NnorTUwrN0syVStaaU5HcmluV1lHY1NHNGd6R0hTOE5pNVZEdTV0MD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
          server: https://10.0.0.68:6443
        name: kubernetes
    contexts:
      - context:
          cluster: kubernetes
          user: kubernetes-admin
        name: kubernetes-admin@kubernetes
    current-context: kubernetes-admin@kubernetes
    kind: Config
    preferences: {}
    users:
      - name: kubernetes-admin
        user:
          client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJYzNmcVp5ODRFZll3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TURBNE1qY3hNelUxTkRSYUZ3MHlNVEE0TWpjeE16VTFORGRhTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXNqWXMrbnpsUkFYRUZXOUgKOFV5YWNjeXRnOHE1a3Y4YjFrZURSby9CN25FYmlyY1dPaFltZklXRDVLV0QzaXQ4N1VYczZtOG1LTHk2Z241dwp1dUVNUzF4WlZSdUJRWUJpUHRSQkdadVZPczZQUlVlUmRUZFk3ZmhXeUdjMXcrSml4U2hPYWw4NnAwTVRjbjA4Cm5VSDF3eDBtYWZCdmZ0bFhHLzRzSk1IaUpEakJpdDNLSVZ1KzZjSHR4MTgyMUo2dUU1SDRRekZYMmNxUzk4dlcKaGpFNWt2ZitCRko5dzl1NnlZYkRiK1ZOWjFKVng3OHFtUFpQalp2dHluTFJwanNiUDZjT3llTHZTR1NJZmQ3bgpxNGJvNWFZVnhuSmZPakpOdFVmQVJuZERlT3lWUHE1VVlRMEpCNjEyUThZOCswMlJidDNoYkxDMzFEakNadmNBCjlkKzlsd0lEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFFcVJLYzZGRThpcVRHSHQ2dDlJL0xSaWJCczU1ZEdNL0VCdAprbTNWSk5NY1hGNFJEbE51amxGaW1ycG0xQ3VWTnBDQ09jZUNQZ2dFU2VvbkZWSVg3eUJnVWR2V1Nla1p3bEhJCk1qNzRHWHgzVXpORFRKTnVqL3NrZVUyRkI0bGpVRXBsT2Nkalcrd1FhVTM2MHE5VGdsWUpMZEdlSmloN3VoOEoKMHRxSUh3VjNzNSsrSlpXR25VZlRRWVl0NDJZWU9qSTR5ZzZ3RzhNanpFS0ozYTN4YTZIT3ozOExQOGFlNmRDKwpTVm84bGt1MFZKdE9WYU5mUkQ4V0dkMDR2RGgvY0tuai9tM1dab0xoQ1Z6ME1CZVk0dk8zcUVSSWlHVGthamd4CkNPTFJ1WmUyLzNGSEpnWkxNZkpDcmhoZzNmMTRCRi9oRGZLNkhiYmg5V0dhVmZacmpiYz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
          client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBc2pZcytuemxSQVhFRlc5SDhVeWFjY3l0ZzhxNWt2OGIxa2VEUm8vQjduRWJpcmNXCk9oWW1mSVdENUtXRDNpdDg3VVhzNm04bUtMeTZnbjV3dXVFTVMxeFpWUnVCUVlCaVB0UkJHWnVWT3M2UFJVZVIKZFRkWTdmaFd5R2MxdytKaXhTaE9hbDg2cDBNVGNuMDhuVUgxd3gwbWFmQnZmdGxYRy80c0pNSGlKRGpCaXQzSwpJVnUrNmNIdHgxODIxSjZ1RTVINFF6RlgyY3FTOTh2V2hqRTVrdmYrQkZKOXc5dTZ5WWJEYitWTloxSlZ4NzhxCm1QWlBqWnZ0eW5MUnBqc2JQNmNPeWVMdlNHU0lmZDducTRibzVhWVZ4bkpmT2pKTnRVZkFSbmREZU95VlBxNVUKWVEwSkI2MTJROFk4KzAyUmJ0M2hiTEMzMURqQ1p2Y0E5ZCs5bHdJREFRQUJBb0lCQUNmOGtmRU9ieDdwTUVzaQo4Z0NKa0tueUpBakxSd0pkclhCU0VYbVdyZ3dEWXd1bjh4dkxkbzFBSkF6OHVtN3lVUkNvUFBXSlJRV2srekJtCmtJTE9Ga1Vlam9qVEtCaDdady96N2p1QnRtVkFqZE1jc04yZE1wNEt0K1kxaWxIT2pSeng0MDVvMmE5SUZLZisKdzhnQlVkU2F5TkRDZXpINk1GcCttSWNTM2R3RVpFWm5SVWFoMnJuODdvNjl2TTdSMTM5SWtTd1I0cnFWRzYzRgpvQzhmc2djRzR1d1VBUnhHekN2QmdGK1BOakRBKzFXZkdMOGYrV0F1VXhpUGg0Ykl1dXJkQjM3TFVoRlpYcm9UCkNOb2JSYm94a0ZxNmtKeWtiOHkzRm92UHFZa0VkemxQS3MyM2FPMWhLa1BuVlU2S1Z4OGFtMmU3bXBKOWVVMngKQkVnQ0x5a0NnWUVBMUhyYnl5dDZrc1dnbWNGRFFXS3VHS0tkalRVSU9jMG9WWDVSd21WdDdkUURFMUY3eDhlbQowSkVqampNY0lzMDU4T29kdnNoU3dFN0NoYXBJTDh4T2hWZ0FFNXlwOUdlQnJWSEd3cnRYUW9HV3dOQThyZmJhCkJOTG9RV0NtVm1MMEVTTEw2RjNXMElzRzRBbFM4blN0R3FJeEo0YmlwZ3hNbWNKeEYyQW1VdXNDZ1lFQTFyWi8KbHlKMlBXVnYvWDA3OG5EbXZmd1dNNEptV1Ric1ZCZEkyTHVROGtBNGFaeHpzODNkakcyWVkrMXMyY0ZEUEhMbgoxWmFuZkR0c3hQek1RWnZONFpiT08zaW92NXMrQ3h2b2RsS3pwZktRdE5RSmhRN1NLR0VDSk5NNE9iRHlNVDdzCkRCTEZrcXRmLzlTVVljM1JpOGwrVzJVOUw2TEZVcUZpUnM5U25RVUNnWUJyMTl1Y3QvbWphdmtEY09yZzBlUUoKRHVnbVZtZFJad1owdGdOclJpM1FTdzR3ak9mbFV6a2duTG5PSDdURkdTS3lXekV6dHRzQlRqZyt2Vm1jSVBjdAppS2VwVEplYU83cStyRVhVYWw5MG11OVg5NWtFdHNpYk1DeS9tZjF5MVJDSmJmN09kbXBrT0Z3Sjg5R015NWxjCmJqdkNreHlRS0tmc1Zwc3R3UmhTZHdLQmdRREIvYWxSbDZWVTIyMVBqZkszblNJY3pNSVNUK0xCQUVxTnYydmwKeGJHejdpb0o0WHo0R3l2c1VLaTZ3Q3owb1JScWJUYmhWM2lEV2Yxd0RTZE9yOEpYNjdUeExwYm9adzZyWXMrbworQmV2KzNTYXJHamRld0tOMFBmcTVadUxKRC91Rm5lV2dPOVpFTlZ2ZzFWOCtFUTNGQm5CNGJwaWE2SDBnak1VCjN3UTVNUUtCZ0hVT3pJWW1HYldaMXhXWnJCRmlja0dEbjJ4Q1FWSjRCa2hFQnJhT1kxNTd1bUlNRHVWYU9GSFgKU0dkK0lidG1RRHoxdHc3TDE0TkZGVFB5M0ZDTlpzV0ZMdTVBZkhidFpZM2lpRVFocUp5RWlLOEZvT1IyRDBwUQpTamYrTldaL20ybzF0ZzNIRUN0dDBrOFVadnhialN5bFN1eSt0T0JLRmcwQTNSdFZDUUY2Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
EOF

Create kube5gnfvo ServiceAccount

cat <<EOF >./kube5gnfvo-sa.yaml
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: kube5gnfvo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: kube5gnfvo
  namespace: default
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube5gnfvo
EOF

kubectl apply -f kube5gnfvo-sa.yaml

Deploy Mysql Database

cat <<EOF >./kube5gnfvo-mysql.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube5gnfvo-mysql
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: kube5gnfvo-mysql
  template:
    metadata:
      labels:
        app: kube5gnfvo-mysql
    spec:
      containers:
      - image: mysql:5.6
        name: kube5gnfvo-mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: kube5gnfvo-mysql
          mountPath: /var/lib/mysql
        volumeMounts:
        - name: mysql-initdb
          mountPath: /docker-entrypoint-initdb.d
      volumes:
      - name: kube5gnfvo-mysql
        persistentVolumeClaim:
          claimName: kube5gnfvo-mysql
      volumes:
      - name: mysql-initdb
        configMap:
          name: mysql-initdb-config
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: kube5gnfvo-mysql
  labels:
    name: kube5gnfvo-mysql
spec:
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    type: DirectoryOrCreate
    path: /mnt/kube5gnfvo-mysql
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kube5gnfvo-mysql
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  selector:
    matchExpressions:
    - key: name
      operator: In
      values: ["kube5gnfvo-mysql"]
---
apiVersion: v1
kind: Service
metadata:
  name: kube5gnfvo-mysql
spec:
  ports:
  - port: 3306
  selector:
    app: kube5gnfvo-mysql
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-initdb-config
data:
  initdb.sql: |
    CREATE DATABASE kube5gnfvo;
EOF

kubectl apply -f kube5gnfvo-mysql.yaml

Deploy kube5gnfvo

cat <<EOF >./kube5gnfvo.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube5gnfvo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube5gnfvo
  template:
    metadata:
      labels:
        app: kube5gnfvo
    spec:
      serviceAccountName: kube5gnfvo
      containers:
      - image: free5gmano/kube5gnfvo-stage2
        name: kube5gnfvo
        env:
        - name: DATABASE_PASSWORD
          value: "password"
        - name: DATABASE_HOST
          value: "kube5gnfvo-mysql"
        - name: DATABASE_PORT
          value: "3306"
        command: ["/bin/sh","-c"]
        args: ['python3 manage.py migrate && python3 manage.py runserver 0:8000']
        ports:
        - containerPort: 8000
          name: kube5gnfvo
        volumeMounts:
        - name: kube5gnfvo-vnf-package
          mountPath: /root/NSD
          subPath: NSD
        - name: kube5gnfvo-vnf-package
          mountPath: /root/VnfPackage
          subPath: VnfPackage
        - name: kube-config
          mountPath: /root/config
          subPath: config
      volumes:
      - name: kube5gnfvo-vnf-package
        persistentVolumeClaim:
          claimName: kube5gnfvo-pvc
      - name: kube-config
        configMap:
          name: kube5gnfvo-config
          items:
          - key: config
            path: config
---
apiVersion: v1
kind: Service
metadata:
  name: kube5gnfvo
spec:
  type: NodePort
  ports:
  - port: 8000
    nodePort: 30888
  selector:
    app: kube5gnfvo
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kube5gnfvo-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  selector:
    matchExpressions:
    - key: name
      operator: In
      values: ["kube5gnfvo"]
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: kube5gnfvo-pv
  labels:
    name: kube5gnfvo
spec:
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    type: DirectoryOrCreate
    path: /mnt/kube5gnfvo
EOF

kubectl apply -f kube5gnfvo.yaml

到這裡就代表佈署部分好了，明天將講使用方式，其實安裝除了一些注意部分之外，其他都在github有介紹，有一些部分有相關套件的連結。