經由與AWS人員討論，在雲端virtual device的部分並不會使用到Greengrass，而是直接使用AWS Iot Core與thing直接做TLS certificate。因此我們把之前的架構圖，雲端的部份改成以下:

其中Unity的環境就是thing，負責跑虛擬的車子，所以我們可以寫一個kubernetes的POD yaml file如下讓unity_env_creator去動態產生。

unity_deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: unity-thing
  name: unity-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      name: unity-thing
  template:
    metadata:
      labels:
        name: unity-thing
    spec:
      containers:
        - image: $(AWS_ACCOUNT_ID).dkr.ecr.$(AWS_REGION).amazonaws.com/iot-frc-controller:latest
          command: ["/bin/bash", "-c"]
          args: ["source /entrypoint.sh"]
          imagePullPolicy: Always
          name: iot-frc-controller
          env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          envFrom:
          - configMapRef:
              name: thing-config
          - secretRef:
              name: aws-configure-secret
          volumeMounts:
          - mountPath: "/iot-things"
            name: thing-pv
            subPathExpr: $(POD_NAME)
      volumes:
        - name: thing-pv
          emptyDir: {}

其中thing-config是該thing相關env設定變數。

apiVersion: v1
kind: ConfigMap
metadata:
  name: thing-config
data:
  AWS_ACCOUNT_ID: xxxxxxxxxxxx
  AWS_REGION: xxxxxxxxxxxx
  TOPIC: "chatter"
  POLICY_NAME: "policy_to_all_clients"
  MESSAGE: "Test from FRC virtual thing"

aws-configure-secret是aws iam憑證資料，用以設定該POD(container)可以操作aws指令的權限。

apiVersion: v1
kind: Secret
metadata:
  name: aws-configure-secret
type: Opaque
data:
  AWS_ACCESS_KEY_ID:xxxxxxxxxxxxxxxxxxxx
  AWS_SECRET_ACCESS_KEY:xxxxxxxxxxxxxxxxxxxx

另外元件裡面包含Dockerfile, requirements.txt, 與entrypoint.sh。

我們還是以ROS為base來build container環境,並安裝python與awsiotsdk
Dockerfile如下:

ARG ROS_DISTRO=foxy
FROM ros:${ROS_DISTRO}

ENV ROS2_WS /ros_workspace

WORKDIR /app
ADD ./entrypoint.sh /app/entrypoint.sh
RUN apt-get update && apt-get install --no-install-recommends jq curl unzip python-is-python3 -y \
    python3-colcon-common-extensions

WORKDIR ${ROS2_WS}
RUN . /opt/ros/foxy/setup.sh && \
    colcon build --symlink-install

RUN rm /bin/sh && ln -s /bin/bash /bin/sh

RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F42ED6FBAB17C654
RUN apt-get update && apt-get install python3-pip -y
RUN apt-get update && apt-get install ros-$ROS_DISTRO-example-interfaces
RUN python3 -m pip install awsiotsdk

WORKDIR /app
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
RUN unzip awscliv2.zip
RUN sudo ./aws/install

RUN chmod +x /app/entrypoint.sh
RUN pip install --upgrade pip
RUN pip install --upgrade setuptools
COPY requirements.txt /app/
RUN pip install -r /app/requirements.txt

ENTRYPOINT ["/app/entrypoint.sh"]

CMD ["bash"]

requirements.txt裡面只有

AWSIoTPythonSDK

AWSIoTPythonSDK是給元件裡面的程式main.py使用aws iot的python api。

entrypoint.sh如下:

#!/bin/bash
source "$ROS2_WS/install/setup.bash"
mkdir ~/.aws
echo "[default] \
aws_access_key_id=$AWS_ACCESS_KEY_ID \
aws_secret_access_key=$AWS_SECRET_ACCESS_KEY" >> ~/.aws/credentials

echo "[default] \
region=$AWS_REGION \
output=json" >> ~/.aws/config

aws iot create-thing --thing-name ${POD_NAME}
HOST=$(aws iot describe-endpoint --endpoint-type iot:Data-ATS | jq -r '.endpointAddress')
CERTARN=$(aws iot create-keys-and-certificate \
    --set-as-active \
    --certificate-pem-outfile certificate.pem \
    --public-key-outfile public.key \
    --private-key-outfile private.key | jq -r '.certificateArn')
aws iot attach-policy --target ${CERTARN} --policy-name ${POLICY_NAME}
aws iot attach-thing-principal \
    --principal ${CERTARN} \
    --thing-name ${POD_NAME}

mkdir ~/certs
curl -o ~/certs/AmazonRootCA1.pem https://www.amazontrust.com/repository/AmazonRootCA1.pem
chmod 745 ~
chmod 700 ~/certs
chmod 644 ~/certs/AmazonRootCA1.pem

python /app/main.py -e ${HOST} -r ~/certs/AmazonRootCA1.pem -c certificate.pem -k private.key -id ${POD_NAME} -t ${TOPIC} -M "$MESSAGE"

其中$POD_NAME為每次EKS(K8S)環境在create新的virtual device(POD or development)所自動帶入的POD Name。

Reference:
create-keys-and-certificate
Test your device and save the Amazon CA cert