iT邦幫忙

2022 iThome 鐵人賽

DAY 14
0
DevOps

IoT Cloud Computing on robotic vehicle系列 第 14

[DEMO] 實體小車的遠端操控 + WebRTC

  • 分享至 

  • xImage
  •  

為了要讓用戶們都可以看到整個DEMO雲端操控地端的過程.我們在地端架設Camera,並透過WebRTC(peer to peer)將影像傳至其他一樣透過browser瀏覽的用戶。
https://ithelp.ithome.com.tw/upload/images/20220929/20005722leYUq8noZr.jpg

在server端使用nodejs與peer library架設WebRTC server

const server = require("https").Server(app);
const { ExpressPeerServer } = require("peer");
const peerServer = ExpressPeerServer(server, {
  debug: true,
});
const io = require("socket.io")(server, {
  cors: {
    origin: '*' /* 先開放所有網域可呼叫,之後可設定白名單 */
  }
});

app.use("/peerjs", peerServer);

io.on("connection", (socket) => {
  socket.on("join-room", (roomId, userId) => {
    socket.join(roomId);
    socket.to(roomId).emit("user-connected", userId);
    socket.on("message", (message) => {
      io.to(roomId).emit("createMessage", message, userId);
    });
  });
});

因為WebRTC是peer to peer,在client端有兩個部份;一個部份是edge端(edge.js)的影像獲取,另一個部份是真正進行monitor(client.js)的用戶端的呈現。

edge.js內容如下:

<script src="https://unpkg.com/peerjs@1.3.1/dist/peerjs.min.js"></script>
<div class="videos__group">
    <div id="video-grid">
    </div>
    <select id="videoSelect" class="options__select" onchange="changeVideoSource(this.value)">
    </select>
</div>
<script>      
const ROOM_ID = "edge";
const socket = io("/");
const videoGrid = document.getElementById("video-grid");
const videoSelect = document.getElementById('videoSelect');
const myVideo = document.createElement("video");
myVideo.muted = true;
const peer = new Peer(undefined, {
  path: "/peerjs",
  host: "/",
  port: "443",
});

document.addEventListener("DOMContentLoaded", async function(){
  let myVideoStream;
  let devices = await navigator.mediaDevices.enumerateDevices();

  if (devices.length > 0) {
    let localContraints = { audio: true }
    let isHasVideoDevice = false
    for (const device of devices) {
      if (device.kind === "videoinput") {
        localContraints.video = { deviceId: device.deviceId ? { exact: device.deviceId } : undefined };
        isHasVideoDevice = true;

        var opt = document.createElement('option');
        opt.value = device.deviceId;
        opt.innerHTML = device.label;
        videoSelect.appendChild(opt);
        videoSelect.value = device.deviceId;
      }
    }
    if (isHasVideoDevice == false) {
      alert(`No Video Devices Available`);
    } else {
      generateVideoStream(localContraints)
    }
  }
  else {
    alert(`No Devices Available`);
  }
})

const generateVideoStream = (localContraints) => {
  navigator.mediaDevices
  .getUserMedia(localContraints)
  .then((stream) => {
    myVideoStream = stream;
    addVideoStream(myVideo, stream);
    peer.on("call", (call) => {
      call.answer(stream);
    });

    socket.on("user-connected", (userId) => {
      const call = peer.call(userId, stream);
    });
  });
}

const changeVideoSource = (val) => {
  if (val) {
    let localContraints = { audio: true }
    localContraints.video = { deviceId: val ? { exact: val } : undefined };
    generateVideoStream(localContraints)
  }
}

peer.on("open", (id) => {
  socket.emit("join-room", ROOM_ID, id);
});

const addVideoStream = (video, stream) => {
  video.srcObject = stream;
  video.addEventListener("loadedmetadata", () => {
    video.play();
    videoGrid.append(video);
  });
};
</script>

另外client.js如下:

<div class="videos__group">
 <div id="video-grid">
 </div>
</div>
<script>
const ROOM_ID = "edge";
const socket = io("/");
const videoGrid = document.getElementById("video-grid");
var peer = new Peer(undefined, {
  path: "/peerjs",
  host: "/",
  port: "443",
});

peer.on("call", (call) => {
  call.answer(null);
  const video = document.createElement("video");
  video.muted = true;
  call.on("stream", (userVideoStream) => {
    addVideoStream(video, userVideoStream);
  });
});

socket.on("user-connected", (userId) => {
  connectToNewUser(userId, null);
});

const connectToNewUser = (userId, stream) => {
  const call = peer.call(userId, stream);
  const video = document.createElement("video");
  call.on("stream", (userVideoStream) => {
    addVideoStream(video, userVideoStream);
  });
};

peer.on("open", (id) => {
  socket.emit("join-room", ROOM_ID, id);
});

const addVideoStream = (video, stream) => {
  video.srcObject = stream;
  video.addEventListener("loadedmetadata", () => {
    video.play();
    videoGrid.append(video);
  });
};
</script>

有關WebRTC的完整實作,可以參考這份程式碼

操作時在web介面上先是點擊publish initialpose,再點擊publish clicked_point,就可以在地端的RVIZ呈現出要車子前往的point(紛紅色的點)所在地。
https://ithelp.ithome.com.tw/upload/images/20220929/20005722qZACRuL4Jt.jpg

如此,雲端與地端就可以透過Web完整控制edge device,並且將監控畫面傳至用戶端。


上一篇
[前置] 前端ROS Topic資料取得與呈現
下一篇
[中場小結] 環境的佈建
系列文
IoT Cloud Computing on robotic vehicle30
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言