iT邦幫忙

2022 iThome 鐵人賽
DAY 26
1
DevOps

那些關於 docker 你知道與不知道的事系列 第 26 篇

Day 26: 真的過橋了!

14th鐵人賽 docker docker network tcpdump iptables
1147 瀏覽

  • 分享至 

  • xImage
    •  

昨天我們做這樣的架構:
https://ithelp.ithome.com.tw/upload/images/20221011/20151857swxXZDbe9X.png

卻發現無法地成功從 ns1 ping 到 ns2，上網查了一下，有的說是因為 veth0 跟 veth1 處於同一個網段，且第一次連接，所以會先發 ARP 封包，但另外一端無法回覆造成的，但嘗試了該篇文章提供的做法，也還是無法成功，最後找到了一個有效的解法，原來是跟 iptables 有關，原本的 iptables 是這樣:

ubuntu@ip-xxx:~$ sudo iptables -L --line-numbers
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination

# 是這裡有問題
Chain FORWARD (policy DROP) 
num  target     prot opt source               destination
1    DOCKER-USER  all  --  anywhere             anywhere
2    DOCKER-ISOLATION-STAGE-1  all  --  anywhere             anywhere
3    ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
4    DOCKER     all  --  anywhere             anywhere
5    ACCEPT     all  --  anywhere             anywhere
6    ACCEPT     all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination

Chain DOCKER (1 references)
num  target     prot opt source               destination

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num  target     prot opt source               destination
1    DOCKER-ISOLATION-STAGE-2  all  --  anywhere             anywhere
2    RETURN     all  --  anywhere             anywhere

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
num  target     prot opt source               destination
1    DROP       all  --  anywhere             anywhere
2    RETURN     all  --  anywhere             anywhere

Chain DOCKER-USER (1 references)
num  target     prot opt source               destination
1    RETURN     all  --  anywhere             anywhere

只需要去修改一下 iptables 的 FORWORD 即可:

ubuntu@ip-xxx:~$ sudo iptables --policy FORWARD ACCEPT

修改完成後，來測試連接看看:

# 直接執行 bash 進入 ns1
ubuntu@ip-xxx:~$ sudo ip netns exec ns1 bash

root@ip-xxx:/home/ubuntu# ping 172.18.0.3 -c 3
PING 172.18.0.3 (172.18.0.3) 56(84) bytes of data.
64 bytes from 172.18.0.3: icmp_seq=1 ttl=64 time=0.063 ms
64 bytes from 172.18.0.3: icmp_seq=2 ttl=64 time=0.058 ms
64 bytes from 172.18.0.3: icmp_seq=3 ttl=64 time=0.060 ms

--- 172.18.0.3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2028ms
rtt min/avg/max/mdev = 0.058/0.060/0.063/0.002 ms

果然可以成功 ping 到了！

補充一下，觀察 iptables FORWARD chain 裡的規則，似乎都跟 docker 有關，為此我開了一台新的 EC2，一樣是安裝 ubuntu 20.04，在完全乾淨的環境下，按照昨天的指令建立完成後，是直接可以從 ns1 去 ping ns2 的
172.18.0.3，之後再來研究為什麼 docker 要加上 FORWARD 且 POLICY 是 DROP 的規則吧。

既然已經連通了，那我就想來錄個封包看看：

A. 在 ns1 裡錄 veth0

root@ip-xxxx:/home/ubuntu# tcpdump -i veth0 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:15:44.734020 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
15:15:44.734100 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64
15:15:45.764223 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 2, length 64
15:15:45.764299 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 2, length 64
15:15:46.788183 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 3, length 64
15:15:46.788239 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 3, length 64
15:15:49.764167 ARP, Request who-has 172.18.0.3 tell 172.18.0.2, length 28
15:15:49.764281 ARP, Request who-has 172.18.0.2 tell 172.18.0.3, length 28
15:15:49.764293 ARP, Reply 172.18.0.2 is-at 9a:b1:13:c0:48:2a, length 28
15:15:49.764302 ARP, Reply 172.18.0.3 is-at ea:8c:3d:7e:90:de, length 28

10 packets captured
10 packets received by filter
0 packets dropped by kernel

B. 在 host 錄 veth0-br

ubuntu@ip-xxx:~$ sudo tcpdump -i veth0-br -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth0-br, link-type EN10MB (Ethernet), capture size 262144 bytes
15:15:44.734028 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
15:15:44.734098 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64
15:15:45.764236 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 2, length 64
15:15:45.764297 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 2, length 64
15:15:46.788192 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 3, length 64
15:15:46.788238 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 3, length 64
15:15:49.764276 ARP, Request who-has 172.18.0.2 tell 172.18.0.3, length 28
15:15:49.764179 ARP, Request who-has 172.18.0.3 tell 172.18.0.2, length 28
15:15:49.764293 ARP, Reply 172.18.0.2 is-at 9a:b1:13:c0:48:2a, length 28
15:15:49.764301 ARP, Reply 172.18.0.3 is-at ea:8c:3d:7e:90:de, length 28
^C
10 packets captured
10 packets received by filter
0 packets dropped by kernel

C. 在 host 錄 veth1-br

ubuntu@ip-xxxx:~$ sudo tcpdump -i veth1-br -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth1-br, link-type EN10MB (Ethernet), capture size 262144 bytes
15:15:44.734068 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
15:15:44.734094 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64
15:15:45.764265 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 2, length 64
15:15:45.764291 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 2, length 64
15:15:46.788214 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 3, length 64
15:15:46.788234 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 3, length 64
15:15:49.764162 ARP, Request who-has 172.18.0.2 tell 172.18.0.3, length 28
15:15:49.764284 ARP, Request who-has 172.18.0.3 tell 172.18.0.2, length 28
15:15:49.764299 ARP, Reply 172.18.0.2 is-at 9a:b1:13:c0:48:2a, length 28
15:15:49.764298 ARP, Reply 172.18.0.3 is-at ea:8c:3d:7e:90:de, length 28

10 packets captured
10 packets received by filter
0 packets dropped by kernel

D. 在 ns2 裡錄 veth0

root@ip-xxxx:/home/ubuntu# tcpdump -i veth1 -nn
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:15:44.734074 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
15:15:44.734092 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64
15:15:45.764272 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 2, length 64
15:15:45.764290 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 2, length 64
15:15:46.788219 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 3, length 64
15:15:46.788234 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 3, length 64
15:15:49.764151 ARP, Request who-has 172.18.0.2 tell 172.18.0.3, length 28
15:15:49.764285 ARP, Request who-has 172.18.0.3 tell 172.18.0.2, length 28
15:15:49.764297 ARP, Reply 172.18.0.3 is-at ea:8c:3d:7e:90:de, length 28
15:15:49.764300 ARP, Reply 172.18.0.2 is-at 9a:b1:13:c0:48:2a, length 28

10 packets captured
10 packets received by filter
0 packets dropped by kernel

這樣分開看有點亂，我們把第一個 ICMP 的 req/res 放在一起看，我們把 seq 是 1 都找出來，並且按照時間排序，我另外標注了是在哪裡錄到的：

ns1 15:15:44.734020 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
br0 15:15:44.734028 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
br1 15:15:44.734068 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
ns2 15:15:44.734074 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 5350, seq 1, length 64
ns2 15:15:44.734092 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64
br1 15:15:44.734094 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64
br0 15:15:44.734098 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64
ns1 15:15:44.734100 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 5350, seq 1, length 64

是不是可以看到封包的流向是按照下圖在傳遞的呢：
https://ithelp.ithome.com.tw/upload/images/20221011/20151857vMOZRYSBt8.png

今天我們已經可以透過 bridge 去讓兩個不同的 net namespace 中的網路介面互相連通了，明天來讓我們看看怎麼樣可以外部通訊吧～


上一篇
Day 25: 來過橋吧！
下一篇
Day 27: 讓我們出海去...欸，等一下，只能先國內旅遊...
系列文
那些關於 docker 你知道與不知道的事 共 32 篇
  1. 28
    Day 28: 為什麼出不了國呢？
  2. 29
    Day 29: 讓我們出國去
  3. 30
    Day 30: 讓我們關掉無條件轉發吧 & 完賽感想
  4. 31
    Day 31: Docker 是怎麼解封國境的呢？
  5. 32
    Day 32: 換我們的 namespace 解封國境
完整目錄
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言
iThome鐵人賽
參賽組數
1064 組
團體組數
40 組
累計文章數
22052 篇
完賽人數
594 人
iT+看影片追技術 看影片追技術 看更多
熱門tag 看更多
15th鐵人賽 16th鐵人賽 13th鐵人賽 14th鐵人賽 12th鐵人賽 11th鐵人賽 鐵人賽 2019鐵人賽 javascript 2018鐵人賽 python 2017鐵人賽 windows php c# windows server linux css react vue.js
熱門問題
熱門回答
熱門文章
IT邦幫忙