iT邦幫忙

2021 iThome 鐵人賽

DAY 25
0
Software Development

Redis還在學系列 第 25

Day25 Redis架構實戰-Sentinel選取Replica機制

Replica選擇切換機制

  • 先剔除不健康的Replica

    • Replica與Master失去連線時間,或是資料落後Master太久的都會優先背剔除
  • 依照Replica-priority進行排序

    • 依照Redis.conf中設定的組態,越小越優先
  • 資料同步複製處理之同步差

    • 如果Replica-priority相同,則會選擇與Master資料同步差異最小的優先選擇
  • RunID

    • 如果Replica-priority相同且資料同步差也相同,則選擇RunID最小的.
    • RunID越小代表Redis Server越先啟動的.
  • 在開始測試Redis Server異常自動切換Master前,先確認目前Sentinel叢集狀態.

# 連線到sentinel叢集
# 目前監控的redis server master是mymaster中的 IP: 127.0.0.1 Port: 6380
# Sentinel 26379 狀態
./redis-cli -h 127.0.0.1 -p 26379
127.0.0.1:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=127.0.0.1:6380,slaves=2,sentinels=3

# Sentinel 26380 狀態
./redis-cli -h 127.0.0.1 -p 26380
127.0.0.1:26380> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=127.0.0.1:6380,slaves=2,sentinels=3

# Sentinel 26381 狀態
./redis-cli -h 127.0.0.1 -p 26381
127.0.0.1:26381> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=127.0.0.1:6380,slaves=2,sentinels=3
  • 連線到目前Redis Server Master確認目前狀態.
# 連線至目前Redis Server (Master) 6380
./redis-cli -h 127.0.0.1 -p 6380
127.0.0.1:6380> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6381,state=online,offset=3204281,lag=0
slave1:ip=127.0.0.1,port=6379,state=online,offset=3204281,lag=0
master_failover_state:no-failover
master_replid:ea3bc11d99568d87b0496fb83c6f2ccc58425baa
master_replid2:a30878092cdd31e404af76eaa7a2d024c8ae843e
master_repl_offset:3204281
second_repl_offset:19299
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2155706
repl_backlog_histlen:1048576
  • 找出目前Redis Server Master並讓其異常終止.
# 
ps -ef|grep redis-server

# kill
kill -9 409413
  • 終止目前Redis Server Master Process

https://ithelp.ithome.com.tw/upload/images/20211010/20111658yraKWkA8Rr.png

  • Redis Server Replica 6379 log顯示內容

https://ithelp.ithome.com.tw/upload/images/20211010/201116580bUrL8VSL7.png

  • Redis Server Replica 6381 log顯示內容

https://ithelp.ithome.com.tw/upload/images/20211010/20111658SNUh1nCKvv.png

  • Sentinel 26379 log 顯示偵測到Redis Server Master異常並進行Replica選舉,切換Replica為Master.

https://ithelp.ithome.com.tw/upload/images/20211010/20111658NGVilW0EuL.png

  • Sentinel 已將Redis Server Replica 6381切換為Master.
# 原Redis Server Master 6380 已偵測到異常
+sdown master mymaster 127.0.0.1 6380

# 投票選取Sentinel 此次進行切換的Leader
+vote-for-leader 6337212980ee71b6db0222c50ad525c94c6c48a8 1

# 選取Sentinel 26380為此次進行切換的Leader
+config-update-from sentinel 6337212980ee71b6db0222c50ad525c94c6c48a8 127.0.0.1 26380 @ mymaster 127.0.0.1 6380

# 切換Redis Server Master 6381為新的Master取代原有的
switch-master mymaster 127.0.0.1 6380 127.0.0.1 6381

https://ithelp.ithome.com.tw/upload/images/20211010/20111658jwVmlyVCZp.png

  • Redis Server Master 6381 顯示已成為Master.

https://ithelp.ithome.com.tw/upload/images/20211010/20111658vGluwurhKw.png

  • Redis Server Replica 6379 與新的Redis Server Master 6381 建立同步.

https://ithelp.ithome.com.tw/upload/images/20211010/20111658nD5SlecJLf.png

  • 將Redis Server 6380 復原重新加入叢集.
Connecting to MASTER 127.0.0.1:6381
Full resync from master: c2f5749e8daef12058d98addb48e954afbc92698:1306638

https://ithelp.ithome.com.tw/upload/images/20211010/20111658RhnTTa0Qor.png

  • Sentinel 會將 Redis Server 6380角色切換為Replica.

https://ithelp.ithome.com.tw/upload/images/20211010/20111658CjI9ohoYam.png

  • 確認結果是否符合異常自動切換的高可用情境.
# 確認Redis Server 6380角色為Replica
./redis-cli -h 127.0.0.1 -p 6380
127.0.0.1:6380> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6381
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:1377655
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:c2f5749e8daef12058d98addb48e954afbc92698
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1377655
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1306639
repl_backlog_histlen:71017

# 確認Sentinel與Redis Server都已恢復
./redis-cli -h 127.0.0.1 -p 26379
127.0.0.1:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=127.0.0.1:6381,slaves=2,sentinels=3

上一篇
Day24 Redis架構實戰-Sentinel叢集架構
下一篇
Day26 Redis架構實戰-Redis叢集架構/Gossip協議
系列文
Redis還在學30

尚未有邦友留言

立即登入留言