iT邦幫忙

2018 iT 邦幫忙鐵人賽
0
自我挑戰組

成長型維運平台 - 單人新手到多人團隊都適用的工具系列 第 26

1-16.監控工具之一:Zabbix -> Elasticsearch

  • 分享至 

  • xImage
  •  

目前還是測試版,千萬不要用在正式環境!!

Zabbix 3.4.5版本支援data存至elasticsearch
Zabbix has recently started to support storage of historical data by means of Elasticsearch instead of a database.
https://www.zabbix.com/documentation/3.4/manual/appendix/install/elastic_search_setup

推測應該跟Type of information資料型態有關
uint:Numeric (unsigned),dbl:Numeric (float)

vi /etc/zabbix/zabbix_server.conf
HistoryStorageURL=http://172.16.1.1:9200
HistoryStorageTypes=uint,dbl,str,log,text

不確定路徑可以用find / -name zabbix.conf.php

vi /etc/zabbix/web/zabbix.conf.php
global $DB, $HISTORY;
$HISTORY['url']   = 'http://172.16.1.1:9200';
$HISTORY['types'] = ['uint','str', 'text', 'log'];

重啟zabbix server即可

設定elasticsearch上zabbix資料的型態
(錯了要把資料全清掉才能重做)

curl -XPUT 172.16.1.1:9200/text -d'

{
 "settings" : {
    "index" : {
       "number_of_replicas" : 1,
       "number_of_shards" : 5
    }
 },
 "mappings" : {
    "values" : {
       "properties" : {
          "itemid" : {
             "type" : "long"
          },
          "clock" : {
             "format" : "epoch_second",
             "type" : "date"
          },
          "value" : {
             "fields" : {
                "analyzed" : {
                   "index" : true,
                   "type" : "text",
                   "analyzer" : "standard"
                }
             },
             "index" : false,
             "type" : "text"
          }
       }
    }
 }
}
'

範本

# uint mapping

{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "type" : "long"
            }
         }
      }
   }
}


# dbl mapping

{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "type" : "double"
            }
         }
      }
   }
}


# str mapping

{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "fields" : {
                  "analyzed" : {
                     "index" : true,
                     "type" : "text",
                     "analyzer" : "standard"
                  }
               },
               "index" : false,
               "type" : "text"
            }
         }
      }
   }
}


# text mapping

{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "fields" : {
                  "analyzed" : {
                     "index" : true,
                     "type" : "text",
                     "analyzer" : "standard"
                  }
               },
               "index" : false,
               "type" : "text"
            }
         }
      }
   }
}


# log mapping

{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "fields" : {
                  "analyzed" : {
                     "index" : true,
                     "type" : "text",
                     "analyzer" : "standard"
               },
               "index" : false,
               "type" : "text"
            }
         }
      }
   }
}

Kibana顯示,格式與SQL相同,要知道itemid才能做圖表
https://ithelp.ithome.com.tw/upload/images/20180108/20077752udS2qQCXjg.png

繪圖,與1-12相同的itemid
https://ithelp.ithome.com.tw/upload/images/20180108/20077752TW7LRNliSX.png

Grafana繪圖 (Display的Null value 要選 Connected 不然圖繪顯示不出來)
https://ithelp.ithome.com.tw/upload/images/20180108/20077752FJNiTUSh4P.png

小插曲
為了要測試找了一台3.2升級3.4,server啟動成功,查看log也沒異常,可是網頁開不起來
System error occurred. Please contact Zabbix administrator.

vi /var/log/zabbix/zabbix_server.log

  2247:20180108:145759.671 database connection re-established
  2247:20180108:145759.702 current database version (mandatory/optional): 03040000/03040005
  2247:20180108:145759.702 required mandatory version: 03040000
  2247:20180108:145801.017 server #0 started [main process]
  2542:20180108:145801.018 server #1 started [configuration syncer #1]

懷疑連網頁至DB問題,檢查conf設定,確定設定皆正常

vi /etc/zabbix/zabbix_server.conf
find / -name zabbix.conf.php
vi /etc/zabbix/web/zabbix.conf.php

刪除所有zabbix檔案 (包含/usr/share/zabbix/及/etc/zabbix/ 底下手動rm刪除)
重新安裝後正常運作


上一篇
5.1-數據分析-安裝anaconda
下一篇
3-7.監控工具之三:Elastic + tshark 封包分析
系列文
成長型維運平台 - 單人新手到多人團隊都適用的工具27
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

1 則留言

0
ted00132
iT邦新手 5 級 ‧ 2020-09-23 16:32:17

真的很有趣,不知道大大有沒有新的突破,畢竟在DBrank中 Elasticsearch算Search Engin不算傳統的RDB,不過大大寫的超好若再加上這兩年的新經驗,應該能成就一本媲美Oreilly等級的電腦書。

Eugene iT邦研究生 5 級 ‧ 2020-09-23 18:34:08 檢舉

學Splunk將所有資料放到同一個平台,方便關聯分析,不過很浪費資源~
很多內容是參考對岸論壇文章實作,遇到困難時可以學習參考

ted00132 iT邦新手 5 級 ‧ 2020-09-24 10:26:12 檢舉

學Splunk將所有資料放到同一個平台,方便關聯分析,不過很浪費資源~

原來如此阿,我早年也是做數據分析出來的,這確實是很實際的作法

很多內容是參考對岸論壇文章實作

大大太謙虛了,網路文章常都是斷斷續續能萃取整合成一篇完整論述甚至提出自己的觀點本身就是一種強大的能力

我要留言

立即登入留言