iT邦幫忙

2021 iThome 鐵人賽

DAY 26
0
Software Development

系統與服務雜談系列 第 26

Log Agent - Fluent Bit Service配置與內建 API

Fluent bit回顧
Log Agent - Fluent Bit 簡介
Log Agent - Fluent Bit 安裝與常見架構模式

Service Configuration

Fluent bit的配置起點都預設先讀取fluent-bit.conf這檔案
這配置通常, 第一個Section是Service, 依序是Input、Parser、Filter、Output

配置方式很像INI
都有Section、Key Value, 透過Indent(4spaces)做縮排, 使用#做行註解

[SERVICE]
    # comment, 註解也必須做indent縮排
    daemon    off
    log_level debug
允許空行, 但空行別做indent, 不然會被認為是key
[INPUT]
    name forward
    listen 0.0.0.0
    port 24224

Service Section Properties

  • Flush

    DataPipe的圖中有的Buffer,
    這值用來描述, Buffer內的資料每隔幾秒批量寫到Output元件內
    預設值是5秒

  • Daemon

    是否以daemon後台方式運作
    以Container形式的話就隨意了XD
    預設是off

  • Log_FIle

    Fluent bit服務的log輸出到指定的路徑
    預設是都輸出到stdout

  • Log_Level

    跟很多application一樣, 也能設定什麼等級的log才做輸出
    有error、warning、info、debug、trace
    預設是info

  • Parsers_File

    指定parsers配置檔案的位子
    這配置可以出現多次, 如果你有多個parse.cnf需要被引入的話

[SERVICE]
    # comment, 註解也必須做indent縮排
    daemon    off
    log_level debug
    Parsers_File parsers.conf
    Parsers_File custom_parsers.conf
  • Plugins_File

    跟Parsers_File一樣, 用來引入plugins

  • Health_Check

    啟動health_check, 需要跟http_server搭配
    預設是off

  • HC_Errors_Count

    fluent bit若連續5次error, 才判定為unhealthy
    預設是5次

  • HC_Retry_Failure_Count

    retry failure的次數
    預設是5次

  • HC_Period

    每次retry的時間間隔
    預設60s
    https://ithelp.ithome.com.tw/upload/images/20211011/20104930njexj9xgQU.jpg

  • HTTP_Server

    是否啟用fluent bit內建的http server
    預設是off

  • HTTP_Listen

    啟動監聽的port
    預設是本機的所有網路接口 0.0.0.0

  • HTTP_Port

    監聽請求的TCP port
    預設是2020
    通常開啟http設定, 是用在fluent bit本身要給人或其他監控服務給監控, 才會特地開

內建REST API

來玩一下fluent bit 提供的REST api

version: '3'
services:
  fluentd:
    image: fluent/fluent-bit
    volumes:
        - ./conf:/fluent-bit/etc
    ports:
      - "24224:24224"
      - "24224:24224/udp"
      - "2020:2020" # mapping http port 
    hostname: fluentd
    logging:
      driver: "json-file"
      options:
          max-size: 100m
          max-file: "5"
[SERVICE]
    flush 1
    Daemon off
    log_level error
    #開啟http server
    http_server on
    #監聽所有網卡
    http_listen 0.0.0.0
    #監聽導到port 2020的請求
    http_port 2020
    storage.metrics on

啟動容器
docker-compose up -d
然後先安裝一下Day15提到的JQ套件
因為它回應是JSON, JQ讓Response好閱讀

  • 顯示所有fluent bit的資訊
# 可以看到版本是v1.8.7, 然後是community版
curl -s http://127.0.0.1:2020  | jq              
{
  "fluent-bit": {
    "version": "1.8.7",
    "edition": "Community",
    "flags": [
      "FLB_HAVE_PARSER",
      "FLB_HAVE_RECORD_ACCESSOR",
      "FLB_HAVE_STREAM_PROCESSOR",
      "FLB_HAVE_TLS",
      "FLB_HAVE_OPENSSL",
      "FLB_HAVE_METRICS",
      "FLB_HAVE_AWS",
      "FLB_HAVE_AWS_CREDENTIAL_PROCESS",
      "FLB_HAVE_SIGNV4",
      "FLB_HAVE_SQLDB",
      "FLB_HAVE_METRICS",
      "FLB_HAVE_HTTP_SERVER",
      "FLB_HAVE_SYSTEMD",
      "FLB_HAVE_FORK",
      "FLB_HAVE_TIMESPEC_GET",
      "FLB_HAVE_GMTOFF",
      "FLB_HAVE_UNIX_SOCKET",
      "FLB_HAVE_PROXY_GO",
      "FLB_HAVE_JEMALLOC",
      "FLB_HAVE_LIBBACKTRACE",
      "FLB_HAVE_REGEX",
      "FLB_HAVE_UTF8_ENCODER",
      "FLB_HAVE_LUAJIT",
      "FLB_HAVE_C_TLS",
      "FLB_HAVE_ACCEPT4",
      "FLB_HAVE_INOTIFY"
    ]
  }
}
  • 顯示服務已經啟動了多久
curl -s http://127.0.0.1:2020/api/v1/uptime | jq 
{
  "uptime_sec": 962,
  "uptime_hr": "Fluent Bit has been running:  0 day, 0 hour, 16 minutes and 2 seconds"
}
  • 顯示服務的各項metrics
    因為這裡我還是用上一篇的配置, 所以它input就顯示forward的一些資訊
curl -s http://127.0.0.1:2020/api/v1/metrics/ | jq          
{
  "input": {
    "forward.0": {
      "records": 52,
      "bytes": 10548
    }
  },
  "filter": {
    "record_modifier.0": {
      "drop_records": 0,
      "add_records": 0
    }
  },
  "output": {
    "stdout.0": {
      "proc_records": 52,
      "proc_bytes": 11692,
      "errors": 0,
      "retries": 0,
      "retries_failed": 0,
      "dropped_records": 0,
      "retried_records": 0
    }
  }
}
  • 顯示服務的各項metrics, 但以Prometheus server的格式
    這裡就不是用JSON格式輸出了, 而是Prometheus Text
curl -s http://127.0.0.1:2020/api/v1/metrics/prometheus  
# HELP fluentbit_filter_add_records_total Fluentbit metrics.
# TYPE fluentbit_filter_add_records_total counter
fluentbit_filter_add_records_total{name="record_modifier.0"} 0 1633928614748
# HELP fluentbit_filter_drop_records_total Fluentbit metrics.
# TYPE fluentbit_filter_drop_records_total counter
fluentbit_filter_drop_records_total{name="record_modifier.0"} 0 1633928614748
# HELP fluentbit_input_bytes_total Number of input bytes.
# TYPE fluentbit_input_bytes_total counter
fluentbit_input_bytes_total{name="forward.0"} 10548 1633928614748
# HELP fluentbit_input_records_total Number of input records.
# TYPE fluentbit_input_records_total counter
fluentbit_input_records_total{name="forward.0"} 52 1633928614748
# HELP fluentbit_output_dropped_records_total Number of dropped records.
# TYPE fluentbit_output_dropped_records_total counter
fluentbit_output_dropped_records_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_errors_total Number of output errors.
# TYPE fluentbit_output_errors_total counter
fluentbit_output_errors_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_proc_bytes_total Number of processed output bytes.
# TYPE fluentbit_output_proc_bytes_total counter
fluentbit_output_proc_bytes_total{name="stdout.0"} 11692 1633928614748
# HELP fluentbit_output_proc_records_total Number of processed output records.
# TYPE fluentbit_output_proc_records_total counter
fluentbit_output_proc_records_total{name="stdout.0"} 52 1633928614748
# HELP fluentbit_output_retried_records_total Number of retried records.
# TYPE fluentbit_output_retried_records_total counter
fluentbit_output_retried_records_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_retries_failed_total Number of abandoned batches because the maximum number of re-tries was reached.
# TYPE fluentbit_output_retries_failed_total counter
fluentbit_output_retries_failed_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_output_retries_total Number of output retries.
# TYPE fluentbit_output_retries_total counter
fluentbit_output_retries_total{name="stdout.0"} 0 1633928614748
# HELP fluentbit_uptime Number of seconds that Fluent Bit has been running.
# TYPE fluentbit_uptime counter
fluentbit_uptime 1164
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1633927450
# HELP fluentbit_build_info Build version information.
# TYPE fluentbit_build_info gauge
fluentbit_build_info{version="1.8.7",edition="Community"} 1
  • Health check
    如果服務健康, 則Status是200, body是OK
    反之則是500, body是error
curl -s -I http://127.0.0.1:2020/api/v1/health
HTTP/1.1 200 OK
Server: Monkey/1.7.0
Date: Mon, 11 Oct 2021 05:05:47 GMT
Transfer-Encoding: chunked

---參考文件
Configureation File
Monitoring


上一篇
Log Agent - Fluent Bit 安裝與常見架構模式
下一篇
Log Agent - Fluent Bit Input元件 與 Tail淺談
系列文
系統與服務雜談32

尚未有邦友留言

立即登入留言