淺談OpenTelemetry - Collector Extensions

14th鐵人賽 opentelemetry

雷N

團隊E04

2022-09-16 01:03:03

1365 瀏覽

分享至

昨天講了Receiver, Processor, Exporter 文章連結
Collector這裡還有個模組很重要的
叫做Extensions

OTel Collector Extensions

Extention主要是用來擴充Collector這service, 提供核心流程之外的插件.
內建的有healthcheck, z-pages, pprof, memory_balllast, 或者是auth相關的(basic, oauth2, oidc等等的)

extensions:
  <extension name>:
    endpoint: <network interface and port to bind to, address:port>
    # other key/value pairs as needed by specific extention type
    
service:
  extensions: [<extension name1>, <extension name2>]

舉個例子

# Example of the extensions available with the core Collector. The list below
# includes all configurable options and their respective default value.
extensions:
  health_check:
    port: 13133
  pprof:
    endpoint: "localhost:1777"
    block_profile_fraction: 0
    mutex_profile_fraction: 0
  zpages:
   endpoint: "localhost:55679"

# The service lists extensions not directly related to data pipelines, but used
# by the service.
service:
  # extensions lists the extensions added to the service. They are started
  # in the order presented below and stopped in the reverse order.
  extensions: [health_check, pprof, zpages]

OTel Collector State and Extensions

OTel collector在服務被啟動時, 會遵守下面的狀態的移轉做事.
可以看到它會先讀取configuration file.
然後先建置Extensions, 啟動Extensions.
再來才是處理Pipeline的部份.

關閉服務時, 則是先關閉Shutdown pipeline.
再來才關閉extensions.

因此Extension, 在建置上, 採用Design pattern的Factory pattern, 需要遵守的介面如下

// Factory is implemented by all component factories.
type Factory interface {
	// Type gets the type of the component created by this factory.
	Type() config.Type

	unexportedFactoryFunc()
}

// ExtensionFactory is a factory for extensions to the service.
type ExtensionFactory interface {
	Factory

	CreateDefaultConfig() config.Extension

	CreateExtension(ctx context.Context, set ExtensionCreateSettings, cfg config.Extension) (Extension, error)

	ExtensionStability() StabilityLevel
}

執行上當作插件掛給pipeline也有對應的介面. 用來通知有關聯的pipeline和receiver, extension能用了沒.

Extension執行的方法就跟以前定義的一樣就Start和Shutdown.

// Component is either a receiver, exporter, processor, or an extension.
type Component interface {
	Start(ctx context.Context, host Host) error

	Shutdown(ctx context.Context) error
}

// Extension is the interface for objects hosted by the OpenTelemetry Collector that
// don't participate directly on data pipelines but provide some functionality
// to the service, examples: health check endpoint, z-pages, etc.
type Extension interface {
	Component
}

// PipelineWatcher is an extra interface for Extension hosted by the OpenTelemetry
// Collector that is to be implemented by extensions interested in changes to pipeline
// states. Typically this will be used by extensions that change their behavior if data is
// being ingested or not, e.g.: a k8s readiness probe.
type PipelineWatcher interface {
	Ready() error

	NotReady() error
}

PPRof Extension

Factory實做

連結
每個extension都實做factory在factory.go

Extention實做

連結
有的檔名會是extension.go, 有的是processor.go

運行Collector搭配zPages extension

zPages extension
zPages用來對collector這服務做基本的診斷是很有幫助的
尤其collector其實沒有UI ...
方便我們在collector啟動後, 了解用了什麼組件

otel-collector-config.yaml

receivers:
  otlp:
    protocols:
      grpc:

exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"
    const_labels:
      label1: value1

  logging:

  zipkin:
    endpoint: "http://zipkin-all-in-one:9411/api/v2/spans"
    format: proto

  jaeger:
    endpoint: jaeger-all-in-one:14250
    tls:
      insecure: true

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679

service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, zipkin, jaeger]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, prometheus]

docker-compose.yaml

version: "2"
services:

  # Jaeger
  jaeger-all-in-one:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"
      - "14268"
      - "14250"

  # Zipkin
  zipkin-all-in-one:
    image: openzipkin/zipkin:latest
    ports:
      - "9411:9411"

  # Collector
  otel-collector:
    image: ${OTELCOL_IMG}
    command: ["--config=/etc/otel-collector-config.yaml", "${OTELCOL_ARGS}"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "1888:1888"   # pprof extension
      - "8888:8888"   # Prometheus metrics exposed by the collector
      - "8889:8889"   # Prometheus exporter metrics
      - "13133:13133" # health_check extension
      - "4317:4317"   # OTLP gRPC receiver
      - "55679:55679" # zpages extension
    depends_on:
      - jaeger-all-in-one
      - zipkin-all-in-one

  prometheus:
    container_name: prometheus
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yaml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"