fastapi

https://signoz.io/blog/opentelemetry-fastapi/

OpenTelemetry

opentelemetry.io

High-quality, ubiquitous, and portable telemetry to enable effective observability

Tempo

Tempo is the tracing puzzle of the Grafana observability portfolio.
Ref: Trace — Log Correlation with Grafana Tempo

Concept

Youtube:初探 OpenTelemetry 工具組：蒐集遙測數據的新標準

Note of Video

初探 OpenTelemetry 工具組：蒐集遙測數據的新標準

(1)video: Signoz Service Map

(2)video: Metrics

(3)video: Traces

Drill Down
(3-1)
點入其中一個Trace ID

(3-2)
點入其中一個Span ID

detail of sql query

Steps

(1) 確認Local可生成監控數據(trace/log/metrics)
(2) k8s Application deploy setting
(3) Grafana or others(ex:SigNoz) 呈現 use tempo (datasource)

Relations

Example Application (OpenTelemetry Agent) <- (OTLP/gRPC)- OpenTelemetry Collector <- Tempo <- (Grafana|Others)

Python

• opentelemetry-instrument is OpenTelemetry Agent
• OTLP is exporter
• OpenTelemetry Collector(port:4317)

:::danger
(需重寫以下...for Python解說)

需新增: dockerfile adjustment, Basic(不改動source code) vs Advanced(動source code)
:::

OpenTelemetry(OTel) with Python

官網參考_Python Application adopt OpenTelemetry

(1) Basic - Run the instrumented app

• Automatic instrumentation

  Automatic instrumentation captures telemetry at the edges of your systems, such as inbound and outbound HTTP requests.

• opentelemetry-distro package

  Install the opentelemetry-distro package, which contains the OpenTelemetry API, SDK and also the tools opentelemetry-bootstrap and opentelemetry-instrument...

(2) Advanced - Modify Code for manual instrumentation to know what’s going on in the application

Code Snippet - Traces
:::spoiler

from random import randint
from flask import Flask

from opentelemetry import trace

# Acquire a tracer
tracer = trace.get_tracer("diceroller.tracer")

app = Flask(__name__)

@app.route("/rolldice")
def roll_dice():
    return str(roll())

def roll():
    # This creates a new span that's the child of the current one
    with tracer.start_as_current_span("roll") as rollspan:
        res = randint(1, 6)
        rollspan.set_attribute("roll.value", res)
        return res

:::

Code Snippet - Metrics

:::spoiler

# These are the necessary import declarations
from opentelemetry import trace
from opentelemetry import metrics

from random import randint
from flask import Flask, request
import logging

# Acquire a tracer
tracer = trace.get_tracer("diceroller.tracer")
# Acquire a meter.
meter = metrics.get_meter("diceroller.meter")

# Now create a counter instrument to make measurements with
roll_counter = meter.create_counter(
    "dice.rolls",
    description="The number of rolls by roll value",
)

app = Flask(__name__)
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@app.route("/rolldice")
def roll_dice():
    # This creates a new span that's the child of the current one
    with tracer.start_as_current_span("roll") as roll_span:
        player = request.args.get('player', default = None, type = str)
        result = str(roll())
        roll_span.set_attribute("roll.value", result)
        # This adds 1 to the counter for the given roll value
        roll_counter.add(1, {"roll.value": result})
        if player:
            logger.warn("{} is rolling the dice: {}", player, result)
        else:
            logger.warn("Anonymous player is rolling the dice: %s", result)
        return result

def roll():
    return randint(1, 6)

:::

Kubernetes (k8s)

:::success
For Deploy to Kubernetes
:::
在不改動 source code 的情況下，將 Opentelemetry 的 Python auto-instrumentation 加入，導入 trace/log/metrics 監控數據生成