--- status: Accepted date: 2026-07-03 applyTo: - "Fuchs/Observability/**" - "Fuchs/Program.cs" - "Fuchs/Services/**" supersededBy: "" --- # 0004 — OpenTelemetry is wired in extensively, without compromising performance ## Context Beyond text logs (see [0003](0003-structured-diagnostic-logging.md)), the solution needs distributed tracing and metrics to understand performance and behavior in production (PDF render durations, email send outcomes, MFR call volume, banking import throughput) without depending on a debugger or manual log-grepping — while never letting the absence of a collector break or slow down the app. ## Decision - All instrumentation is centralized in `Fuchs/Observability/FuchsTelemetry.cs`: one `ActivitySource` (`Fuchs.Intranet`) for tracing and one `Meter` for metrics, exposing named `Counter`/`Histogram` instruments (invoices/reminders/reports rendered, emails/SMS sent/failed, MT940 rows parsed, banking entries skipped/truncated, MFR calls, blob upload success/failure, PDF/report/email durations) plus a `StartActivity` helper. - Wired in `Program.cs` behind `Fuchs:Telemetry:Enabled` (default `true`): `AddOpenTelemetry()` with `AddAspNetCoreInstrumentation`, `AddHttpClientInstrumentation`, `AddSqlClientInstrumentation` for tracing, and `AddAspNetCoreInstrumentation`, `AddHttpClientInstrumentation`, `AddRuntimeInstrumentation` for metrics. - **Collection is always on; export is opt-in.** The OTLP exporter is only added when `Fuchs:Telemetry:OtlpEndpoint` is configured — with no collector present, spans/metrics are simply collected in-process and discarded, so a missing collector can never cause startup failures, exceptions, or blocking calls. Setting `Fuchs:Telemetry:Enabled=false` disables instrumentation entirely. - Per the project-wide Observability standard: every meaningful operation starts an activity via `FuchsTelemetry.StartActivity(...)`, records the matching counter/histogram, and logs entry/result/timing/errors via injected `ILogger` with structured placeholders — this is enforced for new service/handler code, not just the initial wiring. ## Consequences - New business operations worth observing must add a named instrument to `FuchsTelemetry.cs` rather than creating ad-hoc `ActivitySource`/`Meter` instances elsewhere — one source, one meter, keeps exporters and dashboards simple. - Because export is opt-in, local/dev environments get full in-process instrumentation with zero setup; wiring an OTLP collector is purely an ops-side configuration change (`Fuchs:Telemetry:OtlpEndpoint`), not a code change. - Instrumentation must stay cheap on the hot path — use the existing counters/histograms rather than allocating new tags/dictionaries per call where avoidable, and never make a business operation depend on the exporter succeeding. ## Alternatives considered - **Always-on OTLP exporter requiring a collector**: rejected — would make local dev and any environment without a collector fail hard or add latency/timeouts trying to reach one. - **Per-service ActivitySource/Meter instances**: rejected in favor of one centralized `FuchsTelemetry` — avoids scattered instrument names and duplicate registration boilerplate in `Program.cs`.