Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
12 KiB
ZB.MOM.WW.Telemetry
Observability libraries for the ZB.MOM.WW SCADA family (OtOpcUa, MxAccessGateway, ScadaBridge). These are libraries, not a service — each package is linked directly into the consuming application at build time. There is no central telemetry process; instrumentation runs in-process alongside the application.
Backlog / open work: lives in Gitea issues on dohertj2/scadaproj (tea issues list --repo dohertj2/scadaproj) — #13 (OTel collector / OTLP end to end) is closed — proven 2026-11-12 against zb-otel-collector (infra/otel-collector/, evidence components/observability/2026-11-12-otlp-end-to-end-evidence.md); #55 (service.version/site.id/node.role + logs-over-OTLP) is closed by 0.3.0 (2026-11-12). Open: #54 (wonder-tier collector). The history below is a record, not a status tracker; file new follow-ups as issues.
The library normalizes the three-project observability surface: a shared OpenTelemetry Resource driven by a single identity triple (service.name / site.id / node.role), standard instrumentation wiring, Prometheus and OTLP export, and a Serilog bootstrap with enrichers and TraceContextEnricher for trace↔log correlation.
Built at 0.1.0, published to the Gitea NuGet feed, and adopted across all three apps on 2026-06-01 (branch feat/adopt-zb-telemetry per repo, behaviour-preserving). MxAccessGateway's MEL→Serilog migration + metrics export both landed in this pass — they were not actually done beforehand despite the earlier claim. ScadaBridge kept its own LoggerConfigurationFactory until 2026-11-12, when it switched to AddZbSerilog 0.2.0 (ScadaBridge#94). Per-repo result + deferred follow-ons tracked in ~/Desktop/scadaproj/components/observability/GAPS.md.
Packages
| Package | Responsibilities | Key Dependencies |
|---|---|---|
ZB.MOM.WW.Telemetry |
AddZbTelemetry — wires OTel SDK (metrics + tracing), populates shared Resource (service.name, service.namespace, service.version, site.id, node.role, host.name), registers caller-supplied Meters/ActivitySources, adds standard instrumentation (ASP.NET Core, HttpClient, gRPC client, runtime, process), Prometheus always-on exporter, OTLP additive overlay. app.MapZbMetrics() — mounts /metrics. ZbTelemetryOptions — the single options object shared by both packages. 0.3.0: service.version defaults from the entry assembly's informational version; ResourceAttributes; ApplyConfiguration(<App>:Telemetry section); publishes its options in DI for the Serilog log sink. |
Microsoft.AspNetCore.App (framework ref), OpenTelemetry.* stack |
ZB.MOM.WW.Telemetry.Serilog |
AddZbSerilog — shared two-stage Serilog bootstrap: ReadFrom.Configuration-driven sinks, MinimumLevel.Is(Information) default (config-overridable), SiteId/NodeRole/NodeHostname enrichers from ZbTelemetryOptions, TraceContextEnricher (writes trace_id/span_id from Activity.Current), ILogRedactor seam (per-project sensitive-field redaction via RedactionEnricher). 0.2.0: optional ZbSerilogOptions overload — MinimumLevelGovernor (app key outranks Serilog:MinimumLevel) + ConfigureLogger (final filters/enrichers/sinks). 0.3.0: the OTLP log sink follows the AddZbTelemetry registration in the same container (ExportLogs=false opts out). Does NOT freeze Log.Logger — safe for multi-host/test scenarios. |
ZB.MOM.WW.Telemetry, Serilog.* stack |
Consumer matrix
| Consumer | ZB.MOM.WW.Telemetry (core) |
ZB.MOM.WW.Telemetry.Serilog |
|---|---|---|
| OtOpcUa | ✅ 0.3.0 (master 412da27c) — site.id = Cluster id, node.role = plain roles |
✅ 0.3.0 (AddZbSerilog; logs follow AddZbTelemetry) |
| MxAccessGateway | ✅ 0.3.0 (main d9d9ff48) — node.role=standalone, site.id from config |
✅ 0.3.0 (logs follow AddZbTelemetry) |
| ScadaBridge | ✅ 0.3.0 (main a633005f, both roots) |
✅ 0.3.0 AddZbSerilog with MinimumLevelGovernor + ConfigureLogger on a pre-host builder; the exporter is passed explicitly via ScadaBridgeTelemetryConfig |
| HistorianGateway | ✅ 0.3.0 (main ae1dbcc) — node.role=standalone, site.id from config |
✅ 0.3.0 (logs follow AddZbTelemetry) |
All four pin both packages at 0.3.0 on their origin default branches since 2026-11-12 (scadaproj#55; not
deployed — the wonder boxes still run their earlier builds, which have no collector anyway, #54). See components/observability/GAPS.md for the full result.
Build, test, and pack commands
# From ZB.MOM.WW.Telemetry/
# Build
dotnet build ZB.MOM.WW.Telemetry.slnx
dotnet build ZB.MOM.WW.Telemetry.slnx -c Release
# Test (no external dependencies — no running OTel collector, no Serilog backend required)
dotnet test ZB.MOM.WW.Telemetry.slnx
# Pack (two .nupkg files land in artifacts/)
dotnet pack ZB.MOM.WW.Telemetry.slnx -c Release -o ./artifacts
All test assemblies run offline:
| Assembly | Tests |
|---|---|
ZB.MOM.WW.Telemetry.Tests |
38 |
ZB.MOM.WW.Telemetry.Serilog.Tests |
39 |
| Total | 77 |
GeneratePackageOnBuild is off — pack explicitly with the command above.
Status
0.3.0 (2026-11-12, scadaproj#55) — PUBLISHED (both packages, packed from the tree of scadaproj main 2011c8d (committed as df9a328 before rebasing onto main; identical ZB.MOM.WW.Telemetry/ tree); 38 + 39
tests green in Release). SHA256 (feed download = local pack): ZB.MOM.WW.Telemetry.0.3.0.nupkg
a61492383bc854f6108f20da6ec55a16b2e0bf6121cd7ca13c86a951cb29f5e2, ZB.MOM.WW.Telemetry.Serilog.0.3.0.nupkg
04675cead32e2fb2c2cfbe88cdb4b7895b4b1601c53ba379b6215f53c731478c. Adopted by all four apps the same day.
service.version:ServiceVersion = null(default) now resolves to the entry assembly'sAssemblyInformationalVersion(else its assembly version);""omits; any value overrides. Behaviour change from 0.2.x, where null omitted the attribute.ResourceAttributes(extra Resource attributes on all signals; first-class keys reserved, rejected at startup).ApplyConfiguration(IConfiguration section)—Exporter,OtlpEndpoint,ServiceVersion,SiteId,NodeRole,ExportLogs,ResourceAttributes:<key>; absent keys keep code defaults.- Logs over OTLP:
AddZbTelemetryregisters its options in DI. TheAddZbSerilogOTLP log sink follows them when its own options do not select OTLP. Root cause of "no app ever exported a log": the sink keyed only on theAddZbSerilogoptions, which every app filled with the service name alone.ExportLogs = falsekeeps logs local. - Includes the never-published 0.2.1 fixes below, including the stricter OTLP endpoint check. That check
was shipped deliberately: no checked-in config sets an endpoint, it only runs when
Exporter=Otlp, and no wonder tier has a collector (#54). - Feed versions:
0.1.0,0.2.0,0.3.0(0.2.1was never published).
0.2.1 (2026-11-09, scadaproj#48) — never published; shipped inside 0.3.0. P3 review fixes: RedactionEnricher is now registered
AFTER ConfigureLogger, so enrichers added in the hook are redacted (009); dictionary keys that print identically are
all kept (later ones get a #n suffix in the redactor's view; the original key is restored on rebuild) (010); the OTLP
endpoint must be http/https with a host (collector:4317 is now rejected) and the pre-built-options
AddZbTelemetry(services, options) overload validates ServiceName + endpoint like the lambda overloads (011).
Behaviour change: a scheme-less OTLP endpoint that previously passed validation now throws at startup.
0.2.0 (2026-11-09, scadaproj#12) — PUBLISHED 2026-11-12 (both packages, packed from 01c6dc1 — the
0.2.0 code plus the corrected RepositoryUrl, i.e. before the 0.2.1 commit 1786bab; 12 + 30 tests green in Release).
SHA256: ZB.MOM.WW.Telemetry.0.2.0.nupkg 71749cb3c34a0d0aba3eca43a6c12398b427f494e8c4c6f216db0be7d3ab40dc,
ZB.MOM.WW.Telemetry.Serilog.0.2.0.nupkg 2aef391e7ab28061c5b3365c62c73e5c972ba8bcecd88bd0e0471550cfcc40c0
(feed download matches the local pack). First consumer: ScadaBridge (#94). Additive:
a new AddZbSerilog(builder, configure, configureSerilog) overload taking ZbSerilogOptions —
MinimumLevelGovernor (Func<IConfiguration, LogEventLevel?>, applied with MinimumLevel.Is
AFTER ReadFrom.Configuration, so an app-owned key outranks Serilog:MinimumLevel; null defers to
config; namespace Override:* entries still apply) and ConfigureLogger (runs last, for filters such
as ScadaBridge's standby active-node exclusion). The two-argument overload is unchanged. This is
the library half of ScadaBridge's AddZbSerilog adoption; the app-side switch is a follow-on in
that repo.
Built at 0.1.0, published to the Gitea NuGet feed, and adopted across all three apps on
2026-06-01 (branch feat/adopt-zb-telemetry per repo, behaviour-preserving). MxAccessGateway's
MEL→Serilog migration and metrics export both landed in this pass (not beforehand, despite the
earlier claim). Deferred follow-ons (MxGateway ms→s + Meter rename, ScadaBridge app instruments
- Site-node HTTP/1.1 metrics listener, OTLP wiring) are tracked in the component backlog (remaining open items are Gitea issues: #12 min-level governance hook — closed; #13 OTLP collector — closed 2026-11-12):
~/Desktop/scadaproj/components/observability/GAPS.md— adoption status + deferred follow-ons
Design documentation:
~/Desktop/scadaproj/components/observability/spec/SPEC.md— normalized observability target~/Desktop/scadaproj/components/observability/spec/METRIC-CONVENTIONS.md— metric naming reference~/Desktop/scadaproj/components/observability/shared-contract/ZB.MOM.WW.Telemetry.md— proposed shared-library API~/Desktop/scadaproj/components/observability/current-state/— per-project current state (code-verified)
Index history (migrated from the scadaproj root CLAUDE.md)
Moved verbatim out of the scadaproj root
CLAUDE.mdon 2026-09-03 so it loads only when needed. This file is now the authoritative place for this history — append new entries here and keep the root index row short.
Narrative
The observability component is fully populated: a normalized spec,
a metric-conventions reference, a
shared-contract, three
current-state docs, and an adoption GAPS
backlog. Shared = OTel Resource (service.name/site.id/node.role identity triple) + standard instrumentation
(ASP.NET Core, HttpClient, gRPC client, runtime, process) + Prometheus always-on exporter + OTLP opt-in
- Serilog two-stage bootstrap + SiteId/NodeRole/NodeHostname enrichers + TraceContextEnricher (trace_id/span_id)
- ILogRedactor seam; left per-project = application Meters/ActivitySources, sink config, per-operation enrichers, and redaction policies.
The shared library is built and lives in this repo at ZB.MOM.WW.Telemetry/
(.NET 10; 2 packages — ZB.MOM.WW.Telemetry, ZB.MOM.WW.Telemetry.Serilog; 34 tests (verified 2026-07-18);
dotnet pack → 2 nupkgs @ 0.1.0). Adopted across all three apps on 2026-06-01 (branch
feat/adopt-zb-telemetry per repo, behaviour-preserving): AddZbTelemetry (Resource + standard
instrumentation + Prometheus /metrics) everywhere; OtOpcUa + MxGateway on AddZbSerilog (MxGateway's
MEL→Serilog migration + metrics export both landed in this pass — they were not actually done
beforehand despite an earlier claim); ScadaBridge keeps its LoggerConfigurationFactory (min-level
governance) and only adds the shared TraceContextEnricher. Deferred: MxGateway ms→s + Meter
rename, ScadaBridge app instruments + Site-node HTTP/1.1 metrics listener, OTLP wiring. Per-repo
result tracked in components/observability/GAPS.md.
Build/test from ZB.MOM.WW.Telemetry/: dotnet test. Consumer matrix: all three apps consume both
packages after adoption (OtOpcUa, MxGateway Server, ScadaBridge Host + any instrumented project).