[Telemetry] Resource identity gaps and OTLP logs never enabled (found exercising OTLP, #13) #55
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found by #13 reading what actually reached the collector (
components/observability/2026-11-12-otlp-end-to-end-evidence.md):service.versionis absent from every app's Resource (OtOpcUa, ScadaBridge, MxAccessGateway, HistorianGateway). SPEC §1 recommends it fromAssemblyInformationalVersion; every app already stamps the git SHA there. Options: each app setso.ServiceVersion, or the lib defaults it from the entry assembly when null (a behaviour change → needs a Telemetry release + pin bumps).site.id/node.roleabsent for OtOpcUa, MxAccessGateway, HistorianGateway — only ScadaBridge sets them (site.id=otel-probe,node.role=Sitearrived). OtOpcUa has a natural value for both (thecluster-<ClusterId>role and admin/driver roles). Without them two OtOpcUa nodes are distinguishable only byservice.instance.id(= MachineName:PID).ZB.MOM.WW.Telemetry.Serilogadds aWriteTo.OpenTelemetrysink only when the Serilog options carryExporter=Otlp+OtlpEndpoint, but every app callsAddZbSerilog(o => o.ServiceName = ...)with the service name only, so<App>:Telemetry:Exporter=Otlpswitches metrics + traces but never logs (logs.jsonlstayed empty for all four). Wire the same exporter/endpoint into the AddZbSerilog options, or document logs-over-OTLP as out of scope.Per-repo issues once a direction is chosen.
Fixed in the library and adopted in all four apps. Telemetry(.Serilog)
0.3.0is published.Root cause of "no logs": the Serilog OTLP sink only looked at the options passed to
AddZbSerilog. Every app filled those options with the service name alone, and set the exporter only onAddZbTelemetry, a separate options object the Serilog package never saw.Library, 0.3.0 (scadaproj
maincfc7168+2011c8d; the 0.2.1 review fixes ship in the same release):service.version: a nullServiceVersion(the default) now resolves to the entry assembly'sAssemblyInformationalVersion, which is the git SHA.""omits it; any other value overrides it.ResourceAttributes: extra Resource attributes on every signal. First-class keys are reserved and rejected at startup.ApplyConfiguration(<App>:Telemetry): readsExporter,OtlpEndpoint,ServiceVersion,SiteId,NodeRole,ExportLogsandResourceAttributes:*.AddZbTelemetrynow registers its options in DI, and theAddZbSerilogOTLP log sink follows them, soExporter=Otlpsends logs to the same endpoint with the same Resource.ExportLogs=falseopts out. Levels and sinks are unchanged.host:portnow fails whenExporter=Otlp). This is safe to ship: no checked-in config sets an endpoint, and no wonder tier has a collector (#54).ZB.MOM.WW.Telemetry.0.3.0.nupkga61492383bc854f6108f20da6ec55a16b2e0bf6121cd7ca13c86a951cb29f5e2ZB.MOM.WW.Telemetry.Serilog.0.3.0.nupkg04675cead32e2fb2c2cfbe88cdb4b7895b4b1601c53ba379b6215f53c731478cApps (pushed to their default branches, not deployed):
site.id/node.rolemaster412da27ccluster-<id>role) / plain roles (admin,driver)maind9d9ff48MxGateway:Telemetry:SiteId/standalonemainae1dbccTelemetry:SiteId/standalonemaina633005fScadaBridgeTelemetryConfigAt the collector (one throwaway process each; the Resource is identical on metrics, spans and logs):
Log records written inside a request carry their
traceId.Clean-up: the OtOpcUa throwaway ConfigDb was dropped. The windev scratch gateway was removed; the live
MxAccessGwstayed Running with:5130/health/ready200 and its worker PIDs unchanged.Not done: ScadaBridge Central. The rig was busy with the #132 failover drills, so ScadaBridge#133 stays open.
Full record:
components/observability/2026-11-12-otlp-end-to-end-evidence.md(new #55 section), plus GAPS, current-state, SPEC,ZB.MOM.WW.Telemetry/CLAUDE.mdand the index row.