scadaproj/ZB.MOM.WW.Telemetry/README.md

ZB.MOM.WW.Telemetry

Observability libraries for the ZB.MOM.WW SCADA family (OtOpcUa, MxAccessGateway, ScadaBridge). These are libraries, not a service — each package is linked directly into the consuming application at build time. There is no central telemetry process; all instrumentation runs in-process alongside the application.

The library normalizes the three-project observability surface: a shared OpenTelemetry Resource identity, standard instrumentation wiring, Prometheus and OTLP export, and a Serilog bootstrap with enrichers and trace↔log correlation — so metrics, traces, and log lines from the same node carry identical dimensions and can join up in any backend.

---

Packages

Package	Description	Key Dependencies
ZB.MOM.WW.Telemetry	AddZbTelemetry extension, ZbTelemetryOptions, shared OTel Resource builder (ZbResource), standard instrumentation (ASP.NET Core, HttpClient, gRPC client, runtime, process), Prometheus always-on exporter + OTLP opt-in overlay, app.MapZbMetrics() endpoint extension.	Microsoft.AspNetCore.App (framework ref), OpenTelemetry.* stack
ZB.MOM.WW.Telemetry.Serilog	AddZbSerilog extension, shared enrichers (SiteId/NodeRole/NodeHostname), TraceContextEnricher (writes trace_id/span_id from Activity.Current into every log event), ILogRedactor seam (per-project sensitive-field redaction), RedactionEnricher.	ZB.MOM.WW.Telemetry, Serilog.* stack

---

The unifying hinge

The single ZbTelemetryOptions object drives both packages. Its identity triple — ServiceName → OTel Resource service.name, SiteId → site.id, NodeRole → node.role — is applied once and flows automatically to both the OpenTelemetry Resource (so every metric and span carries it) and the Serilog enrichers (so every log event carries it). A metric, a span, and a log line emitted by the same node share identical service.name, site.id, and node.role dimensions, enabling cross-signal correlation in any backend (Grafana, Jaeger, Seq, Loki, etc.) without per-project bookkeeping.

---

Consumer matrix

Consumer	ZB.MOM.WW.Telemetry (core)	ZB.MOM.WW.Telemetry.Serilog
OtOpcUa	yes	yes
MxAccessGateway	yes	yes (logging adopted — MEL → Serilog migration done)
ScadaBridge	yes	yes

All three apps consume both packages after adoption. MxAccessGateway's MEL→Serilog migration is the one in-pass adoption completed on its own branch; OtOpcUa and ScadaBridge adoption is follow-on (tracked in components/observability/GAPS.md).

---

OTel signals

AddZbTelemetry wires all three OpenTelemetry signals in a single call:

Signal	What is wired
Metrics	App Meters (via options.Meters[]) + standard: ASP.NET Core, HttpClient, .NET runtime, process. Exported via Prometheus (always on) with OTLP as an additive overlay.
Traces	App ActivitySources (via options.ActivitySources[]) + standard: ASP.NET Core, HttpClient, gRPC client. Exported via OTLP when Exporter = ZbExporter.Otlp.
Logs	Wired by AddZbSerilog (companion call). Serilog is used as the log sink; logs are bridged to OpenTelemetry via Serilog.Sinks.OpenTelemetry when configured.

Trace↔log correlation is automatic: TraceContextEnricher reads Activity.Current for each log event and attaches trace_id and span_id, so log events produced inside a traced request carry the same span identity as the trace backend.

Redaction reach. A registered ILogRedactor may remove or replace any value, and RedactionEnricher honours both (a removed key is dropped from the event). Scalar properties appear as their unwrapped CLR value; destructured properties are projected into mutable views the redactor can descend into — a {@Object} is an IDictionary<string, object?> of its fields, a logged collection an IList<object?>, a logged dictionary an IDictionary<string, object?> — all recursively, so a field nested inside a destructured object can be masked or removed:

if (properties["command"] is IDictionary<string, object?> command) command["apiKey"] = "***";

Structure type tags and dictionary keys are preserved on rebuild, and untouched properties are left intact (not reallocated). See the ILogRedactor XML doc for the full contract.

---

Exporter options

Prometheus is always wired for metrics regardless of the Exporter setting. OTLP is an additive overlay — set Exporter = ZbExporter.Otlp and OtlpEndpoint to push to a collector in addition to the scrape endpoint.

// Prometheus only (default — scrape /metrics)
builder.AddZbTelemetry(o =>
{
    o.ServiceName = "mxgateway";
    o.SiteId      = config["Site:Id"];
    o.NodeRole    = "standalone";
    o.Meters      = ["ZB.MOM.WW.MxGateway"];
});

// OTLP overlay (metrics + traces pushed to collector; /metrics still active)
builder.AddZbTelemetry(o =>
{
    o.ServiceName  = "mxgateway";
    o.SiteId       = config["Site:Id"];
    o.NodeRole     = "standalone";
    o.Meters       = ["ZB.MOM.WW.MxGateway"];
    o.Exporter     = ZbExporter.Otlp;
    o.OtlpEndpoint = "http://collector:4317";
});

// Mount the Prometheus scrape endpoint (call after app.UseRouting())
app.MapZbMetrics();   // → /metrics

// Serilog bootstrap (same options object drives enrichers)
builder.AddZbSerilog(o =>
{
    o.ServiceName = "mxgateway";
    o.SiteId      = config["Site:Id"];
    o.NodeRole    = "standalone";
});

---

Building and testing

# from ZB.MOM.WW.Telemetry/
dotnet build ZB.MOM.WW.Telemetry.slnx
dotnet test  ZB.MOM.WW.Telemetry.slnx

All test assemblies run with no external dependencies (no running OTel collector, no Serilog backend):

Assembly	Tests
ZB.MOM.WW.Telemetry.Tests	12
ZB.MOM.WW.Telemetry.Serilog.Tests	17
Total	29

---

Packing

dotnet pack ZB.MOM.WW.Telemetry.slnx -c Release -o ./artifacts

Produces two .nupkg files in artifacts/:

ZB.MOM.WW.Telemetry.0.1.0.nupkg
ZB.MOM.WW.Telemetry.Serilog.0.1.0.nupkg

GeneratePackageOnBuild is off — pack explicitly as above. Both packages are versioned lockstep from Directory.Build.props.

---

Status

Built at 0.1.0. MxAccessGateway logging adopted (MEL → Serilog migration, on its own branch). Broader OtOpcUa and ScadaBridge telemetry adoption deferred. Adoption is tracked in the component backlog:

• ~/Desktop/scadaproj/components/observability/GAPS.md

Design documentation lives alongside that backlog:

• ~/Desktop/scadaproj/components/observability/spec/SPEC.md — normalized observability target
• ~/Desktop/scadaproj/components/observability/spec/METRIC-CONVENTIONS.md — metric naming reference
• ~/Desktop/scadaproj/components/observability/shared-contract/ZB.MOM.WW.Telemetry.md — proposed API
• ~/Desktop/scadaproj/components/observability/current-state/ — per-project current state (code-verified)