docs(observability): Telemetry 0.3.1 published, adopted by all four apps, export spans gone at the collector (#56)

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-12 22:21:07 -05:00
parent 71b8595742
commit 41a5e4d187
3 changed files with 103 additions and 13 deletions
+1 -1
View File
@@ -162,7 +162,7 @@ each project's **code-verified current state**, and the **gaps** between. See
| Auth (login / identity / authz) | **2026-11-12: `0.5.0` PUBLISHED** (5 pkgs, SHA-verified) — adds passwordless `ILdapGroupLookup.GetUserGroupsAsync` (service account; `Found`/`UserNotFound`/`UserDisabled`/`AmbiguousUser`/`DirectoryUnavailable`/`NotSupported`/`ProviderDisabled`; ScadaBridge#46) and is the first published build of 0.4.2–0.4.4 (typed `ApiKeyPepperUnavailableException`; Sqlite 10.0.13; user-bind identity proof (#40/#50); SQL Server key ids case-exact; `ApiKeyFailure.StoreUnavailable`). scadaproj#51 closed: live-verified vs NAM AD 24/24. **ScadaBridge adopts 0.5.0 (#46); OtOpcUa/mxgw/HistorianGateway/BTDB stay on their pins.** Adopted. Feed carries **`0.4.1`** (2026-10-01; 0.4.0 the same day; opt-in user bind) — **all four apps pin `0.4.1` on their default branches since 2026-11-09** (scadaproj#5: OtOpcUa `1ecd01d3`, ScadaBridge `37fcd296`, mxgw `ace12f81`, HistorianGateway `0d06483`; `BindMode` left at `ServiceAccount`; NOT deployed — wonder boxes still run `0.3.0` builds). History below refers to the earlier `0.3.0` pins. **All four apps pin `0.3.0`** (2026-09-03, all on their default branch and pushed — and all four are DEPLOYED at those tips on DEV vd03 and QA vq03/vq04 since 2026-09-10: OtOpcUa `b8b4c18f`, mxgw `4f931e4`, HistorianGateway `d8d890c`; ScadaBridge already had it on `main` via the `inbound-api-key-shared-store` work, `e45c4af9`). **0.4.0 (2026-10-01) — opt-in LDAP user bind:** adds `LdapOptions.BindMode` (`ServiceAccount` default | `User`), `UserBindTemplate` and `AcceptDomainQualifiedNames`; in `User` mode `LdapAuthService` binds as the signing-in user and reads their own groups on the same connection, so no service account is needed. Bind failures map to `BadCredentials` (bad password, unusable name, or bind result 49) or `GroupLookupFailed` (zero/multiple entries or no groups after a good bind), other bind/connect/search errors fail over to the next endpoint, and `ServiceAccount` stays the default so no existing consumer needs to change. **0.4.1** fixes the review findings: `Username` is canonicalised from the entry's `UserNameAttribute`, empty passwords are refused in both bind modes (a `ServiceAccount`-mode behaviour change), and the docs prefer the down-level `EXAMPLE\{0}` template over UPN. First consumer: BTDB Manager; ScadaBridge/HistorianGateway/OtOpcUa/MxAccessGateway stay on `0.3.0` until they move to AD. `0.2.1` is a functional fix, not housekeeping — `0.1.5` cannot log in against an AD domain root at all. `FallbackServers` (0.2.0) live-gated on the redundant local GLAuth pair. Version narrative, consumer matrix, `tools/LdapProbe`: [`ZB.MOM.WW.Auth/CLAUDE.md`](ZB.MOM.WW.Auth/CLAUDE.md). | Shared `ZB.MOM.WW.Auth` lib | [`components/auth/`](components/auth/) | [`ZB.MOM.WW.Auth/`](ZB.MOM.WW.Auth/) |
| UI Theme (layout / tokens / components) | **2026-11-09: all five consumers pin `0.5.0` on their default branches (scadaproj#9; not deployed, no browser check yet); **`0.5.3` published 2026-11-12** (TechButton `disabled`, nav-state WeakSet, opt-in `ThemeShell ScrollNav`; 0.5.3 shell `100dvh` fill, #10; no consumer pins it yet — ScadaBridge#61 is first).** Adopted. Feed `0.2.0`–`0.5.0` + `0.5.3`; **five** consumers (the ignitionoee router joined), all on `0.4.1` and pushed (2026-08-11); Every deployed wonder box (DEV vd03 + QA vq03/vq04) runs its repo tip as of 2026-09-10, so the deployed Theme is 0.4.1 everywhere. **On bump, TRIM the local `.btn` block, don't delete it** — mxgw's carried more than sizing. Narrative: [`ZB.MOM.WW.Theme/CLAUDE.md`](ZB.MOM.WW.Theme/CLAUDE.md). | Shared `ZB.MOM.WW.Theme` RCL | [`components/ui-theme/`](components/ui-theme/) | [`ZB.MOM.WW.Theme/`](ZB.MOM.WW.Theme/) |
| Health (readiness / liveness / active-node) | **Adopted** — lib `0.3.0`, all 4 apps reference it + wire `MapZbHealth`. `0.2.1`/`0.3.0` changed the active tier's behaviour (standby now answers **503**; active = oldest `Up` member via `ClusterActiveNode`, not `RoleLeader` — the two diverge after any restart), and both Akka apps' private copies were deleted in its favour (merged + pushed). Narrative: [`ZB.MOM.WW.Health/CLAUDE.md`](ZB.MOM.WW.Health/CLAUDE.md). | Shared `ZB.MOM.WW.Health` lib | [`components/health/`](components/health/) | [`ZB.MOM.WW.Health/`](ZB.MOM.WW.Health/) |
| Observability (metrics / traces / logs) | **Adopted** — **`0.3.0` published 2026-11-12** (feed `0.1.0`, `0.2.0`, `0.3.0`; `0.2.1` never published, shipped inside 0.3.0). **All 4 apps pin `0.3.0`** on their default branches (#55: OtOpcUa `412da27c`, ScadaBridge `a633005f`, mxgw `d9d9ff48`, HistorianGateway `ae1dbcc`; not deployed) and call `AddZbTelemetry` + `AddZbSerilog`. 0.3.0: `service.version` defaults to the entry assembly's informational version (git SHA); `ResourceAttributes` + `ApplyConfiguration(<App>:Telemetry)`; the Serilog OTLP log sink follows `AddZbTelemetry`'s exporter, so `Exporter=Otlp` now sends **logs** too; stricter endpoint check (scheme-less `host:port` fails when `Exporter=Otlp`). Resource identity: OtOpcUa `site.id`=Cluster id / `node.role`=plain roles; mxgw + HG `node.role=standalone`, `site.id` from config; ScadaBridge unchanged. **OTLP proven end to end 2026-11-12** (#13 + #55, both closed) against `zb-otel-collector` on `10.100.0.35:4317` ([`infra/otel-collector/`](infra/otel-collector/)): metrics, traces and logs with the full Resource from all four apps (ScadaBridge Site role; Central → ScadaBridge#133). **Wonder tiers on OTLP since 2026-11-12 (#54, closed):** one loopback-only `ZbOtelCollector` (otelcol-contrib 0.162.0, NSSM, `NT SERVICE\ZbOtelCollector`, `E:\ApiInstall\OtelCollector`, file + Prometheus `127.0.0.1:8889` exporters, ≤ ~1.8 GB per box) on vd03/vq03/vq04. OtOpcUa, ScadaBridge Central + Site, mxgw and HG export to `http://127.0.0.1:4317` via service env (Overview has no exporter binding). The deployed builds send metrics + traces; logs follow when 0.3.0 is deployed. Record: [`infra/otel-collector/wonder/`](infra/otel-collector/wonder/). Evidence: [`components/observability/2026-11-12-otlp-end-to-end-evidence.md`](components/observability/2026-11-12-otlp-end-to-end-evidence.md). Narrative: [`ZB.MOM.WW.Telemetry/CLAUDE.md`](ZB.MOM.WW.Telemetry/CLAUDE.md). | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) |
| Observability (metrics / traces / logs) | **Adopted** — **`0.3.1` published 2026-11-12** (feed `0.1.0`, `0.2.0`, `0.3.0`, `0.3.1`; `0.2.1` never published, shipped inside 0.3.0). **All 4 apps pin `0.3.1`** on their default branches (#56: OtOpcUa `f3178c5f`, ScadaBridge `95584e2f`, mxgw `453fe015`, HistorianGateway `dffb5e2`; earlier 0.3.0 pins #55: `412da27c` / `a633005f` / `d9d9ff48` / `ae1dbcc`; not deployed) and call `AddZbTelemetry` + `AddZbSerilog`. 0.3.0: `service.version` defaults to the entry assembly's informational version (git SHA); `ResourceAttributes` + `ApplyConfiguration(<App>:Telemetry)`; the Serilog OTLP log sink follows `AddZbTelemetry`'s exporter, so `Exporter=Otlp` now sends **logs** too; stricter endpoint check (scheme-less `host:port` fails when `Exporter=Otlp`). 0.3.1 (#56, closed): telemetry's own OTLP export calls (the Serilog log sink's `LogsService/Export`) are no longer traced as gRPC/HttpClient client spans; the apps' own outgoing spans are unchanged. Resource identity: OtOpcUa `site.id`=Cluster id / `node.role`=plain roles; mxgw + HG `node.role=standalone`, `site.id` from config; ScadaBridge unchanged. **OTLP proven end to end 2026-11-12** (#13 + #55, both closed) against `zb-otel-collector` on `10.100.0.35:4317` ([`infra/otel-collector/`](infra/otel-collector/)): metrics, traces and logs with the full Resource from all four apps (ScadaBridge Site role; Central → ScadaBridge#133). **Wonder tiers on OTLP since 2026-11-12 (#54, closed):** one loopback-only `ZbOtelCollector` (otelcol-contrib 0.162.0, NSSM, `NT SERVICE\ZbOtelCollector`, `E:\ApiInstall\OtelCollector`, file + Prometheus `127.0.0.1:8889` exporters, ≤ ~1.8 GB per box) on vd03/vq03/vq04. OtOpcUa, ScadaBridge Central + Site, mxgw and HG export to `http://127.0.0.1:4317` via service env (Overview has no exporter binding). The deployed builds send metrics + traces; logs follow when 0.3.0 is deployed. Record: [`infra/otel-collector/wonder/`](infra/otel-collector/wonder/). Evidence: [`components/observability/2026-11-12-otlp-end-to-end-evidence.md`](components/observability/2026-11-12-otlp-end-to-end-evidence.md). Narrative: [`ZB.MOM.WW.Telemetry/CLAUDE.md`](ZB.MOM.WW.Telemetry/CLAUDE.md). | Shared `ZB.MOM.WW.Telemetry` lib + `.Serilog` | [`components/observability/`](components/observability/) | [`ZB.MOM.WW.Telemetry/`](ZB.MOM.WW.Telemetry/) |
| Config + validation (options / startup validation) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed — the "local only / not yet pushed" caveat was stale) | Shared `ZB.MOM.WW.Configuration` lib | [`components/configuration/`](components/configuration/) | [`ZB.MOM.WW.Configuration/`](ZB.MOM.WW.Configuration/) |
| Audit (event model + writer seam) | Adopted (lib `0.1.0`, on the feed; **all 4 apps**, pushed to origin) | Shared `ZB.MOM.WW.Audit` lib | [`components/audit/`](components/audit/) | [`ZB.MOM.WW.Audit/`](ZB.MOM.WW.Audit/) |
| Galaxy Repository (object-hierarchy SQL browse + gRPC service) | **Adopted** (feed `0.1.0` + `0.2.0`; consumed at `0.2.0` by HistorianGateway **and mxaccessgw**, whose Server wires `AddZbGalaxyRepository` — verified 2026-07-18; the prior "mxaccessgw adoption is a follow-on" claim was stale) | Shared `ZB.MOM.WW.GalaxyRepository` lib | _(design in histsdk + design doc 2026-06-23)_ | [`ZB.MOM.WW.GalaxyRepository/`](ZB.MOM.WW.GalaxyRepository/) |
+32 -11
View File
@@ -2,7 +2,7 @@
Observability libraries for the **ZB.MOM.WW SCADA family** (OtOpcUa, MxAccessGateway, ScadaBridge). These are **libraries, not a service** — each package is linked directly into the consuming application at build time. There is no central telemetry process; instrumentation runs in-process alongside the application.
**Backlog / open work:** lives in Gitea issues on `dohertj2/scadaproj` (`tea issues list --repo dohertj2/scadaproj`) — #13 (OTel collector / OTLP end to end) is **closed** — proven 2026-11-12 against `zb-otel-collector` (`infra/otel-collector/`, evidence `components/observability/2026-11-12-otlp-end-to-end-evidence.md`); #55 (`service.version`/`site.id`/`node.role` + logs-over-OTLP) is **closed** by `0.3.0` (2026-11-12). Open: #54 (wonder-tier collector). The history below is a record, not a status tracker; file new follow-ups as issues.
**Backlog / open work:** lives in Gitea issues on `dohertj2/scadaproj` (`tea issues list --repo dohertj2/scadaproj`) — #13 (OTel collector / OTLP end to end) is **closed** — proven 2026-11-12 against `zb-otel-collector` (`infra/otel-collector/`, evidence `components/observability/2026-11-12-otlp-end-to-end-evidence.md`); #55 (`service.version`/`site.id`/`node.role` + logs-over-OTLP) is **closed** by `0.3.0` (2026-11-12); #56 (OTLP export calls traced as client spans) is **closed** by `0.3.1` (2026-11-12). Open: #54 (wonder-tier collector). The history below is a record, not a status tracker; file new follow-ups as issues.
The library normalizes the three-project observability surface: a shared OpenTelemetry Resource driven by a single identity triple (`service.name` / `site.id` / `node.role`), standard instrumentation wiring, Prometheus and OTLP export, and a Serilog bootstrap with enrichers and `TraceContextEnricher` for trace↔log correlation.
@@ -23,13 +23,13 @@ The library normalizes the three-project observability surface: a shared OpenTel
| Consumer | `ZB.MOM.WW.Telemetry` (core) | `ZB.MOM.WW.Telemetry.Serilog` |
|---|:---:|:---:|
| **OtOpcUa** | ✅ **0.3.0** (`master` `412da27c`) — `site.id` = Cluster id, `node.role` = plain roles | ✅ **0.3.0** (`AddZbSerilog`; logs follow `AddZbTelemetry`) |
| **MxAccessGateway** | ✅ **0.3.0** (`main` `d9d9ff48`) — `node.role=standalone`, `site.id` from config | ✅ **0.3.0** (logs follow `AddZbTelemetry`) |
| **ScadaBridge** | ✅ **0.3.0** (`main` `a633005f`, both roots) | ✅ **0.3.0** `AddZbSerilog` with `MinimumLevelGovernor` + `ConfigureLogger` on a pre-host builder; the exporter is passed explicitly via `ScadaBridgeTelemetryConfig` |
| **HistorianGateway** | ✅ **0.3.0** (`main` `ae1dbcc`) — `node.role=standalone`, `site.id` from config | ✅ **0.3.0** (logs follow `AddZbTelemetry`) |
| **OtOpcUa** | ✅ **0.3.1** (`master` `f3178c5f`) — `site.id` = Cluster id, `node.role` = plain roles | ✅ **0.3.0** (`AddZbSerilog`; logs follow `AddZbTelemetry`) |
| **MxAccessGateway** | ✅ **0.3.1** (`main` `453fe015`) — `node.role=standalone`, `site.id` from config | ✅ **0.3.0** (logs follow `AddZbTelemetry`) |
| **ScadaBridge** | ✅ **0.3.1** (`main` `95584e2f`, both roots) | ✅ **0.3.0** `AddZbSerilog` with `MinimumLevelGovernor` + `ConfigureLogger` on a pre-host builder; the exporter is passed explicitly via `ScadaBridgeTelemetryConfig` |
| **HistorianGateway** | ✅ **0.3.1** (`main` `dffb5e2`) — `node.role=standalone`, `site.id` from config | ✅ **0.3.0** (logs follow `AddZbTelemetry`) |
All four pin both packages at **`0.3.0`** on their origin default branches since 2026-11-12 (scadaproj#55; not
deployed — the wonder boxes still run their earlier builds, which have no collector anyway, #54). See `components/observability/GAPS.md` for the full result.
All four pin both packages at **`0.3.1`** on their origin default branches since 2026-11-12 (scadaproj#56; `0.3.0`
earlier the same day, #55; not deployed — the wonder boxes still run their earlier builds, which have no collector anyway, #54). See `components/observability/GAPS.md` for the full result.
---
@@ -53,9 +53,9 @@ All test assemblies run offline:
| Assembly | Tests |
|---|---|
| `ZB.MOM.WW.Telemetry.Tests` | 38 |
| `ZB.MOM.WW.Telemetry.Serilog.Tests` | 39 |
| **Total** | **77** |
| `ZB.MOM.WW.Telemetry.Tests` | 54 |
| `ZB.MOM.WW.Telemetry.Serilog.Tests` | 40 |
| **Total** | **94** |
`GeneratePackageOnBuild` is off — pack explicitly with the command above.
@@ -63,6 +63,27 @@ All test assemblies run offline:
## Status
**`0.3.1` (2026-11-12, scadaproj#56) — PUBLISHED** (both packages, packed from scadaproj `main` `85654d7`, the
commit recorded in the nuspec; 54 + 40 tests green in Release). SHA256 (feed download = local pack):
`ZB.MOM.WW.Telemetry.0.3.1.nupkg` `baf6661d03802b3781ff9f95cb2a15477a2bcfda8c26b79c0c6e850eec1480a3`,
`ZB.MOM.WW.Telemetry.Serilog.0.3.1.nupkg` `188fb0d3cade77cb8f84458fb5c5770f88ad9b4a007b897f895ad19c476e4931`.
Adopted by all four apps the same day.
- **Telemetry's own OTLP transport is not traced.** The SDK's OTLP exporters run inside
`SuppressInstrumentationScope`; `Serilog.Sinks.OpenTelemetry` does not, so every log batch produced a root
`opentelemetry.proto.collector.logs.v1.LogsService/Export` gRPC client span (scadabridge 146, mxgateway 13,
historian-gateway 10, otopcua 6 at the collector). `ZbOtlpExportCallFilter.IsExportCall` matches any
`/opentelemetry.proto.collector.*` path (any host) and OTLP/HTTP `/v1/{traces,metrics,logs}` on the configured
endpoint's host:port. HttpClient instrumentation drops those via `FilterHttpRequestMessage`. The gRPC client
instrumentation has **no Filter option**, so its `EnrichWithHttpRequestMessage` callback, which runs at start
before the HTTP request, clears `Recorded` + `IsAllDataRequested`. That also leaves the child HTTP activity
unsampled. Every other outgoing call keeps its span.
- Tests: predicate theory (16) + an end-to-end run against the loopback OTLP receiver. Positive controls: the app's
gRPC and HttpClient calls are traced. Negatives: the real log-sink export, `/v1/logs` and a collector path over
HttpClient are not. The receiver's own ASP.NET Core server spans share the process, so assertions key on
client-span markers (span name / `url.full`). Mutation-checked: no gRPC un-record, no HttpClient filter, and
"everything is export" each fail it. `VersionOf_reads_the_informational_version` now derives the expected prefix
from the assembly version instead of a `0.3.0` literal.
**`0.3.0` (2026-11-12, scadaproj#55) — PUBLISHED** (both packages, packed from the tree of scadaproj `main` `2011c8d` (committed as `df9a328` before rebasing onto main; identical `ZB.MOM.WW.Telemetry/` tree); 38 + 39
tests green in Release). SHA256 (feed download = local pack): `ZB.MOM.WW.Telemetry.0.3.0.nupkg`
`a61492383bc854f6108f20da6ec55a16b2e0bf6121cd7ca13c86a951cb29f5e2`, `ZB.MOM.WW.Telemetry.Serilog.0.3.0.nupkg`
@@ -80,7 +101,7 @@ tests green in Release). SHA256 (feed download = local pack): `ZB.MOM.WW.Telemet
- Includes the never-published **0.2.1** fixes below, including the stricter OTLP endpoint check. That check
was shipped deliberately: no checked-in config sets an endpoint, it only runs when `Exporter=Otlp`, and no
wonder tier has a collector (#54).
- Feed versions: `0.1.0`, `0.2.0`, `0.3.0` (`0.2.1` was never published).
- Feed versions: `0.1.0`, `0.2.0`, `0.3.0`, `0.3.1` (`0.2.1` was never published).
**`0.2.1` (2026-11-09, scadaproj#48) — never published; shipped inside `0.3.0`.** P3 review fixes: `RedactionEnricher` is now registered
AFTER `ConfigureLogger`, so enrichers added in the hook are redacted (009); dictionary keys that print identically are
@@ -384,7 +384,76 @@ The rig stays on the rebuilt `482ac665` image.
- **scadaproj#56:** the Serilog OTLP sink's own `LogsService/Export` calls are traced as gRPC client spans in
**all four apps** (scadabridge 127, mxgateway 13, historian-gateway 10, otopcua 6 at the collector). The SDK's
trace and metric exporters suppress their own instrumentation, and `Serilog.Sinks.OpenTelemetry` does not. This
is noise, not a loop (0 log records about it). Fix in `ZB.MOM.WW.Telemetry` with an instrumentation filter.
is noise, not a loop (0 log records about it). Fix in `ZB.MOM.WW.Telemetry` with an instrumentation filter. **Fixed in 0.3.1, same day — see below.**
- **ScadaBridge#134:** rig images report `service.version = "1.0.0"` with no `+<sha>`, because `.dockerignore`
excludes `.git/`. Fix by passing `SourceRevisionId` as a build arg. Wonder deploys, published from a checkout, are
unaffected.
---
# OTLP export calls no longer traced — scadaproj#56 (2026-11-12, same day)
## Fix: `ZB.MOM.WW.Telemetry(.Serilog)` 0.3.1 — published
The SDK's OTLP exporters suppress their own instrumentation. `Serilog.Sinks.OpenTelemetry` does not, so
`AddGrpcClientInstrumentation()` traced every log batch it sent. `AddZbTelemetry` now classifies an outgoing call
with `ZbOtlpExportCallFilter.IsExportCall`: any `/opentelemetry.proto.collector.*` path on any host, or
OTLP/HTTP `/v1/{traces,metrics,logs}` on the configured `OtlpEndpoint`'s host:port.
- **HttpClient:** `FilterHttpRequestMessage` returns `false` for an export call.
- **gRPC client:** `GrpcClientTraceInstrumentationOptions` has no `Filter` in `1.15.1-beta.1`. The
`EnrichWithHttpRequestMessage` callback runs synchronously in the instrumentation's start handler, before
`HttpClient.SendAsync` (checked in the contrib source of that tag). For an export call it clears
`ActivityTraceFlags.Recorded` and `IsAllDataRequested`, so the export processor skips the span and the child
HTTP activity is unsampled under the default parent-based sampler.
- A processor that drops spans in `OnEnd` was rejected. It only works when it is registered before every exporter,
and an app could add an exporter first. The instrumentation hooks do not depend on processor order.
Published from scadaproj `main` `85654d7` (the commit in the nuspec). SHA256, feed download = local pack:
`ZB.MOM.WW.Telemetry.0.3.1.nupkg` `baf6661d03802b3781ff9f95cb2a15477a2bcfda8c26b79c0c6e850eec1480a3`,
`ZB.MOM.WW.Telemetry.Serilog.0.3.1.nupkg` `188fb0d3cade77cb8f84458fb5c5770f88ad9b4a007b897f895ad19c476e4931`.
Registration lists `0.1.0, 0.2.0, 0.3.0, 0.3.1` for both ids.
**Tests (54 + 40, Release):**
- `OtlpExportCallFilterTests`: 16 cases. The positives are the three collector services, a collector path on
another host, and `/v1/*` on the endpoint. The negatives are ScadaBridge→site, HG→historian and mxgw paths, REST,
a `collectorx` look-alike, `/v1/logs` on another host or port, and relative URIs.
- `Otlp_export_calls_are_not_traced_but_the_apps_own_calls_are`: a host runs against the loopback OTLP receiver.
The positive controls are an app gRPC call (`zb.test.v1.AppService/Ping`) and an HttpClient call, both traced. The
negatives are a real Serilog-sink log batch (the test asserts it arrived), an HttpClient `POST /v1/logs` to the
endpoint, and a collector path over HttpClient; none is traced. The receiver's own ASP.NET Core server spans land
in the same export because the instrumentation listens process-wide. So the assertions key on client-span markers:
the protobuf span name for gRPC and `url.full` for HttpClient.
- Mutations: removing the gRPC un-record fails the test (*"LogsService/Export call was traced"*). Removing the
HttpClient filter fails it (`url.full …/v1/logs` found). Making `IsExportCall` always true fails it too
(*"positive control: the app's gRPC call was not traced"*). It passed 5 of 5 repeated runs.
## Apps (pushed to their default branch; no wonder-box deploys)
| App | Commit | Pin | Tests (Debug, against 0.3.1 — `project.assets.json` confirms it on every project) |
|---|---|---|---|
| OtOpcUa | `master` `f3178c5f` | `Directory.Packages.props` | `Host.IntegrationTests` 325 passed / 2 skipped |
| MxAccessGateway | `main` `453fe015` | `Server.csproj` | NonWindows solution: 1446 + 5 passed (28 skipped) |
| HistorianGateway | `main` `dffb5e2` | `Server.csproj` | 186 + 206 + 1414 passed (76 skipped) |
| ScadaBridge | `main` `95584e2f` | `Directory.Packages.props` | `Host.Tests` 721 passed |
## Collector before/after
The collector-wide count of `LogsService/Export` spans (`zb-otel-collector`, all runs to date) was the same before
and after both probe runs: **scadabridge 146, mxgateway 13, historian-gateway 10, otopcua 6**. The issue reported
127 for scadabridge; the ScadaBridge#133 centrals added more before the rig was restored. Each probe ran as a
throwaway local process on 0.3.1 with `Exporter=Otlp`, `OtlpEndpoint=http://10.100.0.35:4317`,
`OTEL_METRIC_EXPORT_INTERVAL=10000` and `site.id=otel-probe-56`, counted from its own start time:
| Run | Log export batches / records arrived | `*Service/Export` spans | App client spans still arriving |
|---|---|---|---|
| HistorianGateway `Development`, `:15220`/`:15221`, scratch stores, health/metrics GETs + 3 `HistorianStatus/Probe` | 15 / 168 | **0** | `HistoryService/GetInterfaceVersion` 14 (GrpcNetClient), `historian.Connect` 14, server spans |
| ScadaBridge **Site** `95584e2f`, own site id, own 1-node mesh `localhost:18082`, gRPC `:18083`, metrics `:18084`, central `localhost:18999` unreachable, scratch LocalDb/audit/secrets | 27 / 224 | **0** | `CentralControlService/Heartbeat` 17, `ReconcileSite` 2, `ReportSiteHealth` 2 (GrpcNetClient), server spans |
So the log sink exported throughout, the gRPC client instrumentation stayed on, and no export call was traced.
mxgw is proven by the library tests and its suite only; no windev scratch run was made. The docker rigs were not
touched.
The wonder boxes (#54) still run their earlier builds. They will emit `LogsService/Export` spans once a 0.3.0 build
is deployed, unless 0.3.1 is deployed instead.