docs(archreview): R2-10 resilience observability shipped
v2-ci / build (pull_request) Successful in 3m47s
v2-ci / unit-tests (pull_request) Failing after 8m12s

This commit is contained in:
Joseph Doherty
2026-07-13 10:50:02 -04:00
parent cc5ecb5868
commit 1e3552dbd5
6 changed files with 28 additions and 7 deletions
+2 -3
View File
@@ -175,9 +175,8 @@ Closed since 9cad9ed0: **hard-kill failover** (`HardKillFailoverTests`, in-proce
**U9 — LOW: `EnsureVariable` silently ignores changed historize-intent on an existing node** (`OtOpcUaNodeManager.cs:1345-1349`) — invariant still lives in two files' comments rather than an assert.
**U10 — LOW (NEW): The `DriverResilienceStatusTracker` has no reader — pipeline observability is log-only.**
The tracker is registered, fed by every retry/breaker event, and consumed by nothing (Admin `/hosts` Stream E.2/E.3 never shipped). The interim mitigation (retry/breaker-open/breaker-close logging in `DriverResiliencePipelineBuilder.cs:124-170`, added with #10) is real and correctly gated, but breaker state — now a production behavior that can reject calls (`BrokenCircuitException`) — is invisible to operators except in logs. Known residual from FOLLOWUP-10; recorded here so the review tracks it.
*Recommendation:* ship the `/hosts` resilience column, or at minimum a metric per breaker-open.
**U10 — LOW (NEW): The `DriverResilienceStatusTracker` has no reader — pipeline observability is log-only.** — ✅ **REMEDIATED** (R2-10, branch `r2/10-resilience-observability`). The tracker now has a production reader: a periodic (5 s) full-snapshot publisher (`DriverResilienceStatusPublisherService`, driver nodes, registered next to the tracker with a wiring guard `ResilienceStatusReaderRegistrationTests`) → `driver-resilience-status` DPS topic → per-admin-node `DriverResilienceStatusBridge``IDriverResilienceStatusStore` → a resilience section on `DriverStatusPanel.razor` (breaker-OPEN / consecutive-failures / in-flight / staleness chips, read in-process per the self-HubConnection ban). Two tracker truthfulness gaps fixed along the way: `RecordBreakerClose` + `IsBreakerOpen` (breaker-open state is now derivable) and `RecordSuccess` wired onto the invoker success paths (`ConsecutiveFailures` is now a true consecutive counter). Mirrors the driver-health flow one-for-one. *(Original finding:)* The tracker was registered, fed by every retry/breaker event, and consumed by nothing (Admin `/hosts` Stream E.2/E.3 never shipped); the interim mitigation was retry/breaker logging in `DriverResiliencePipelineBuilder.cs`.
*Live gate (T10) pending:* docker-dev breaker-open observation on both centrals + staleness dim.
---