docs(archreview): record #10 IMPLEMENTED (Core.Abstractions seam) + residual findings

Updates FOLLOWUP-10 + STATUS.md now that #10 is wired:
- The filed "thread CapabilityInvoker into the actor" was infeasible — Runtime is
  deliberately Polly-free (refs Core.Abstractions, not Core). Shipped a seam mirroring
  IDriverFactory (branch fix/archreview-crit10-wire-capability-invoker @ 62556c24).
- Triage corrections: only 6 of 7 sites are real (GenericDriverNodeManager is test-only
  scaffolding, re-annotated); tier-defaults only (ResilienceConfig not in the artifact).
- Observability finding: the resilience-status tracker has no reader (Stream E.2/E.3
  never shipped) → added retry/breaker logging as the interim surface + live-verify enabler.
- Verification: negative-control, recording-invoker routing, real-invoker actor-context,
  analyzer + logging tests; clean solution build; Runtime 358 / Core 238 / Analyzers 32.
- Residual: live behavioral gate (rig recipe recorded) + the ResilienceConfig-in-artifact
  sub-gap follow-up.
This commit is contained in:
Joseph Doherty
2026-07-08 21:42:17 -04:00
parent 9cd1f8c4e7
commit 877cb9bc56
2 changed files with 78 additions and 10 deletions
@@ -1,8 +1,71 @@
# Follow-up #10 — Wire CapabilityInvoker into the production dispatch layer (RESILIENCE-DISPATCH-GAP)
> **Status:** OPEN · **Surfaced by:** guard 07/C-1 (wiring the OTOPCUA0001 analyzer) on
> `fix/archreview-c1-wire-analyzer` `f0082af5` · **Task:** TaskCreate #10 · **Severity:** likely High ·
> **Effort:** ML · **Risk:** Medium (behavior-changing hot path). See [`STATUS.md`](STATUS.md).
> **Status:** IMPLEMENTED (code-complete + unit/analyzer/context-verified) · live behavioral gate PENDING ·
> **Branch:** `fix/archreview-crit10-wire-capability-invoker` @ `62556c24` (off `fix/archreview-c1-wire-analyzer`,
> NOT master — the analyzer is the standing guard + the pragmas to remove live there) · **Surfaced by:** guard
> 07/C-1 · **Task:** #10 · **Severity:** High · **Risk:** Medium (behavior-changing hot path). See [`STATUS.md`](STATUS.md).
## OUTCOME (2026-07-08) — implemented as a Core.Abstractions SEAM, not a direct thread
**The filed plan below was INFEASIBLE as written.** It assumed `DriverInstanceActor` could reference
`CapabilityInvoker` directly. It cannot: **Runtime is deliberately Polly-free** — it references
`Core.Abstractions`, NOT `Core` (where `CapabilityInvoker` + Polly live), the exact boundary `IDriverFactory`
documents ("so the Runtime project doesn't pull in Core which would drag in Polly + driver hosting"). Threading
the concrete invoker in would reverse that architectural decision.
**Fix shipped = a seam mirroring `IDriverFactory`:**
- `Core.Abstractions`: `IDriverCapabilityInvoker` (+ `NullDriverCapabilityInvoker` pass-through),
`IDriverCapabilityInvokerFactory` (+ null factory).
- `Core`: `CapabilityInvoker` implements the interface; new `DriverCapabilityInvokerFactory` builds a per-instance
invoker over the singleton pipeline builder + status tracker + `DriverFactoryRegistry.GetTier`.
- `Runtime`: `DriverInstanceActor` takes an `IDriverCapabilityInvoker` (default = pass-through) and wraps its
**6** dispatch sites; `DriverHostActor.SpawnChild` injects the real invoker built by the factory (resolved from
DI in the Runtime SCE like `IDriverFactory`).
- `Host`: `DriverFactoryBootstrap` registers the tracker + builder + concrete factory.
- **Analyzer**: `IDriverCapabilityInvoker.ExecuteAsync/ExecuteWriteAsync` added as wrapper homes (interface-typed
invoker calls must suppress OTOPCUA0001 too). All 6 `RESILIENCE-DISPATCH-GAP` pragmas removed.
**Two triage corrections vs. the filed table below:**
1. Only **6** sites are real (all in `DriverInstanceActor`). The 7th — `GenericDriverNodeManager` — is **test-only
scaffolding** (zero production references; the Server has its own address-space path). Its call is NOT wired;
the pragma is re-annotated accordingly, not left as a "tracked gap".
2. **Tier-DEFAULT policy only.** `DriverInstanceSpec` (the deploy artifact) does not carry the per-instance
`ResilienceConfig` JSON, so overrides aren't applied yet. **Sub-gap → new follow-up:** plumb `ResilienceConfig`
through the composer/artifact so per-instance overrides reach the invoker.
**Observability finding + fix.** The resilience-status tracker has **no reader** (Admin `/hosts`, Phase 6.1 Stream
E.2/E.3, was never built) and the pipeline builder logged nothing — so the pipeline ran **invisibly**. Added
retry / breaker-open / breaker-close **structured logging** to `DriverResiliencePipelineBuilder` (optional ILogger,
wired in the Host DI). This is the operator-facing observability surface AND the prerequisite for any live-verify.
**Verification done (deterministic + local):**
- Negative control: unwrapping any site fails the Runtime build with OTOPCUA0001.
- Recording-invoker Runtime test: write routes via `ExecuteWriteAsync` with the `IPerCallHostResolver` host;
subscribe via `ExecuteAsync` — proves runtime routing + capability + host key.
- **Real-`CapabilityInvoker` actor-context test**: a genuinely-yielding discovery driven through the real invoker
publishes `DiscoveredNodesReady` — proving the invoker's internal `ConfigureAwait(false)` does NOT leak to the
actor's `await` (the one path pass-through can't cover; needed a test-only Core ref in Runtime.Tests).
- Analyzer test: the interface is a valid wrapper home. Pipeline-builder test: retry events are logged.
- Full solution builds clean (0 errors); Runtime.Tests 358, Core.Tests 238, Analyzers 32 green; pass-through
default keeps existing dispatch tests byte-for-byte unchanged.
**RESIDUAL — live behavioral gate (pending, recipe below).** The two real risks (routing + actor-context) are now
covered deterministically. What's unrun is an end-to-end rig check. Recipe:
1. `docker compose -f docker-dev/docker-compose.yml build central-1 central-2` (builds the Host FROM SOURCE →
picks up this branch; note: the container restore hits the private Gitea feed — may need feed creds/nuget.config
in the build context).
2. Deploy a Modbus driver (AdminUI `:9200` or `POST /api/deployments`) pointing at the modbus sim
(`10.100.0.35:5020`).
3. Force an **exception** on an invoker-wrapped call: bounce the sim (`ssh dohertj2@10.100.0.35 docker restart <sim>`)
while the actor is (re)subscribing/discovering — reads are subscription-based (NOT invoker-wrapped) and device
rejects return bad-status (no exception), so only connection-level exceptions on subscribe/discover/write trip
retry/breaker.
4. Observe the `Driver resilience retry` / `circuit-breaker OPENED` log lines in the central-1 container logs, and
recovery (`circuit-breaker closed`). Also confirms the Host DI resolves + injects the factory end-to-end.
---
## (Original filed analysis — retained for history; superseded by OUTCOME above)
## The finding