chore(gates): close adminui-followups by audit; R2-03 T11/T12 + reconfigure Task 3 PASSED, R2-10 re-diagnosed
The two items deferment.md §0.1 named as the remaining untracked engineering. adminui-followups (19 pending / 0 completed) — CLOSED by the overlap audit the plan itself asks for first, with no code written. All 19 tasks are shipped, superseded by v3, or moot. WS1's premise had also expired: it assumed driver-embedded tag lists in DriverConfig are the runtime source of truth, which v3 ended (the artifact populates RawTags). Its capability shipped in the v3 shape — TagConfigEditorMap + DeviceFormMap, both verified reachable from RawTagModal/DeviceModal rather than merely present. Live gates on docker-dev (image rebuilt: the rig was running a build 3 h older than the last code commit; parity asserted per container plus a positive check that the symbols under test are in the shipped DLLs): - R2-03 T11 PASSED — build 0 errors, Runtime.Tests 555/555, RoslynVirtualTagEvaluator 20/20. - R2-03 T12 PASSED — VT 42 -> 0x80020000 -> 42, staged WITHOUT a deployment (a Modbus register flipped on the fixture) so the VT child survived; a deploy-driven trip creates a fresh child with no last value, which is correct-by-design and masked the result twice. P7 confirmed: two byte-identical redeploys, zero disturbance. - driver-reconfigure Task 3 PASSED — a faulted driver adopted a corrected config and connected with the node's startedAt unchanged and RestartCount=0. - R2-10 T10 still OPEN, but the recorded cause was wrong. Not ConnectionBackoff throttling (the breaker opens on 2 failures / 30 s): the dead-endpoint failure mode never enters the resilience pipeline. IDriver.InitializeAsync is not invoker-wrapped (DriverInstanceActor.cs:707) and ModbusDriver swallows per-tag failures in BOTH directions, so even the plan's fallback trigger cannot work. Live-proven: a driver timing out every read against a blackhole reported ConsecutiveFailures=0 while the panel read "All targets healthy." Finding from T12: the Bad publish's documented "carrying the last-known value" is unobservable by any client — subscription and a raw Read service call both return NULL. The actor is correct; every existing failure assertion checked StatusCode alone, so the value half was asserted nowhere. Adds VirtualTagActorTests.The_Bad_result_carries_the_last_known_value, which proves the actor emits it. The documentation is what overstates.
This commit is contained in:
Joseph Doherty
@@ -79,7 +79,7 @@
|
||||
{
|
||||
"id": "R2-03-T11",
|
||||
"subject": "Full-suite sweep (Runtime.Tests whole project + RoslynVirtualTagEvaluatorTests + solution build) and STATUS.md / tasks.json bookkeeping",
|
||||
"status": "deferred-live",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
"R2-03-T4",
|
||||
"R2-03-T5",
|
||||
@@ -87,18 +87,20 @@
|
||||
"R2-03-T10"
|
||||
],
|
||||
"note": "integration/full sweep; serialized heavy pass (build clean, Runtime.Tests 372/372, RoslynVirtualTagEvaluatorTests 20/20 verified locally; whole-solution/cross-suite sweep deferred per controller memory constraint)",
|
||||
"openNote": "STILL OPEN \u2014 live-blocked on a data-less rig: every materialised node reads Bad_WaitingForInitialData because no driver data flows, so a VT cannot be staged Good->Bad. Needs a reachable driver fixture feeding the VT. Unit-verified meanwhile."
|
||||
"openNote": "STILL OPEN \u2014 live-blocked on a data-less rig: every materialised node reads Bad_WaitingForInitialData because no driver data flows, so a VT cannot be staged Good->Bad. Needs a reachable driver fixture feeding the VT. Unit-verified meanwhile.",
|
||||
"closureNote": "PASSED 2026-08-07. Solution build 0 errors; Runtime.Tests 555/555 (3 deliberate EquipmentDeviceBindingRetired skips); RoslynVirtualTagEvaluatorTests 20/20. Note the suite lives in Host.IntegrationTests, which a solution-wide --filter does not reach. Evidence: docs/plans/2026-08-07-adminui-followups-and-live-gates.md \u00a72."
|
||||
},
|
||||
{
|
||||
"id": "R2-03-T12",
|
||||
"subject": "Live-/run gate on docker-dev (rebuild BOTH centrals): author working VT, break it, Client.CLI read shows StatusCode Bad with last value, fix and read Good; P7 no-recompile spot-check on identical redeploy",
|
||||
"status": "deferred-live",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
"R2-03-T11"
|
||||
],
|
||||
"note": "docker-dev live-/run; serial live pass",
|
||||
"openNote": "STILL OPEN \u2014 same blocker as T11. Confirmed still open 2026-07-27 (deferment.md \u00a78.5)."
|
||||
"openNote": "STILL OPEN \u2014 same blocker as T11. Confirmed still open 2026-07-27 (deferment.md \u00a78.5).",
|
||||
"closureNote": "PASSED 2026-08-07 on docker-dev (both centrals rebuilt; image parity asserted per container). Good(42) -> Bad(0x80020000 BadInternalError) -> Good(42), staged WITHOUT a deployment by flipping a Modbus holding register on the fixture, so the VT child survived the transition (a deploy-driven trip creates a fresh child with no last value \u2014 correct per the inputs-ready gate). P7: two byte-identical redeploys (identical revisionHash) caused zero disturbance, Ever-went-BAD=0. FINDING: the Bad publish does NOT carry the last-known value to the client (subscription AND raw Read both return NULL), contradicting VirtualTagActor.cs:22 and this plan STATUS row. The actor is correct \u2014 new test VirtualTagActorTests.The_Bad_result_carries_the_last_known_value proves it emits 42; the value is lost in the OPC UA layer. No existing test asserted that half. Evidence: docs/plans/2026-08-07-adminui-followups-and-live-gates.md \u00a73."
|
||||
}
|
||||
],
|
||||
"lastUpdated": "2026-07-27"
|
||||
}
|
||||
"lastUpdated": "2026-08-07"
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"planPath": "archreview/plans/R2-10-resilience-observability-plan.md",
|
||||
"lastUpdated": "2026-07-27",
|
||||
"lastUpdated": "2026-08-07",
|
||||
"tasks": [
|
||||
{
|
||||
"id": "T1",
|
||||
@@ -74,13 +74,14 @@
|
||||
{
|
||||
"id": "T10",
|
||||
"subject": "Live-/run gate on docker-dev: rebuild BOTH centrals, S7 dead-endpoint breaker-open, both-replica chips, recovery clear, staleness dim",
|
||||
"status": "deferred-live",
|
||||
"status": "blocked-structural",
|
||||
"blockedBy": [
|
||||
"T6",
|
||||
"T8",
|
||||
"T9"
|
||||
],
|
||||
"openNote": "PARTIAL \u2014 the resilience PIPELINE is live-verified (Polly CircuitBreaker -> Retry -> Timeout -> CapabilityInvoker in central-1 stack traces), but breaker-OPEN state was never forced: R2-09's ConnectionBackoff throttles retries so failures don't accumulate fast enough to open the Polly breaker. The reliable trigger is a rapid idempotent write to a dead endpoint. Confirmed still open 2026-07-27 (deferment.md \u00a78.5)."
|
||||
"openNote": "PARTIAL \u2014 the resilience PIPELINE is live-verified (Polly CircuitBreaker -> Retry -> Timeout -> CapabilityInvoker in central-1 stack traces), but breaker-OPEN state was never forced: R2-09's ConnectionBackoff throttles retries so failures don't accumulate fast enough to open the Polly breaker. The reliable trigger is a rapid idempotent write to a dead endpoint. Confirmed still open 2026-07-27 (deferment.md \u00a78.5).",
|
||||
"closureNote": "ATTEMPTED 2026-08-07, NOT ACHIEVABLE AS SPECIFIED \u2014 and the reason is structural, not environmental. The breaker only sees failures that propagate out of a CapabilityInvoker-wrapped call, and the dead-endpoint path produces none: (1) IDriver.InitializeAsync is NOT wrapped (DriverInstanceActor.cs:707 calls it directly; the invoker wraps Read/Write/Subscribe/Discover/AlarmAcknowledge only), so a driver that cannot connect contributes zero breaker input \u2014 refuting this task's own premise; (2) ModbusDriver swallows per-tag failures in BOTH directions \u2014 ReadAsync -> Degraded health (:365-375,:838-855), WriteAsync -> WriteResult status \u2014 so even the fallback trigger this plan suggests (rapid idempotent write-to-dead-endpoint) cannot work. Live-proven: MAIN-modbus pointed at a TCP blackhole went DEGRADED with every read timing out (0x80050000) while the tracker held ConsecutiveFailures=0 and the panel showed \"All targets healthy.\" The breaker is configured to open on 2 failures in a 30 s window at ratio 1.0, so the previously-recorded ConnectionBackoff-throttling explanation is not the operative one. VERIFIED LIVE regardless: panel renders on /hosts (rehomed by #546; this plan's \"8 driver pages\" placement is stale), RESILIENCE block present, and the publisher emits real tracker state every 5 s. To close: wrap the connect path, stop drivers swallowing capability failures, or use a driver whose wrapped call fails AFTER a successful init. Evidence: docs/plans/2026-08-07-adminui-followups-and-live-gates.md \u00a74."
|
||||
},
|
||||
{
|
||||
"id": "T11",
|
||||
@@ -91,4 +92,4 @@
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -253,6 +253,25 @@ section: R2-03 T11/T12 is deferred-live pending a data-fed driver fixture, and R
|
||||
breaker-OPEN state was never forced — the pipeline is live-proven but the explicit `breaker OPENED`
|
||||
line + resilience chip were not triggered.)*
|
||||
|
||||
> **Update 2026-08-07 (live-gate run) — one of the two closed, the other re-diagnosed.**
|
||||
> **R2-03 T11 + T12 ✅ PASSED.** The "data-less rig" block is gone: the Modbus fixture at
|
||||
> `10.100.0.35:5020` is now reachable from inside the rig, so a VT could be staged Good and then
|
||||
> tripped. Good(42) → `0x80020000` → Good(42), staged *without a deployment* (a Modbus register was
|
||||
> flipped on the fixture) so the VT child survived the transition; P7 confirmed by two byte-identical
|
||||
> redeploys causing zero disturbance. ⚠️ **But the R2-03 row below overstates one thing:** "carrying
|
||||
> last-known value" is **not observable by any client** — subscription *and* raw Read both return
|
||||
> `Value = NULL` with the Bad status. The actor is correct (new test
|
||||
> `VirtualTagActorTests.The_Bad_result_carries_the_last_known_value` proves it emits the value); the
|
||||
> value is lost in the OPC UA layer, and no test asserted that half.
|
||||
> **R2-10 T10 ❌ still OPEN — but the recorded cause was wrong.** It is not that ConnectionBackoff
|
||||
> throttles retries below the threshold (the breaker opens on 2 failures in a 30 s window). The
|
||||
> dead-endpoint failure mode never enters the resilience pipeline at all: `IDriver.InitializeAsync`
|
||||
> is **not** invoker-wrapped (`DriverInstanceActor.cs:707`), and `ModbusDriver` swallows per-tag
|
||||
> failures in **both** directions — so even this plan's suggested fallback trigger (rapid idempotent
|
||||
> write-to-dead-endpoint) cannot work. Live-proven: a driver timing out every read against a
|
||||
> blackhole showed `ConsecutiveFailures = 0` and a panel reading *"All targets healthy."*
|
||||
> Full evidence: [`docs/plans/2026-08-07-adminui-followups-and-live-gates.md`](../../docs/plans/2026-08-07-adminui-followups-and-live-gates.md).
|
||||
|
||||
### Residual follow-ups — tracked in Gitea (2026-07-15)
|
||||
|
||||
The remaining non-blocking items are filed as issues (none gate the remediation, which is complete):
|
||||
|
||||
+36
-8
@@ -54,10 +54,10 @@ So everything below is **untracked work**. Nothing in it is a data-loss or outag
|
||||
| What | Nature | Where |
|
||||
|---|---|---|
|
||||
| **4 live gates blocked on hardware/licence** — archreview R2-03 (VT Good→Bad on a data-fed rig), R2-10 (force breaker-OPEN), Galaxy `ScanStateProbeParityTests` (licence-gated), v2 GA exit criteria (~~FOCAS live-CNC smoke #54~~ *closed — live-validated on a real 31i-B 2026-06-25, see `docs/plans/2026-06-25-focas-pdu-v3-30i-b-support.md`*, OPC UA CTT pass, Ignition 8.3 cutover) | Cannot be closed from this box. Not decisions — inputs. | §5 |
|
||||
| **1 deferred live-verify task** — `2026-06-14-driver-reconfigure-while-faulted` Task 3, subject literally "DEFERRED (skipped)" | Now runnable: docker-dev login is disabled, so `/run` verification needs no operator. | §5 |
|
||||
| ~~**1 deferred live-verify task** — `2026-06-14-driver-reconfigure-while-faulted` Task 3~~ ✅ **PASSED 2026-08-07.** A faulted driver ("initialize failed: Connection refused") adopted a corrected config and connected while faulted, with the node's `startedAt` byte-identical before and after and `RestartCount=0` — no restart. | §5 |
|
||||
| **G-7 residual** — Sql / MTConnect / Calculation / OpcUaClient have no Contracts-side `Inspect` | **The deploy gate is weaker than the authoring gate.** `TagConfigValidator` covers 11 of 12, so config arriving by **CSV import or API** — the paths that skip the UI — meets only the weaker gate. | §1.2 |
|
||||
| ~~**3 carried-open archreview findings** — the failure-visibility trio~~ **Update 2026-08-07 (truth sweep): mostly CLOSED.** 01/S-1 **FIXED** — `AddressSpaceApplier.cs:95-97` tallies swallowed removal-pass failures into `AddressSpaceApplyOutcome.FailedNodes` (comment cites 01/S-1). 03/S4 **FIXED** — `PrimaryGatePolicy.cs` default-denies on unknown role when a driver peer exists (doc-comment "Fixes archreview 03/S4"), live-proven R2-04 T15. 06/S-1 **PARTIAL** — fail-closed advise path + `galaxy.writes.unconfirmed` meter shipped in `GatewayGalaxyDataWriter.cs`; residual = gateway-side WriteComplete correlation (cross-repo, mxaccessgw). | Residual is only the 06/S-1 cross-repo half. | §6.3 |
|
||||
| **1 genuinely unexecuted plan** — `2026-05-29-adminui-followups`, **19 pending / 0 completed** | The only substantial engineering left. ⚠️ Its first step is an **overlap audit**, not code: several items look superseded by v3's `TagConfigEditorMap`. | §6.2 |
|
||||
| ~~**1 genuinely unexecuted plan** — `2026-05-29-adminui-followups`, **19 pending / 0 completed**~~ ✅ **CLOSED 2026-08-07 by the overlap audit it asked for — it was bookkeeping drift, not backlog.** All 19 tasks are shipped, superseded by v3, or moot; **no code was needed.** The warning was right and then some: WS1's whole *premise* died with v3 (driver-embedded tag lists in `DriverConfig` are no longer the runtime source of truth — the artifact populates `RawTags`), and its capability was delivered in the v3 shape (`TagConfigEditorMap` + `DeviceFormMap`, both verified **reachable**, not merely present). WS2/WS3 shipped; WS4's target files were deleted by v3 Batch 2 + #542. Per-task evidence in the `.tasks.json`. | §6.2 |
|
||||
| **11 other open plan tasks** — ~~5 akka-hosting partials (likely v3-superseded)~~ *(verified superseded 2026-08-07 — F8/F9/F10/F13/F14 marked completed; `Phase7Composer`/`ScriptedAlarmActor` deleted, evaluators/SDK sinks/security policies all production-wired)*, 4+1 followups-batch (open *by choice*), fixedtree Task 11, 1 manual smoke checklist | Small, and two of the groups are decisions already taken. | §6.2 |
|
||||
| **Review-coverage gap** — 10 shipped projects never reviewed (Calculation, Historian.Gateway, Mqtt +Browser/Contracts, MTConnect +Contracts, Sql +Browser/Contracts) while **3 modules still review the retired Wonderware backend** | Re-verified 2026-08-04 against `code-reviews/`. Unchanged since the audit. | §6.3 |
|
||||
| **`RawBrowseModal` stale-open race** (added 2026-08-05) — `OnParametersSetAsync` never re-checks staleness after its awaits, so a superseded open's late resolve can install the wrong device's browse config. The identical shape was found by review and **fixed** in `RawTagModal`/`RawManualTagEntryModal` (post-await staleness re-check; see `docs/plans/2026-08-05-raw-modal-picker-live-config-design.md` §6); the browse modal was deliberately left out of that branch's scope. | Lower stakes than the fixed pair (the modal resolves the device it was opened for and shows its own error banner), but the fix pattern is proven and cheap to port. | design doc §6 |
|
||||
@@ -410,9 +410,9 @@ mere env-var presence, so it cannot pass vacuously.
|
||||
| ~~**Tag-set drift detection** (shipped `e77c8a35`)~~ | ✅ **CLOSED BY DELETION** — removed under Gitea **#523** rather than gated. The gate it needed (a device whose tag set actually changes) required exactly the traffic the removal objected to. | §9, 2026-07-28 (#523) |
|
||||
| **Continuous-historization value capture** (#491) | ✅ **PASSED** `e77ea4aa` (2026-08-04) — and it found a **production-fatal defect the code-complete reading missed**: `ConfigureAwait(false)` in the Akka handler destroyed `Self`, the actor restarted, and the restart discarded the (empty-seeded) ref set. One value captured, then permanent silence. Two follow-ups filed and closed the same day: **#537** (capture had no Primary gate — both pair nodes wrote duplicates) and **#538** (a poison tag pinned the FIFO head, re-writing every healthy tag forever). | `docs/plans/2026-08-01-continuous-historization-live-gate.md`; `ad744734`, `e77ea4aa`, `e7fbe977`, `4bdce910` |
|
||||
| **Wave-0 universal discovery browser** full tree render (#468) | ✅ **PASSED** `c2c3c522` (2026-08-01) — unblocked by **building** the missing sim capability (an `ab_server` patch) rather than waiting for hardware. The gate then found a decoder defect **350 unit tests agreed with**. | tracking doc §Wave 0 |
|
||||
| **archreview R2-03** — VT Good→Bad on a data-fed rig | **OPEN** — explicitly "live-blocked on this data-less rig" | `archreview/plans/STATUS.md` |
|
||||
| **archreview R2-10** — force breaker-OPEN live | **OPEN** — breaker-OPEN state never forced live | `archreview/plans/R2-10-*.tasks.json` |
|
||||
| **Driver-reconfigure-while-faulted** Task 3 | **OPEN** — task subject literally `"Task 3: Live /run verification — DEFERRED (skipped)"`, `blockedBy: [1]`. ⚠️ **The block no longer holds:** docker-dev runs the AdminUI with `Security__Auth__DisableLogin`, so `/run` verification needs no operator sign-in. This is the one gate on this list that is *runnable today*. | `2026-06-14-driver-reconfigure-while-faulted-plan.md.tasks.json` — ⚠️ **§5 cited `2026-07-14`, which does not exist**; corrected 2026-08-04 |
|
||||
| **archreview R2-03** (T11 + T12) — VT Good→Bad on a data-fed rig | ✅ **PASSED 2026-08-07.** The rig is no longer data-less — the Modbus fixture is reachable from inside it — so the VT was staged Good and tripped **without a deployment** (a register was flipped on the fixture, so the VT child survived and could hold a last value): `42 → 0x80020000 → 42`, plus the P7 identical-redeploy spot-check (zero disturbance). ⚠️ **Found:** the Bad publish's "carrying the last-known value" is **unobservable by any client** — subscription *and* raw Read both return NULL. The actor is correct (new regression test pins it); the docs overstate what a client sees. | `docs/plans/2026-08-07-adminui-followups-and-live-gates.md` §2–3 |
|
||||
| **archreview R2-10** — force breaker-OPEN live | ❌ **STILL OPEN — but the recorded cause was wrong, and the real one is structural.** Not ConnectionBackoff throttling (the breaker opens on 2 failures / 30 s): the dead-endpoint failure mode **never enters the resilience pipeline**. `IDriver.InitializeAsync` is not invoker-wrapped (`DriverInstanceActor.cs:707`), and `ModbusDriver` swallows per-tag failures in **both** directions, so even the plan's own fallback trigger (write-to-dead-endpoint) cannot work. Live-proven: a driver timing out every read against a blackhole reported `ConsecutiveFailures = 0` and *"All targets healthy."* — **the panel is permanently quiet for the failure operators most care about.** | `docs/plans/2026-08-07-adminui-followups-and-live-gates.md` §4 |
|
||||
| **Driver-reconfigure-while-faulted** Task 3 | ✅ **PASSED 2026-08-07** (the block indeed no longer held — docker-dev runs `DisableLogin`). A driver faulted on a bad device config adopted a corrected config **while faulted** and connected, with the node's `startedAt` byte-identical before/after and `RestartCount=0` — no restart, which is the whole point (pre-fix it retried the stale config forever). | `docs/plans/2026-08-07-adminui-followups-and-live-gates.md` §5 |
|
||||
| Galaxy `ScanStateProbeParityTests` | **OPEN** — licence-gated (one `$WinPlatform` on this box) | `docs/v2/Galaxy.ParityRig.md:180` |
|
||||
| **v2 GA exit criteria** — FOCAS live-CNC wire smoke (#54), OPC UA CTT/compliance pass, Ignition 8.3 redundancy cutover | **OPEN** — manual/hardware/external-tool | `docs/v2/v2-release-readiness.md:105-120` |
|
||||
| mesh Phase 4 | ✅ **PASSED** `1281aebf` | CLAUDE.md said open — **corrected**, §7.2 |
|
||||
@@ -454,18 +454,19 @@ bookkeeping, not backlog.
|
||||
> ✅ **Re-counted 2026-08-04 by parsing every `docs/plans/*.tasks.json`, not by reading this table.** The
|
||||
> five rows below are confirmed unchanged, and the sweep found **one file this section missed**:
|
||||
> `2026-06-14-driver-reconfigure-while-faulted-plan.md.tasks.json` (3 completed / 1 pending — the §5 live
|
||||
> gate). Total open plan tasks across the tree: **31** — 19 of them in the single AdminUI file. Every
|
||||
> gate). **Update 2026-08-07: the count is now 11, not 31** — the 19 AdminUI tasks closed by audit and the
|
||||
> 1 reconfigure task closed by a passing live gate. Total open plan tasks across the tree: ~~**31** — 19 of them in the single AdminUI file~~. Every
|
||||
> other `.tasks.json` is fully closed; `2026-07-24-mesh-phase6` carries one task marked
|
||||
> `resolved-no-flip`, which is a recorded decision, not an open item.
|
||||
|
||||
| File | Open | Note |
|
||||
|---|---|---|
|
||||
| **`2026-05-29-adminui-followups.md.tasks.json`** ⬅ **now the largest open item in `docs/plans/`** | **19 pending / 0 completed** | **The largest unexecuted plan in the tree.** Generic `CollectionEditor<TRow>`; per-driver device/tag editors ×7; typed resilience form; `RoleMapper.Merge` + DB-backed LDAP→role mapping + `RoleGrants.razor` + FleetAdmin authz. **Several items look superseded by v3's `TagConfigEditorMap` — verify overlap before executing.** |
|
||||
| ~~**`2026-05-29-adminui-followups.md.tasks.json`**~~ ✅ **CLOSED 2026-08-07** | ~~19 pending~~ **0 open** | **It was never backlog.** The overlap audit this row demanded found all 19 tasks shipped, superseded by v3, or moot — **no code written.** `CollectionEditor<TRow>` + the OpcUaClient endpoint editor shipped in the plan's own shape; the per-driver editors ×6 arrived as v3's `TagConfigEditorMap` + `DeviceFormMap`; WS2/WS3 (resilience form, `RoleMapper.Merge`, DB LDAP→role, `RoleGrants`, FleetAdmin) all shipped; WS4's comment targets were deleted with their files. ⚠️ WS1's premise is dead, not merely superseded: driver-embedded tag lists are no longer the runtime source of truth in v3. |
|
||||
| `2026-06-19-followups-batch.md.tasks.json` | 4 pending + 1 partial | B4 F10b surgical DataType/IsArray writes and B5 alarm-severity `SetSeverity` are **OPEN-by-choice, not built** |
|
||||
| `2026-06-26-otopcua-fixedtree-followups.md.tasks.json` | 1 pending | Task 11 build + regression |
|
||||
| `2026-05-26-akka-hosting-alignment-plan.md.tasks.json` | 5 partial | F8/F9/F10/F13/F14 — oldest open items; **likely superseded by v3, verify** |
|
||||
| `2026-05-28-adminui-driver-pages-plan.md.tasks.json` | 1 partial | 10.4 manual smoke checklist |
|
||||
| `2026-06-14-driver-reconfigure-while-faulted-plan.md.tasks.json` ⬅ **added 2026-08-04** | 1 pending | Task 3 live `/run` verify, self-described "DEFERRED (skipped)". **Runnable now** — see §5. |
|
||||
| ~~`2026-06-14-driver-reconfigure-while-faulted-plan.md.tasks.json`~~ ✅ **CLOSED 2026-08-07** | ~~1 pending~~ **0 open** | Task 3 live `/run` verify **PASSED** — see §5. |
|
||||
|
||||
`docs/plans/2026-07-23-mesh-phase4-…tasks.json` Task 9 was the only `deferred`-flagged mesh task —
|
||||
**it has since landed** (`3a590a0c`), so that file is now stale too.
|
||||
@@ -1278,6 +1279,33 @@ drivers), 3 carried-open archreview findings led by 01/S-1 (`AddressSpaceApplier
|
||||
|
||||
---
|
||||
|
||||
### 2026-08-07 — adminui-followups closed by audit; three of four runnable gates PASSED
|
||||
|
||||
Took the two items §0.1 named as the remaining untracked engineering. Full evidence:
|
||||
[`docs/plans/2026-08-07-adminui-followups-and-live-gates.md`](docs/plans/2026-08-07-adminui-followups-and-live-gates.md).
|
||||
|
||||
- **`2026-05-29-adminui-followups` — CLOSED, no code written.** §0.1 called it "the only substantial
|
||||
engineering left"; the overlap audit it demanded shows all 19 tasks shipped, superseded, or moot.
|
||||
The lesson is the one this register keeps re-learning: **a `.tasks.json` is not evidence.** The
|
||||
plan's *premise* had also expired — WS1 assumed driver-embedded tag lists are the runtime source of
|
||||
truth, which v3 ended. Supersession claims were checked for **reachability** (`TagConfigEditorMap`
|
||||
is dispatched from `RawTagModal.razor:79`), because "superseded by X" where X is dead surface is
|
||||
this repo's signature failure.
|
||||
- **R2-03 T11 ✅ / T12 ✅.** Both blockers the previous pass recorded are gone — the Modbus fixture is
|
||||
now reachable from inside the rig. The trip had to be staged **without a deployment** (flip a
|
||||
register on the fixture) because a deploy creates a fresh VT child with no last value, which is
|
||||
correct-by-design and had masked the result twice.
|
||||
⚠️ **Finding:** "Bad carrying the last-known value" is **unobservable by any client** — subscription
|
||||
and raw Read both return NULL. The actor is right; no test asserted that half, so one was added
|
||||
(`The_Bad_result_carries_the_last_known_value`). The *documentation* is what is wrong.
|
||||
- **R2-10 T10 ❌ still open — recorded cause was wrong.** Not retry throttling. The dead-endpoint
|
||||
failure mode never enters the resilience pipeline: `InitializeAsync` is not invoker-wrapped and
|
||||
`ModbusDriver` swallows per-tag failures in both directions. **Operator consequence:** the panel
|
||||
reads *"All targets healthy."* for a driver timing out every read against a blackhole.
|
||||
- **driver-reconfigure Task 3 ✅** — corrected config adopted while faulted, `RestartCount=0`.
|
||||
- **Method note:** the rig image was 3 h older than the last code commit. Rebuilt, then asserted image
|
||||
parity **per container** *and* that the specific symbols under test were in the shipped DLLs.
|
||||
|
||||
### 2026-08-05 — MTConnect write-back; the MQTT closure records reconciled
|
||||
|
||||
Recorded from branch `feature/mqtt-mtconnect-write`. This is exactly the drift shape the 2026-08-04
|
||||
|
||||
@@ -1,26 +1,46 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-05-29-adminui-followups.md",
|
||||
"branch": "feat/adminui-followups",
|
||||
"lastUpdated": "2026-08-07",
|
||||
"closureNote": "CLOSED 2026-08-07 by the overlap audit deferment.md §6.2 asked for before executing. This file recorded 19 pending / 0 completed and was called 'the largest unexecuted plan in the tree' and 'the only substantial engineering left' — that was bookkeeping drift, not backlog. Every task is shipped, superseded by v3, or moot; per-task evidence is in each closureNote below. The plan's own premise died with v3: WS1 rests on 'driver-embedded tag/device lists in DriverConfig JSON are the runtime source of truth', but v3 authors tags as Tag rows in /raw and the deploy artifact populates the driver's RawTags (ModbusDriverFactoryExtensions.cs:98-102), so hand-editing a list on the driver form no longer decides what the driver polls. The capability WS1 wanted was delivered in the v3 shape instead: per-driver typed tag editors via TagConfigEditorMap (11 editors, dispatched from RawTagModal.razor:79 + RawManualTagEntryModal.razor:92) and per-driver device forms via DeviceFormMap (9 forms, dispatched from DeviceModal.razor:45), both guarded by DriverFormMapParityTests + DriverDispatchMapParityTests. Verified reachable, not merely present — a supersession claim naming an unreachable replacement would repeat the dead-surface class this repo has hit repeatedly.",
|
||||
"tasks": [
|
||||
{"id": 11, "plan": 1, "subject": "Task 1: Generic CollectionEditor<TRow> component", "status": "pending"},
|
||||
{"id": 12, "plan": 2, "subject": "Task 2: Modbus tag editor (proof) + tests", "status": "pending", "blockedBy": [11]},
|
||||
{"id": 13, "plan": 3, "subject": "Task 3: AbCip device+tag editors + tests", "status": "pending", "blockedBy": [11]},
|
||||
{"id": 14, "plan": 4, "subject": "Task 4: AbLegacy device+tag editors + tests", "status": "pending", "blockedBy": [11]},
|
||||
{"id": 15, "plan": 5, "subject": "Task 5: TwinCAT device+tag editors + tests", "status": "pending", "blockedBy": [11]},
|
||||
{"id": 16, "plan": 6, "subject": "Task 6: FOCAS device+tag editors + tests", "status": "pending", "blockedBy": [11]},
|
||||
{"id": 17, "plan": 7, "subject": "Task 7: S7 tag editor + tests", "status": "pending", "blockedBy": [11]},
|
||||
{"id": 18, "plan": 8, "subject": "Task 8: OpcUaClient endpoint-URL editor + tests", "status": "pending", "blockedBy": [11]},
|
||||
{"id": 19, "plan": 9, "subject": "Task 9: ResilienceFormModel + tests", "status": "pending"},
|
||||
{"id": 20, "plan": 10, "subject": "Task 10: Typed resilience form in DriverResilienceSection", "status": "pending", "blockedBy": [19]},
|
||||
{"id": 21, "plan": 11, "subject": "Task 11: RoleMapper.Merge overload + tests", "status": "pending"},
|
||||
{"id": 22, "plan": 12, "subject": "Task 12: Register ILdapGroupRoleMappingService in DI", "status": "pending"},
|
||||
{"id": 23, "plan": 13, "subject": "Task 13: Wire DB merge into AuthEndpoints.LoginAsync", "status": "pending", "blockedBy": [21, 22]},
|
||||
{"id": 24, "plan": 14, "subject": "Task 14: Add FleetAdmin authorization policy", "status": "pending"},
|
||||
{"id": 25, "plan": 15, "subject": "Task 15: RoleGrants.razor global CRUD (FleetAdmin-gated)", "status": "pending", "blockedBy": [22, 24]},
|
||||
{"id": 26, "plan": 16, "subject": "Task 16: LdapGroupRoleMapping service tests (global CRUD)", "status": "pending"},
|
||||
{"id": 27, "plan": 17, "subject": "Task 17: Delete stale source comments", "status": "pending", "blockedBy": [12, 13, 14, 15, 16, 17, 18, 20, 25]},
|
||||
{"id": 28, "plan": 18, "subject": "Task 18: Strip now-true rendered notes", "status": "pending", "blockedBy": [12, 13, 14, 15, 16, 17, 18, 25]},
|
||||
{"id": 29, "plan": 19, "subject": "Task 19: Full verification (build + test + /run)", "status": "pending", "blockedBy": [20, 23, 26, 27, 28]}
|
||||
],
|
||||
"lastUpdated": "2026-05-29"
|
||||
{"id": 11, "plan": 1, "subject": "Task 1: Generic CollectionEditor<TRow> component", "status": "completed",
|
||||
"closureNote": "SHIPPED — Components/Shared/Drivers/CollectionEditor.razor exists with the designed shape (working-copy modal, commit-on-save, NewRow factory, Validate delegate)."},
|
||||
{"id": 12, "plan": 2, "subject": "Task 2: Modbus tag editor (proof) + tests", "status": "completed",
|
||||
"closureNote": "SUPERSEDED — delivered as Uns/TagEditors/ModbusTagConfigEditor.razor via TagConfigEditorMap, not as a CollectionEditor over an embedded JSON list. The embedded list it would have edited is no longer the runtime source of truth."},
|
||||
{"id": 13, "plan": 3, "subject": "Task 3: AbCip device+tag editors + tests", "status": "completed",
|
||||
"closureNote": "SUPERSEDED — DeviceForms/AbCipDeviceForm.razor + Uns/TagEditors/AbCipTagConfigEditor.razor."},
|
||||
{"id": 14, "plan": 4, "subject": "Task 4: AbLegacy device+tag editors + tests", "status": "completed",
|
||||
"closureNote": "SUPERSEDED — DeviceForms/AbLegacyDeviceForm.razor + Uns/TagEditors/AbLegacyTagConfigEditor.razor."},
|
||||
{"id": 15, "plan": 5, "subject": "Task 5: TwinCAT device+tag editors + tests", "status": "completed",
|
||||
"closureNote": "SUPERSEDED — DeviceForms/TwinCATDeviceForm.razor + Uns/TagEditors/TwinCATTagConfigEditor.razor."},
|
||||
{"id": 16, "plan": 6, "subject": "Task 6: FOCAS device+tag editors + tests", "status": "completed",
|
||||
"closureNote": "SUPERSEDED — DeviceForms/FocasDeviceForm.razor + Uns/TagEditors/FocasTagConfigEditor.razor."},
|
||||
{"id": 17, "plan": 7, "subject": "Task 7: S7 tag editor + tests", "status": "completed",
|
||||
"closureNote": "SUPERSEDED — Uns/TagEditors/S7TagConfigEditor.razor (plus DeviceForms/S7DeviceForm.razor, which this task did not ask for)."},
|
||||
{"id": 18, "plan": 8, "subject": "Task 8: OpcUaClient endpoint-URL editor + tests", "status": "completed",
|
||||
"closureNote": "SHIPPED — and it is the one task delivered in the plan's own shape: Forms/OpcUaClientDriverForm.razor is the sole production consumer of CollectionEditor. Endpoints genuinely still live in the driver config, which is why this one survived the v3 change and Tasks 2-7 did not."},
|
||||
{"id": 19, "plan": 9, "subject": "Task 9: ResilienceFormModel + tests", "status": "completed",
|
||||
"closureNote": "SHIPPED — Components/Shared/Drivers/ResilienceFormModel.cs + tests/Server/...AdminUI.Tests/ResilienceFormModelTests.cs."},
|
||||
{"id": 20, "plan": 10, "subject": "Task 10: Typed resilience form in DriverResilienceSection", "status": "completed",
|
||||
"closureNote": "SHIPPED — DriverResilienceSection.razor binds ResilienceFormModel and keeps the raw-JSON escape hatch at :67. NARROWER than designed, deliberately: two of the designed fields no longer exist. RecycleIntervalSeconds was deleted with the scheduled-recycle machinery (Gitea #522; see the tombstone comments at DriverResilienceOptions.cs:22 and ResilienceFormModel.cs:26) and the bulkhead knobs were deleted end-to-end by archreview R2-02 because Build never added a concurrency limiter. Authoring either today would re-create an inert knob."},
|
||||
{"id": 21, "plan": 11, "subject": "Task 11: RoleMapper.Merge overload + tests", "status": "completed",
|
||||
"closureNote": "SHIPPED — RoleMapper.Merge(baseline, dbRows) consumed by OtOpcUaGroupRoleMapper.MapAsync; covered by OtOpcUaGroupRoleMapperTests + CanonicalAdminRolesTests."},
|
||||
{"id": 22, "plan": 12, "subject": "Task 12: Register ILdapGroupRoleMappingService in DI", "status": "completed",
|
||||
"closureNote": "SHIPPED — Configuration/ServiceCollectionExtensions.cs:34 (EF-backed, Scoped). A driver-only node additionally binds NullLdapGroupRoleMappingService (Host/Program.cs:351) because mesh Phase 4 left it with no ConfigDb."},
|
||||
{"id": 23, "plan": 13, "subject": "Task 13: Wire DB merge into AuthEndpoints.LoginAsync", "status": "completed",
|
||||
"closureNote": "SHIPPED — AuthEndpoints.cs:100 calls roleMapper.MapAsync behind the shared IGroupRoleMapper<string> seam, which applies the appsettings GroupToRole baseline and merges system-wide DB grants."},
|
||||
{"id": 24, "plan": 14, "subject": "Task 14: Add FleetAdmin authorization policy", "status": "completed",
|
||||
"closureNote": "SHIPPED — AdminUiPolicies.FleetAdmin registered at Security/ServiceCollectionExtensions.cs:160 (RequireRole Administrator; the policy NAME was kept stable across the AdminRole rename)."},
|
||||
{"id": 25, "plan": 15, "subject": "Task 15: RoleGrants.razor global CRUD (FleetAdmin-gated)", "status": "completed",
|
||||
"closureNote": "SHIPPED — RoleGrants.razor:2 carries [Authorize(Policy = AdminUiPolicies.FleetAdmin)]; Add (:132 CreateAsync, IsSystemWide=true/ClusterId=null as designed) + Delete (:148) + the read-only appsettings fallback panel. No in-place Update, which is correct rather than missing: a row is only (LdapGroup, Role), so editing either field IS a different grant — delete-then-add is the whole of 'update' here."},
|
||||
{"id": 26, "plan": 16, "subject": "Task 16: LdapGroupRoleMapping service tests (global CRUD)", "status": "completed",
|
||||
"closureNote": "SHIPPED — tests/Core/...Configuration.Tests/LdapGroupRoleMappingServiceTests.cs."},
|
||||
{"id": 27, "plan": 17, "subject": "Task 17: Delete stale source comments", "status": "completed",
|
||||
"closureNote": "MOOT — the comments went with their files. FleetStatusHub.cs was deleted with the whole zero-client fleet-status channel (Gitea #542); DriverEditRouter.razor, DriverTypePicker.razor and DriverEdit.razor were deleted by v3 Batch 2, which retired the routed /clusters/{id}/drivers flow. Greps for the named comments (TODO(3.3/3.4), 'legacy DriverEdit', 'until the bridge lands') return nothing outside bin/obj."},
|
||||
{"id": 28, "plan": 18, "subject": "Task 18: Strip now-true rendered notes", "status": "completed",
|
||||
"closureNote": "MOOT — same cause: the driver pages that rendered 'list-editor coming in a follow-up phase' and 'typed-form-ifying Polly is a follow-up' no longer exist. Neither string survives anywhere in src/."},
|
||||
{"id": 29, "plan": 19, "subject": "Task 19: Full verification (build + test + /run)", "status": "completed",
|
||||
"closureNote": "DONE 2026-08-07 as part of the same session's live-gate run: solution build 0 errors, and the AdminUI surfaces this plan targets were driven on the docker-dev rig alongside the four live gates. See deferment.md §9."}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -1,10 +1,36 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-06-14-driver-reconfigure-while-faulted-plan.md",
|
||||
"tasks": [
|
||||
{"id": 0, "subject": "Task 0: Feature branch (already created)", "status": "completed"},
|
||||
{"id": 1, "subject": "Task 1: Adopt corrected config mid-(re)connect (DriverInstanceActor)", "status": "completed", "blockedBy": [0]},
|
||||
{"id": 2, "subject": "Task 2: Docs + bookkeeping", "status": "completed", "blockedBy": [1]},
|
||||
{"id": 3, "subject": "Task 3: Live /run verification — DEFERRED (skipped)", "status": "pending", "blockedBy": [1]}
|
||||
{
|
||||
"id": 0,
|
||||
"subject": "Task 0: Feature branch (already created)",
|
||||
"status": "completed"
|
||||
},
|
||||
{
|
||||
"id": 1,
|
||||
"subject": "Task 1: Adopt corrected config mid-(re)connect (DriverInstanceActor)",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"subject": "Task 2: Docs + bookkeeping",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
1
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"subject": "Task 3: Live /run verification",
|
||||
"status": "completed",
|
||||
"blockedBy": [
|
||||
1
|
||||
],
|
||||
"closureNote": "PASSED 2026-08-07 on docker-dev. DRV-ce2fb5217f5d faulted on a bad device config (:5999, \"initialize failed: Connection refused\"); a corrected config (:5020) was deployed while it was faulted and the driver adopted it and connected (10:22:13) with central-1 startedAt byte-identical before and after and RestartCount=0 \u2014 no node restart. The \"DEFERRED (skipped)\" block no longer held: docker-dev runs the AdminUI with DisableLogin, so /run needs no operator sign-in. Evidence: docs/plans/2026-08-07-adminui-followups-and-live-gates.md \u00a75."
|
||||
}
|
||||
],
|
||||
"lastUpdated": "2026-06-14"
|
||||
}
|
||||
"lastUpdated": "2026-08-07"
|
||||
}
|
||||
@@ -0,0 +1,217 @@
|
||||
# 2026-08-07 — adminui-followups closure + the four runnable live gates
|
||||
|
||||
**Scope.** The two items `deferment.md` §0.1 named as the remaining untracked engineering:
|
||||
the unexecuted `2026-05-29-adminui-followups` plan, and the live gates that need only the
|
||||
docker-dev rig (no hardware, no licence). There are exactly **four** such gate tasks:
|
||||
archreview **R2-03 T11** and **T12**, **R2-10 T10**, and
|
||||
`2026-06-14-driver-reconfigure-while-faulted` **Task 3**. The other open gates (Galaxy
|
||||
`ScanStateProbeParityTests`, the v2 GA exit criteria) need a licence or external tooling and were
|
||||
not attempted.
|
||||
|
||||
**Tree:** `master` @ `21009874`. **Rig:** docker-dev, image rebuilt for this run.
|
||||
|
||||
---
|
||||
|
||||
## 0. Rig provenance (done before anything was gated)
|
||||
|
||||
The rig was running an image built **2026-08-06 17:55**, three hours *older* than the last code
|
||||
commit (`8b10d327`, 2026-08-06 21:09). Gating on it would have been meaningless. Rebuilt, both
|
||||
centrals `--force-recreate`, then:
|
||||
|
||||
- **Image parity asserted per container** (`:9200` round-robins central-1/central-2, so a single
|
||||
check proves nothing): both on `sha256:49d52d72…`.
|
||||
- **Positive check that the code under test is in the artifact** — not merely that the image is
|
||||
new: `RecordBreakerClose` + `IsBreakerOpen` present in the shipped `Core.dll`,
|
||||
`DriverResilienceStatusPublisherService` in `OtOpcUa.Host.dll`, `EvaluationResult` +
|
||||
`ClearCompiledScripts` in `Runtime.dll`; assemblies stamped 2026-08-07 09:29 UTC.
|
||||
|
||||
---
|
||||
|
||||
## 1. adminui-followups — CLOSED by audit, no code written
|
||||
|
||||
`deferment.md` called this "the largest unexecuted plan in the tree" and "the only substantial
|
||||
engineering left" (19 pending / 0 completed). The plan's own first step is an overlap audit, and the
|
||||
audit says **every task is already shipped, superseded by v3, or moot.** Per-task evidence is in
|
||||
`docs/plans/2026-05-29-adminui-followups.md.tasks.json`; the summary:
|
||||
|
||||
| Group | Verdict |
|
||||
|---|---|
|
||||
| WS1 Tasks 1, 8 (`CollectionEditor<TRow>`, OpcUaClient endpoints) | **Shipped in the plan's own shape** — `CollectionEditor.razor`, consumed by `OpcUaClientDriverForm.razor` |
|
||||
| WS1 Tasks 2–7 (per-driver tag/device editors ×6) | **Superseded by v3** — `TagConfigEditorMap` (11 editors) + `DeviceFormMap` (9 device forms) |
|
||||
| WS2 Tasks 9, 10 (resilience typed form) | **Shipped**, deliberately narrower — see below |
|
||||
| WS3 Tasks 11–16 (LDAP→role map) | **Shipped** end to end, incl. the login merge at `AuthEndpoints.cs:100` |
|
||||
| WS4 Tasks 17, 18 (comment cleanup) | **Moot** — the files were deleted by v3 Batch 2 and #542 |
|
||||
|
||||
Two things worth carrying forward:
|
||||
|
||||
- **The plan's premise died with v3.** WS1 rests on "driver-embedded tag/device lists in
|
||||
`DriverConfig` JSON are the runtime source of truth". In v3 tags are `Tag` rows authored in `/raw`
|
||||
and the deploy artifact populates the driver's `RawTags`
|
||||
(`ModbusDriverFactoryExtensions.cs:98-102`), so editing a list on a driver form no longer decides
|
||||
what the driver polls. The capability was delivered, in a different shape.
|
||||
- **WS2 shipped narrower than designed, correctly.** Two designed fields no longer exist:
|
||||
`RecycleIntervalSeconds` went with the scheduled-recycle machinery (#522) and the bulkhead knobs
|
||||
were deleted end-to-end by archreview R2-02 because `Build` never added a concurrency limiter.
|
||||
Authoring either today would re-create an inert knob.
|
||||
|
||||
The supersession claims were checked for **reachability**, not just existence —
|
||||
`TagConfigEditorMap.Resolve` is dispatched from `RawTagModal.razor:79` and
|
||||
`RawManualTagEntryModal.razor:92`, `DeviceFormMap.Resolve` from `DeviceModal.razor:45`, both guarded
|
||||
by `DriverFormMapParityTests` + `DriverDispatchMapParityTests`. A "superseded by X" whose X is itself
|
||||
dead surface is the exact failure this repo has hit repeatedly.
|
||||
|
||||
---
|
||||
|
||||
## 2. R2-03 T11 — full-suite sweep ✅ PASSED
|
||||
|
||||
| Leg | Result |
|
||||
|---|---|
|
||||
| Solution build | **0 errors** |
|
||||
| `Runtime.Tests` (whole project) | **555/555**, 3 skipped |
|
||||
| `RoslynVirtualTagEvaluatorTests` | **20/20** |
|
||||
|
||||
The 3 skips are the deliberate `EquipmentDeviceBindingRetired` tombstones, matching the count
|
||||
`deferment.md` §4.4 predicts. (The plan names the suite `RoslynVirtualTagEvaluatorTests`; it lives in
|
||||
`Host.IntegrationTests`, not a project a solution-wide `--filter` reaches by default.)
|
||||
|
||||
---
|
||||
|
||||
## 3. R2-03 T12 — VT fail→Bad→recover ✅ PASSED, with one finding
|
||||
|
||||
Both blockers recorded against the previous attempt are **gone**: the Modbus fixture at
|
||||
`10.100.0.35:5020` is now reachable from inside the rig (so driver data flows and a VT can be staged
|
||||
Good), which is what made this gate runnable at all.
|
||||
|
||||
**Method.** The first two attempts produced a Bad result whose value was empty, but each followed a
|
||||
deploy — and a deploy creates a fresh VT child with no last value, which is *correct by design* (the
|
||||
inputs-ready gate that suppresses a cold-start flash, `VirtualTagActor.cs:246`). To isolate the
|
||||
variable the trip was made **data-driven and deployment-free**: the script fails on a raw tag's
|
||||
value, and that value was flipped by writing Modbus holding register 205 directly on the fixture, so
|
||||
the VT child survives across the transition.
|
||||
|
||||
```
|
||||
[09:51:57] GateVt = 42 (0x00000000) Good
|
||||
[09:52:51] GateVt = (0x80020000) Bad_InternalError <- reg205 := 1, no deployment
|
||||
[09:53:26] GateVt = 42 (0x00000000) Good <- reg205 := 0, no deployment
|
||||
```
|
||||
|
||||
- ✅ **Degradation** — script failure degrades the node to `0x80020000 BadInternalError` instead of
|
||||
freezing on the last Good. Server-side cause confirmed in the log:
|
||||
`VirtualTag VT-gatevt: script execution threw / System.InvalidOperationException`.
|
||||
- ✅ **Recovery** — returns to Good with the same value, exercising the dedup bypass.
|
||||
- ✅ **P7 no-recompile spot-check** — two byte-identical redeploys (`revisionHash` identical across
|
||||
both, `94f68785…`) caused **no disturbance at all**: `Ever went BAD: 0`, VT held Good=42.
|
||||
Production has no compile log or metric, so the observable is the *consequence* of not
|
||||
recompiling; the gate itself is pinned by `VirtualTagHostActorTests` (exactly two clears across
|
||||
first-generation / identical-redeploy / edited-expression).
|
||||
|
||||
### ⚠️ Finding — "carrying the last-known value" never reaches the client
|
||||
|
||||
`VirtualTagActor`'s xmldoc (`:22`) and the R2-03 row in `archreview/plans/STATUS.md` both state the
|
||||
Bad publish carries the last-known value. **A client never sees it.** Two independent channels agree:
|
||||
|
||||
- subscription: `GateVt = (0x80020000)` — value empty;
|
||||
- raw OPC UA **Read** service (which returns a `DataValue` rather than throwing):
|
||||
`StatusCode 0x80020000, Value <NULL>`.
|
||||
|
||||
The VT logic is **not** at fault. `OnEvaluationFailed` sends `_hasLastValue ? _lastValue : null`, and
|
||||
a new regression test — `VirtualTagActorTests.The_Bad_result_carries_the_last_known_value` — proves
|
||||
the actor emits `Bad` carrying `42`. The value is lost below the actor, in the OPC UA layer
|
||||
(`OtOpcUaNodeManager.WriteValue` assigns `variable.Value` unconditionally, so the loss is at
|
||||
Read/Publish encoding, where a Bad `StatusCode` conventionally carries no value).
|
||||
|
||||
**Why no test caught it:** every existing failure assertion in `VirtualTagActorTests` checks
|
||||
`.StatusCode` alone (`:87, :101, :123, :142, :180`) — the "carrying the last-known value" half was
|
||||
asserted **nowhere**. That test is now added.
|
||||
|
||||
Not filed as a defect because the client-visible behaviour may be the correct OPC UA reading; what is
|
||||
certainly wrong is the **documentation**, which promises an operator something no client can observe.
|
||||
|
||||
### Confirmed still true: the Client.CLI `read` limitation
|
||||
|
||||
`read` surfaces a Bad status as a `ServiceResultException` from `FetchValueAsync` rather than
|
||||
printing it — exactly as the previous pass recorded. `ReadCommand.cs:52` does print `Status:`, but
|
||||
the SDK throws before reaching it. **Use `subscribe`** (prints value *and* status, tracks
|
||||
`everBad`), or a raw Read.
|
||||
|
||||
---
|
||||
|
||||
## 4. R2-10 T10 — breaker-OPEN ❌ NOT ACHIEVABLE AS SPECIFIED
|
||||
|
||||
The gate asks to point a driver at a dead endpoint and watch the breaker open. **It cannot open that
|
||||
way, for a structural reason** — not because of the rig. Three attempts, each failing at a different
|
||||
point, together characterise it:
|
||||
|
||||
| Attempt | What happened |
|
||||
|---|---|
|
||||
| Spare Modbus driver → blackhole (`sql:1433`, accepts TCP, never answers Modbus) | Driver had **no pollable tags** (its only tag skipped: "tag config did not map to a definition") — no capability calls at all |
|
||||
| `MAIN-modbus` (13 live tags) → same blackhole | Driver **connected**, reads failed (`MainPressure` → `0x80050000 BadTimeout`, row showed DEGRADED / "Modbus socket closed mid-response") — yet tracker stayed `ConsecutiveFailures = 0`, panel showed **"All targets healthy."** |
|
||||
| `line3-sql` → unusable connection string (Sql `ReadAsync` *does* propagate) | Fails in **`InitializeAsync`** (liveness check) before ever reaching a wrapped Read |
|
||||
|
||||
**Root cause.** The breaker only sees failures that propagate as exceptions out of a
|
||||
`CapabilityInvoker`-wrapped call. On the dead-endpoint path nothing does:
|
||||
|
||||
1. **`IDriver.InitializeAsync` is not wrapped.** `DriverInstanceActor.cs:707` calls it directly; the
|
||||
invoker wraps only Read / Write / Subscribe / Discover / AlarmAcknowledge (`:774, :827, :870,
|
||||
:899, :1081`). A driver that cannot connect therefore contributes **zero** breaker input — which
|
||||
directly refutes the gate's premise.
|
||||
2. **Modbus swallows per-tag failures, in both directions.** `ReadAsync` converts them to
|
||||
`Degraded` health (`:365-375, :838-855`) and `WriteAsync` converts them to a `WriteResult` status
|
||||
code — neither throws. So even the fallback trigger the plan suggests ("a rapid idempotent
|
||||
write-to-dead-endpoint") cannot work for Modbus.
|
||||
|
||||
The breaker itself is configured to open on **2 failures in a 30 s window** at ratio 1.0
|
||||
(`DriverResiliencePipelineBuilder.cs:168-175`), so the previously-recorded explanation — that
|
||||
R2-09's ConnectionBackoff throttles retries below the threshold — is not the whole story and is
|
||||
arguably not the operative one: the failures never enter the pipeline at all.
|
||||
|
||||
**Operator consequence worth recording:** for the scenario operators care about most — a driver that
|
||||
cannot reach its device — the resilience panel is **permanently quiet**. It showed *"All targets
|
||||
healthy."* for a driver that was DEGRADED, timing out every read against a blackhole.
|
||||
|
||||
**What WAS verified live:** the panel renders on `/hosts` (rehomed there by #546 — the plan's "on the
|
||||
driver page" step is stale, v3 Batch 2 retired those pages), the expanded row shows the RESILIENCE
|
||||
block, and `DriverResilienceStatusPublisherService` is publishing real tracker state every 5 s
|
||||
(`BreakerOpen`, `ConsecutiveFailures`, `CurrentInFlight`, `LastBreakerOpenUtc` all observed on the
|
||||
wire). The reader/store/panel chain is live; only the breaker-open **state** remains unforced.
|
||||
|
||||
> Note: `DriverResilienceStatusChanged` going to `$DeadLetterWithNoSubscribers` on the node is
|
||||
> **expected**, not a defect — the rig runs Phase-5 telemetry in `Grpc` mode, where the node always
|
||||
> double-publishes and central consumes via the gRPC dialer, leaving the DPS copy unsubscribed.
|
||||
|
||||
**To actually close T10** someone must either wrap the connect path, stop drivers swallowing
|
||||
capability failures, or pick a driver whose *wrapped* call fails after a successful init (Sql's
|
||||
`ReadAsync` propagates — it needs a backend that dies *after* initialization, e.g. `docker pause` on
|
||||
a dedicated fixture). Left OPEN with this root cause recorded.
|
||||
|
||||
---
|
||||
|
||||
## 5. driver-reconfigure-while-faulted Task 3 ✅ PASSED
|
||||
|
||||
The task was deferred in 2026-06 with "skip the live test part"; its block ("user-driven sign-in")
|
||||
no longer holds because docker-dev runs the AdminUI with `DisableLogin`.
|
||||
|
||||
```
|
||||
before central-1 startedAt=2026-08-07T09:50:29.939876456Z restarts=0
|
||||
10:16:18 DriverInstance DRV-ce2fb5217f5d: initialize failed: Connection refused <- bad config (:5999)
|
||||
10:22:13 DriverInstance DRV-ce2fb5217f5d: connected <- corrected config (:5020)
|
||||
10:22:13 DriverInstance DRV-ce2fb5217f5d: subscribed to 1 refs
|
||||
after central-1 startedAt=2026-08-07T09:50:29.939876456Z restarts=0
|
||||
```
|
||||
|
||||
The driver **adopted the corrected config and connected while faulted**, and `startedAt` is
|
||||
byte-identical before and after with `restarts=0` — **no node restart**, which is the whole point
|
||||
(pre-fix it would retry the stale config forever).
|
||||
|
||||
---
|
||||
|
||||
## 6. Rig state
|
||||
|
||||
Restored to as-found: Modbus device back on `10.100.0.35:5020`, `line3-sql` back on `DevSql`,
|
||||
all `ResilienceConfig` overrides cleared, the `SC-gatevt` script restored to its original body.
|
||||
Post-restore verification: `MainPressure = 7 (Good)`, `GateVt = 7 (Good)` — the VT is back on its
|
||||
original one-line body, so it mirrors `MainPressure` again rather than the `×6` used during the gate.
|
||||
|
||||
One anomaly recorded rather than explained: both centrals were **recreated** at 09:49:40/09:49:50
|
||||
(new `Created`, `RestartCount=0`) by something this session did not issue. It postdates the
|
||||
observations above and the old container's logs went with it, so it is noted, not attributed.
|
||||
@@ -128,6 +128,31 @@ public sealed class VirtualTagActorTests : RuntimeActorTestBase
|
||||
recovered.Value.ShouldBe(42);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// 02/S13: the Bad result must <b>carry the last-known value</b>, not just the Bad status.
|
||||
/// <para>Every other failure assertion in this file checks <c>StatusCode</c> alone, so the
|
||||
/// "carrying the last-known value" half of the S13 contract — stated in
|
||||
/// <c>VirtualTagActor</c>'s own xmldoc and in the R2-03 STATUS.md row — was asserted nowhere.
|
||||
/// Added while live-gating R2-03 T12, where a client observed the VT go Bad with a NULL value
|
||||
/// and there was no test to say whether the actor or the OPC UA layer dropped it.</para>
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void The_Bad_result_carries_the_last_known_value()
|
||||
{
|
||||
var parent = CreateTestProbe();
|
||||
var evaluator = new QueuedEvaluator(
|
||||
VirtualTagEvalResult.Ok(42), VirtualTagEvalResult.Failure("boom"));
|
||||
var actor = parent.ChildActorOf(VirtualTagActor.Props("vt-1", "expr", evaluator: evaluator));
|
||||
|
||||
actor.Tell(new VirtualTagActor.DependencyValueChanged("a", 1, DateTime.UtcNow));
|
||||
parent.ExpectMsg<VirtualTagActor.EvaluationResult>().Value.ShouldBe(42);
|
||||
|
||||
actor.Tell(new VirtualTagActor.DependencyValueChanged("a", 2, DateTime.UtcNow));
|
||||
var bad = parent.ExpectMsg<VirtualTagActor.EvaluationResult>();
|
||||
bad.StatusCode.ShouldBe(OpcUaStatus.BadInternalError);
|
||||
bad.Value.ShouldBe(42);
|
||||
}
|
||||
|
||||
/// <summary>02/S13: an evaluator that THROWS (not just returns Failure) also degrades quality —
|
||||
/// the try/catch path publishes a Bad EvaluationResult too.</summary>
|
||||
[Fact]
|
||||
|
||||
Reference in New Issue
Block a user