dohertj2/lmxopcua
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(archreview): 2026-07-12 re-review at f6eaa267 + 12 round-2 remediation plans

Browse Source 
Re-ran all seven domain reviews at master f6eaa267 (reports rewritten in
place, each with a prior-finding status table): all 4 round-1 Criticals
verified closed; new top findings are the S7 connect-timeout OCE regression
(05/STAB-14), the ResilienceConfig operator-authorable brick (01/S-6), and
a batch of resilience-seam Mediums. 00-OVERALL.md carries the updated
maturity matrix + 12-item action list.

Adds R2-01..R2-12 design/implementation plans (one per action item, house
format + bite-sized TDD task breakdowns + co-located .tasks.json; 193 tasks,
~18-24 dev-days). STATUS.md updated: round-1 topology marked historical
(all merged+pushed), re-review findings table + plan pointers added.
This commit is contained in:
Joseph Doherty
2026-07-12 23:52:23 -04:00
parent f6eaa267a0
commit 1891f5d6a7
34 changed files with 7962 additions and 901 deletions
Download Patch File Download Diff File Expand all files Collapse all files
archreview/00-OVERALL.md
+65 -75
View File
@@ -1,8 +1,10 @@
# OtOpcUa — Deep Architecture Review: Overall Report
**Date:** 2026-07-08
**Commit:** `9cad9ed0` (master, clean tree)
**Method:** Seven parallel domain reviews, each scoring the same four dimensions (stability, performance, conventions, underdeveloped areas) with severity-rated, file:line-referenced findings. This document is the synthesis; the detail lives in the per-domain reports.
**Date:** 2026-07-12
**Commit:** `f6eaa267` (master, clean tree)
**Method:** Seven parallel domain re-reviews, same scopes and four dimensions (stability, performance, conventions, underdeveloped areas) as the original. This is the **first update** of the 2026-07-08 review at `9cad9ed0`: every prior finding was re-verified against code (not against remediation claims), and each domain got a fresh-eyes pass with extra scrutiny on the newly merged remediation code itself. Per-domain reports carry a `Prior-finding status (9cad9ed0 → f6eaa267)` table; remediation history is in [`plans/STATUS.md`](plans/STATUS.md).
**Delta reviewed:** all 11 arch-review remediation branches merged and pushed to `origin/master` (4 Criticals, 3 systemic guards, #10 CapabilityInvoker dispatch wiring, #13 ResilienceConfig artifact plumbing, #11 Crit-1 premise correction, docs branch, docker-dev rig fix), plus the fixdocs XML-doc sweep (`3b5ef439`) and the HistorianGateway Client/Contracts 0.1.0→0.2.0 bump (`f6eaa267`). 84 files, +9218/−217.
## Report index
@@ -16,106 +18,94 @@
| 06 | [Gateway integrations](06-gateway-integrations.md) | Driver.Galaxy(+Browser,+Contracts), Driver.Historian.Gateway |
| 07 | [Client, tooling & engineering system](07-client-tooling-engineering.md) | Client.CLI/Shared/UI, Analyzers, build/CPM, CI, test architecture, repo hygiene |
## Consolidated maturity matrix (1–5)
## Consolidated maturity matrix (1–5, previous score in parentheses)
| Subsystem | Stability | Performance | Conventions | Underdeveloped |
|---|---|---|---|---|
| 01 Core & composition | 4 | 3 | 4 | 3 |
| 02 Scripting & alarms | 3.5 | 3 | 4 | **2** |
| 03 Server & runtime | 3 | 3 | 4 | 3 |
| 04 AdminUI | 4 | 3 | 3 | 3 |
| 05 Protocol drivers | 3 | **2** | 3 | 3 |
| 06 Gateway integrations | 4 | 4 | 4 | 3 |
| 07 Client & engineering | **2.5** | 3.5 | 3 | 3 |
| 01 Core & composition | 4 (4) | 3 (3) | 4 (4) | 3 (3) |
| 02 Scripting & alarms | 3.5 (3.5) | 3 (3) | 4 (4) | 3 (**2**) |
| 03 Server & runtime | 3 (3) | 3 (3) | 4 (4) | 4 (3) |
| 04 AdminUI | 4 (4) | 3 (3) | 3 (3) | 3 (3) |
| 05 Protocol drivers | 3 (3) | **2** (2) | 3 (3) | 3 (3) |
| 06 Gateway integrations | 4 (4) | 4 (4) | 4 (4) | 3 (3) |
| 07 Client & engineering | 3.5 (**2.5**) | 3.5 (3.5) | 4 (3) | 3 (3) |
**Read of the matrix:** the system's *design* is strong — conventions score 3–4 everywhere, layering is disciplined (Akka-free OpcUaServer, SDK-free Runtime, clean gateway seams), and there is essentially zero TODO/stub debt. The weaknesses cluster in *operational* stability (failover, reconnect, timeout paths that only fire under fault) and in the *verification system* (CI proves little; distributed and fault-injection behavior is outside the test net). This is the profile of a codebase built by careful review + live-verification rather than by automated gates — it works, but its correctness under failure is largely unproven.
**Read of the matrix:** the remediation round did exactly what it targeted. The verification system moved most (07 stability 2.5→3.5, conventions 3→4: whole-solution CI, fail-on-skip, tree-wide analyzer — all verified live with negative controls), and the two engine gaps closed (02 underdeveloped 2→3; 03 underdeveloped 3→4 via the failover test + forwarding guard). Nothing regressed at the matrix level. What kept scores from rising further: the untouched High backlog (failure visibility, AdminUI authz, driver perf/concurrency) and **new second-order findings inside the remediation code itself** — the sharpest of which are one High regression (S7) and one operator-authorable driver-brick (ResilienceConfig).
---
## Critical findings (fix first)
## Status of the original four Criticals
These four can each take down or silently corrupt production data flow, and none is caught by any existing test.
| # | Original Critical | Verdict at `f6eaa267` |
|---|---|---|
| 1 | Split-brain resolver never activated | **Premise corrected (#11) + closed** — SBR was already active on master (Akka.Cluster.Hosting applies `SplitBrainResolverOption.Default` → pre-existing akka.conf keep-oldest). Now explicit in code (`KeepOldestOption{DownIfAlone=true}`) and guarded by the hard-kill failover integration test (`HardKillFailoverTests`, runs in CI). |
| 2 | VT script timeout ineffective + ALC cache leak | **Fixed** (`7fd44f0f`) — production path routes through `TimedScriptEvaluator`; `CompiledScriptCache` + apply-boundary clear. Residuals: new 02/S13 (failure/timeout leaves the node stale-Good, below) and 02/S12 (clear-vs-in-flight race). |
| 3 | S7 no reconnect | **Fixed** (`25c0c6f6`, live-verified vs python-snap7) — but the fix carries a **new High regression, 05/STAB-14** (below). |
| 4 | TwinCAT orphaned ADS subscriptions | **Fixed** (`af318fb4`, fake-client verified; no live fixture exists) — residual 05/STAB-16 (replay failure is log-only). |
1. **Split-brain resolver is configured but never activated** — the SBR block exists in `akka.conf:40-46`, but nothing sets `ClusterOptions.SplitBrainResolver` (`src/Core/ZB.MOM.WW.OtOpcUa.Cluster/ServiceCollectionExtensions.cs:80-84`), so Akka runs with default NoDowning. A hard-crashed node is never downed: **cluster singletons and the driver role-leader never fail over on a hard crash**, and a partition leaves both redundancy sides advertising ServiceLevel 240 indefinitely. Invisible to the 2-node harness because it only exercises graceful stops. *(Report 03, S1)*
## Critical/High findings — updated ranking
2. **Production virtual-tag script timeout is ineffective** — `Host/Engines/RoslynVirtualTagEvaluator.cs:102-106` calls `ScriptEvaluator.RunAsync` directly, bypassing `TimedScriptEvaluator` (which exists and is tested). Script execution is synchronous and only checks the token at entry, so one user-authored infinite loop hangs the owning `VirtualTagActor` forever. *(Report 02, U2)*
**New in this round (all inside or activated by the remediation delta):**
3. **S7 driver has no reconnect path** — the PLC connection is opened only in `InitializeAsync` (`Driver.S7/S7Driver.cs:165-167`); the probe detects the outage but nothing repairs it. A PLC reboot permanently kills the driver instance until redeploy. Zero S7 reconnect tests. *(Report 05)*
1. **05/STAB-14 (High, regression)** — S7's ensure-wrapper rethrows connect-*timeout* `OperationCanceledException` unconditionally, and `PollLoopAsync`'s bare `catch (OCE) { return; }` then **permanently, silently kills every S7 subscription poll** after an unreachable-host outage (PLC dead-to-SYN, not refusing). The live bounce test can't see it — `docker restart` yields connection-refused, not timeout. Two-line filter fix + a fake-factory connect-timeout test (`S7Driver.cs:488, 1000, 1383, 1404, 1206-1208`).
2. **01/S-6 (High)** — a valid-JSON `ResilienceConfig` with `timeoutSeconds: 0` (or out-of-range) passes the parser but makes Polly throw `ValidationException` at pipeline build **on every capability call** — an operator-authorable permanent driver-brick (the AdminUI form has no range validation), violating the parser's explicit never-brick contract (`DriverResilienceOptionsParser.cs:91-95`, `DriverResiliencePipelineBuilder.cs:111-114`, `ResilienceFormModel.cs:32-37`).
3. **05/STAB-15 (High)** — S7's `IsS7ConnectionFatal` misses framing/desync errors and cancellation-during-I/O (half-read PDU kept → permanent stream desync — the Modbus STAB-3 failure mode reproduced), and the probe catch never marks the handle dead (`S7Driver.cs:1239-1251, 1542`).
4. **03/S12 = 01/S-8 (Medium, two reviewers independently)** — write dispatch hardcodes `isIdempotent: true`; `WriteIdempotentAttribute` has zero readers, so the moment an operator enables Write retries via #13, **command-shaped writes get retried** (`DriverInstanceActor.cs:597-601`).
5. **02/S13 (Medium)** — the timeout fix converted a visible hang into an invisible fault: a script that starts failing/timing out never degrades node quality, leaving the last Good value **stale-Good forever** (`VirtualTagActor.cs:123-128`; the dormant engine maps to BadInternalError, the live path doesn't).
6. **01/U-6 (Medium)** — the resilience **bulkhead is a fully dead knob**: documented, defaulted, parsed, and authorable in AdminUI, but `Build` never adds a concurrency limiter (`DriverResiliencePipelineBuilder.cs:99-177`).
7. **06/S-11 (Medium)** — an enabled-but-misconfigured `ServerHistorian` still crash-loops the Host with a raw `UriFormatException` (proven live in the #12 rig session; only the config side was fixed) (`HistorianGatewayClientAdapter.cs:45`).
8. **06/U-7 (Medium)** — the 0.2.0 bump's proto3-optional `HistorianTagDefinition.DataType` change silently alters `GatewayTagProvisioner` semantics (Boolean was wire-0 → provisioned as Float under 0.1.0, honored as Int1 under 0.2.0); pre-bump tags mismatch and the live EnsureTags leg has never run on 0.2.0 (`GatewayTagProvisioner.cs:57`).
9. **04/C-7 + 04/U-5 + 03/U10 (Medium)** — the resilience authoring/observability surface is half-built now that #10/#13 made it production-live: the AdminUI form round-trip is lossy (malformed JSON silently overwritten, unknown keys stripped), and `DriverResilienceStatusTracker` has **no reader anywhere** (Admin `/hosts` Stream E.2/E.3 never shipped) — operators author live retry/breaker config blind.
4. **TwinCAT native ADS subscriptions are orphaned after reconnect** — the default subscription mode is never re-registered on the replacement client after reconnect (`TwinCATDriver.cs:615-619`, `AdsTwinCATClient.cs:422-436`); data silently stops with no Bad status. *(Report 05)*
**Carried open from 2026-07-08 (re-verified, no remediation branch targeted them):**
## High-severity findings (system-wide risk ranking)
**Data-integrity / failure-visibility group** — the system tends to report success it hasn't earned:
- **Deploy applier swallows sink failures**: a deploy whose `RebuildAddressSpace()` threw still reports success to the deploy coordinator; `AddressSpaceApplyOutcome` has no failure field (`AddressSpaceApplier.cs:150-153, 373-383`). *(01, S-1)*
- **Galaxy write success is optimistic**: empty status arrays translate to Good, and a failed supervisory-advise still lets the write "proceed" though it can never reach the Galaxy — write-outcome self-correction structurally cannot fire for Galaxy (`GatewayGalaxyDataWriter.cs:163-169, 234-243, 281-302`). *(06, S-1)*
- **Primary gate defaults to allow while role is unknown** (`DriverHostActor.cs:1018-1026`): a booting secondary services field-device writes for up to a 10 s window; compounds with at-most-once DPS delivery. *(03, S4)*
- **Modbus transport desyncs permanently after a response timeout** — timeout/TxId-mismatch aren't classified socket-fatal, leaving the single-flight stream misaligned (`ModbusTcpTransport.cs:155-165, 233-235`). *(05)*
- **AbCip races concurrent ops on shared libplctag handles with no lock** (`AbCipDriver.cs:544-575`) — torn values with Good status; AbLegacy guards this exact hazard. FOCAS has the sibling bug on plain-Dictionary caches. *(05)*
**Availability / blocking group:**
- **LDAP authentication block-bridges OPC UA session activation** with an unconfigurable 10 s connect timeout and no pooling (`OpcUaApplicationHost.cs:271-273`) — an LDAP outage serially stalls SDK threads. *(03, S2)*
- **HistoryRead block-bridges per node sequentially** on SDK request threads bounded only by the 30 s gateway timeout — large batches × slow historian can exhaust the request pool (`OtOpcUaNodeManager.cs:1902-2203`). *(03, S3)*
- **Any structural deploy change (one added tag) full-rebuilds the address space and kills every client subscription server-wide** — the surgical path covers only attribute edits/renames. Surgical pure-adds are the highest-leverage performance fix in the repo. *(03, P1)*
- **Evaluator-cache ALC accretion**: the live path caches script evaluators in a raw `ConcurrentDictionary` never cleared on republish, leaking collectible AssemblyLoadContexts across script edits; the purpose-built `CompiledScriptCache` is unused. *(02, U3)*
**Security / authorization group:**
- **AdminUI's largest mutating surface is effectively ungated** — GlobalUns, EquipmentPage, ClusterEdit, NodeEdit, all 8 driver pages, AclEdit, and Reservations carry only bare `[Authorize]`: any authenticated user, including read-only, can mutate UNS/driver/ACL config. Authorization is a three-idiom mix (bare / role-string / policy). *(04, #1)*
**Verification-system group:**
- **CI gates only 5 of 47 test projects** (`.github/workflows/v2-ci.yml:47-52`); Client (388 tests), Tooling, all driver and most Core suites never run in CI. The integration job runs fixture-less, so probes `Assert.Skip` and the badge is green — "skipped" is indistinguishable from "passed". *(07, S-1/S-2)*
- **The sole custom analyzer (OTOPCUA0001) is referenced by zero consuming projects** — the CapabilityInvoker-wrapping rule it enforces is inert in every build. One `ProjectReference` in `Directory.Build.props` fixes it. *(07, C-1)*
- **Failure-visibility group** — applier swallows sink failures / no failure field in `AddressSpaceApplyOutcome` (01/S-1); Galaxy write success optimistic (06/S-1); primary gate default-allow on unknown role (03/S4). STATUS.md queues these as "next"; still the top carried batch.
- **04/C-1 (High)** — AdminUI's largest mutating surface still bare `[Authorize]` (GlobalUns, EquipmentPage, cluster/driver editors, Reservations); three-idiom authz mix. Stakes **rose** this round: the ungated surface now includes live resilience config.
- **Availability** — LDAP block-bridging (03/S2), HistoryRead block-bridging (03/S3), full-rebuild-per-tag-add killing all subscriptions (03/P1 — now unblocked by the U2 forwarding guard, work not started).
- **Driver fleet** — Modbus timeout/TxId desync (05/STAB-3), AbCip shared-handle race / FOCAS dictionary race, `ResolveHost` mis-keying, dead `onError` sink, `ReadEnum` ×6, silent enum defaulting on TagConfig paths.
- **Repo hygiene** — 7 AVEVA DLLs in `lib/`, Wonderware husk dirs, tracked planning files, 5 undocumented CLI commands — all untouched (07).
---
## Cross-cutting themes
## Cross-cutting themes — updated
These emerged independently from multiple reviewers and are the real architectural signal — individual bugs are symptoms of these.
### 1. "Built-but-never-wired" — the guards work, and the pattern still recurred
The two systemic guards shipped and are live (analyzer tree-wide with negative control; reflection-exhaustive sink-forwarding guard; whole-solution CI + fail-on-skip). The analyzer immediately earned its keep by surfacing the RESILIENCE-DISPATCH-GAP, which was then genuinely closed (`bacea1a4` — pragmas removed, not suppressed). **But the pattern reproduced inside the remediation itself**: the bulkhead knob parses-but-never-builds (01/U-6), `WriteIdempotentAttribute` has zero readers (01/S-8), and `DriverResilienceStatusTracker` has no reader (03/U10). The guards catch *interface-forwarding* and *unwrapped-dispatch* inertness; they don't catch *option-parsed-but-unapplied* inertness. A cheap next guard: a test asserting every parsed `DriverResilienceOptions` property influences the built pipeline.
### 1. "Built-but-never-wired" is the house failure mode
At least six independent instances: the dormant `GatewayTagProvisioner` (PR #423, since fixed), the F10b `DeferredAddressSpaceSink` inertness bug (fixed), the analyzer wired to zero projects, `GatewayHistorianDataSource.RefreshConnectionStateAsync` with no production caller (health flags permanently false), the SBR HOCON block with no activating registration, and the entire ~2.4k-line `Core.VirtualTags` engine stack that production bypasses in favor of `VirtualTagHostActor` + `RoslynVirtualTagEvaluator` — which is precisely why the timeout and cache defenses (built into the dormant engine's path) are missing from the live one. **Recommendation:** every hook/capability/config block needs a *production-caller* assertion, not just unit tests of the component — e.g. a reflection-exhaustive forwarding test for `DeferredAddressSpaceSink` (only ~5 of 10 members are asserted today), a startup log line proving SBR is active, and wiring the analyzer repo-wide since it was built to catch exactly this class.
### 2. Fixes that can't be seen by their own verification
The round's sharpest lesson: **05/STAB-14 sits exactly in the blind spot of the live test that validated the fix** (bounce = refused; the bug needs timeout). Same genre: the TwinCAT replay is fake-client-only verified (no fixture exists), and 02/S13 turned a loud failure (hang) into a silent one (stale-Good). When a fix targets a fault path, the verification must enumerate the fault *variants* (refused vs timeout vs desync vs cancel), not just one reproduction.
### 2. Fixes don't flow back to shared templates
Every High in the driver fleet is a fix present in one sibling and absent in another: S7 forked the poll loop (gaining backoff the shared `PollGroupEngine` lacks, keeping a CTS race the engine fixed); AbLegacy guards the handle race AbCip has; OpcUaClient's reconnect machine is exemplary while S7 has none; the `ReadEnum` helper is copy-pasted 6×; no driver passes `onError` to the poll engine so its error sink is dead fleet-wide. The same pattern exists in Core as **duplication-by-convention**: TagConfig intent parsing is byte-parity-replicated in four projects, synced only by "MUST parse identically" comments. **Recommendation:** consolidate into `TagConfigIntent.Parse` in Commons (the `EquipmentScriptPaths` precedent) and treat `PollGroupEngine` + a shared reconnect/backoff primitive as the single home for fleet fixes.
### 3. Optimistic success — unchanged, and now with new instances
The carried trio (applier, Galaxy writes, primary gate) remains open, and the round added stale-Good VT values (02/S13), log-only TwinCAT replay failure (05/STAB-16), and blind-authored resilience config (04/U-5). This is now the largest coherent batch of open work.
### 3. Optimistic success — failure signals don't reach operators
The "Safe*" swallow-and-log wrapper pattern (never fail the caller) recurs across the applier, Galaxy writes, and the primary gate's default-allow. Individually defensible; together they mean a failed deploy reports `rebuilt=true`, a lost Galaxy write reports Good, and a booting secondary accepts writes. **Recommendation:** audit each swallow site for an operator-visible signal (health flag, meter, alert row) — the pattern is fine only when the failure surfaces *somewhere*.
### 4. The verification system is materially better
Biggest mover: all 47 test projects gate CI, skips can't masquerade as green (strict on the unit leg), the analyzer is load-bearing, and the failover + forwarding guards exist with negative controls. Remaining: E2E workflow still gates zero tests (07/S-3), integration matrix runs redundant fixtureless legs (07/P-5), driver suites still carry ~100 fixed sleeps now exposed in CI, no bUnit (04/U-1), and behavioral wiring tests cover only 2 of 6 dispatch sites (03/U6).
### 4. The verification system trusts live-verification that CI can't reproduce
Repo-wide: no bUnit (razor `@code` logic is only live-verified), CI runs ~10% of suites, integration tier skip-gates to green, DPS delivery/actor supervision/hard-kill failover have zero tests, `OpcUaServer.IntegrationTests` has exactly one test, and driver test debt concentrates on reconnect/concurrency — exactly where the four Criticals live. The project's own history (F10b, redundancy-state delivery, enum serialization) shows live-`/run` catching what 250+ green unit tests missed. **Recommendation:** expand `v2-ci.yml` to all unit suites (cheap, immediate), make skips visible (report skipped counts as a failure condition for the integration job), and add a small fault-injection tier (hard-kill a node, drop a PLC connection) even if only run nightly against the docker host.
### 5. Documentation drift — mostly closed, one new instance
CLAUDE.md's Known Limitation 2 and the ScriptAnalysis policy drift are fixed and verified. New drift: `archreview/plans/STATUS.md` merged to master still claiming "all LOCAL — nothing pushed" (corrected alongside this report), and the C6 "Phase 7 plan" residue survived fixdocs.
### 5. Documentation drift on load-bearing claims
CLAUDE.md's KNOWN LIMITATION 2 (empty historized-ref set) is **stale — closed in code** (applier `FeedHistorizedRefs` → recorder `UpdateHistorizedRefs`), confirmed independently by two reviewers; ScriptAnalysis endpoints are documented as FleetAdmin-gated but use `Roles="Administrator,Designer"`; `docs/Client.CLI.md` documents 8 of 14 commands. **Recommendation:** update CLAUDE.md now (and add the missing restart-convergence test + live value-capture verification that the limitation's closure deserves).
### 6. Equipment-tag (TagConfig-JSON) paths are second-class citizens
Across drivers, the equipment-tag parse/resolve path lags the authored-tag path: silent enum defaulting on typos, forced `Writable:true` (FOCAS is read-only yet defaults writable), `ResolveHost` mis-keying refs to the first device (breaking per-host breaker isolation in 4 drivers), and capability gates bypassed. Since Galaxy-standard-driver made TagConfig the primary authoring surface, this path *is* the product. **Recommendation:** a dedicated hardening pass on the `EquipmentTagRefResolver` seam across all drivers, plus the parse-strictness unification from theme 2.
### 7. Repo hygiene debt
Retired `Historian.Wonderware*` directories still on disk (absent from slnx); 7 committed proprietary AVEVA DLLs in `lib/` referenced by nothing (license/redistribution risk); `pending.md`/`current.md` tracked despite pending.md's own "never stage" rule; five stale planning files at root. (`sql_login.txt` was verified NOT committed — gitignored, local-only.)
### 6. The resilience seam is the new second-class citizen
Six of this round's nine notable new findings live on the #10/#13 resilience surface. The seam's architecture is clean (correct Polly-free layering, null objects, actor-context preservation — all verified), but its edges are unfinished: no range validation (S-6), dead bulkhead (U-6), hardcoded idempotency (S-8), lossy form (C-7), no status reader (U-5), stale-pipeline respawn race (S-7/S13-03). One focused hardening pass closes all of them.
---
## Prioritized action list
## Prioritized action list (updated)
| # | Action | Source | Effort | Impact |
|---|---|---|---|---|
| 1 | Activate the split-brain resolver (`ClusterOptions.SplitBrainResolver`) + a hard-kill failover test | 03 S1 | Small | Critical — failover currently doesn't work |
| 2 | Route production VT evaluation through `TimedScriptEvaluator` + `CompiledScriptCache` | 02 U2/U3 | Small | Critical — user script can hang an actor; ALC leak |
| 3 | Add S7 reconnect + TwinCAT ADS re-registration after reconnect (use OpcUaClient's machine as the template) | 05 | Medium | Critical — PLC power-cycle currently defeats both |
| 4 | Classify Modbus timeout/TxId-mismatch as socket-fatal; lock AbCip handles (copy AbLegacy's guard) | 05 | Small | High — silent data corruption/desync |
| 5 | Gate AdminUI mutating pages with real policies (one authz idiom, constants not literals) | 04 | Medium | High — any authenticated user can mutate config |
| 6 | Wire OTOPCUA0001 analyzer repo-wide; expand CI to all unit suites; fail-on-skip for integration | 07 | Small | High — restores meaning to green CI |
| 7 | Add a failure field to `AddressSpaceApplyOutcome`; surface Galaxy write-loss and gate default-deny on unknown role | 01/06/03 | Medium | High — failure visibility |
| 8 | Surgical pure-adds in the address-space applier (stop full rebuilds killing all subscriptions per tag add) | 03 P1 | Large | High — biggest operational perf win |
| 9 | Async LDAP + configurable timeout; channel-ize HistoryRead bridging | 03 S2/S3 | Medium | Medium-High — outage resilience |
| 10 | Consolidate TagConfig parsing (`TagConfigIntent.Parse` in Commons) + driver `ReadEnum`/parse-strictness unification + `ResolveHost` fix | 01 C-1 / 05 | Medium | Medium — kills the byte-parity duplication class |
| 11 | Update CLAUDE.md (Known Limitation 2 closed; ScriptAnalysis policy) + Client.CLI docs; delete Wonderware dirs, `lib/` AVEVA DLLs, untrack planning files | 03/04/06/07 | Small | Medium — doc trust + license risk |
| 12 | Retire or converge the dormant `Core.VirtualTags` engine; delete `MemoryRecycle`/`IDriverSupervisor` dead machinery | 02 U1 / 01 U-2 | Medium | Medium — removes the built-vs-runs divergence |
| 1 | S7: filter connect-timeout OCE in the ensure-wrapper + poll-loop catches; broaden fatal classification (framing/cancel); mark handle dead in probe catch. Add fake-factory connect-timeout test | 05 STAB-14/15 | Small | **Critical-adjacent — regression silently kills S7 subscriptions** |
| 2 | ResilienceConfig hardening pass: range-validate in parser (clamp + diagnostic, never brick) and AdminUI form; wire or delete the bulkhead knob; read `WriteIdempotentAttribute` (or default writes non-idempotent); fix lossy form round-trip | 01 S-6/U-6/S-8, 04 C-7 | Small-Medium | High — operator-authorable driver-brick; command writes retried |
| 3 | Publish Bad quality on VT script failure/timeout (match the dormant engine's BadInternalError); optionally gate the apply-boundary cache clear on changed scripts | 02 S13/P7/S12 | Small | High — silent stale-Good data |
| 4 | Failure-visibility trio: `AddressSpaceApplyOutcome` failure field; Galaxy fail-closed writes; primary-gate default-deny | 01 S-1, 06 S-1, 03 S4 | Medium | High — carried from round 1 |
| 5 | Gate AdminUI mutating pages with real policies (stakes rose: resilience config is on the ungated surface) | 04 C-1 | Medium | High |
| 6 | ServerHistorian fail-fast options validation (misconfig → clear startup error, not crash-loop); assess 0.2.0 DataType provisioning delta + run the live EnsureTags leg | 06 S-11/U-7 | Small | Medium-High |
| 7 | Surgical pure-adds in the applier (prerequisite guard in place since round 1) | 03 P1 | Large | High — biggest operational perf win |
| 8 | Async LDAP + configurable timeout; channel-ize HistoryRead bridging | 03 S2/S3 | Medium | Medium-High |
| 9 | Driver fleet carried batch: Modbus fatal-classification, AbCip/FOCAS locking, `ResolveHost`, `onError`, TwinCAT replay retry + Bad-quality | 05 | Medium | Medium-High |
| 10 | Resilience observability: a `/hosts` reader for `DriverResilienceStatusTracker` (Stream E.2/E.3) | 03 U10, 04 U-5 | Medium | Medium — operators currently blind |
| 11 | TagConfig parse consolidation (`TagConfigIntent.Parse` in Commons) + `ReadEnum` unification | 01 C-1/P-1, 05 | Medium | Medium |
| 12 | Hygiene: delete `lib/` AVEVA DLLs + Wonderware husks, untrack planning files, document 5 CLI commands, E2E workflow or delete it, dedupe CI integration matrix | 07 | Small | Medium — license risk + doc trust |
## Overall assessment
OtOpcUa is a well-architected system with unusually disciplined conventions, clean layering, strong seams, and near-zero TODO debt — the design work is mature. Its exposure is concentrated in three places: **fault paths** (failover, reconnect, timeout — where all four Criticals live and where test coverage is thinnest), **failure visibility** (a systematic bias toward optimistic success), and **the verification gap** (CI proves ~10% of what the test suite could; the "built-but-never-wired" pattern has now recurred six times because nothing asserts production wiring). The prioritized list above front-loads small, high-leverage fixes: items 1–6 are roughly a week of work and eliminate every Critical plus the two systemic guards (analyzer + CI) that prevent the pattern from recurring.
The remediation round was real: all four Criticals are verifiably closed in code (one by premise-correction), both systemic guards are live and already caught one genuine gap, and the verification system went from "proves ~10%" to whole-solution with skip-detection — the single biggest maturity move available, and it happened. The honest counterweight is that this round's scrutiny of the fixes found what fixes always carry: **one High regression (S7 timeout-vs-refused), one High operator-authorable brick (ResilienceConfig validation), and a cluster of half-wired edges on the new resilience seam** — the house "built-but-never-wired" pattern reproducing inside the code meant to eliminate it. Items 1–3 above are small and close everything new-and-sharp from this round; items 4–5 are the carried Highs that have now waited two rounds and should not wait a third.