Commit Graph

2142 Commits

Author SHA1 Message Date
Joseph Doherty b5bf4b73ff fix(adminui): gate every page with named policies — ConfigEditor on the mutating surface (R2-05, 04/C-1) 2026-07-13 09:56:08 -04:00
Joseph Doherty 183b72b7cb feat(security): add ConfigEditor + AuthenticatedRead AdminUI policies behind AdminUiPolicies constants 2026-07-13 09:52:13 -04:00
Joseph Doherty 1676c8f40f Merge branch 'docs/archreview-r2-rereview-and-plans'
v2-ci / build (push) Successful in 3m10s
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Failing after 4m16s
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Failing after 1m46s
v2-ci / unit-tests (push) Failing after 8m29s
2026-07-12 23:52:32 -04:00
Joseph Doherty 1891f5d6a7 docs(archreview): 2026-07-12 re-review at f6eaa267 + 12 round-2 remediation plans
Re-ran all seven domain reviews at master f6eaa267 (reports rewritten in
place, each with a prior-finding status table): all 4 round-1 Criticals
verified closed; new top findings are the S7 connect-timeout OCE regression
(05/STAB-14), the ResilienceConfig operator-authorable brick (01/S-6), and
a batch of resilience-seam Mediums. 00-OVERALL.md carries the updated
maturity matrix + 12-item action list.

Adds R2-01..R2-12 design/implementation plans (one per action item, house
format + bite-sized TDD task breakdowns + co-located .tasks.json; 193 tasks,
~18-24 dev-days). STATUS.md updated: round-1 topology marked historical
(all merged+pushed), re-review findings table + plan pointers added.
2026-07-12 23:52:23 -04:00
Joseph Doherty f6eaa267a0 chore(deps): bump ZB.MOM.WW.HistorianGateway Client + Contracts 0.1.0 -> 0.2.0
v2-ci / build (push) Successful in 3m21s
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Failing after 4m11s
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Failing after 1m43s
v2-ci / unit-tests (push) Failing after 9m18s
Consume the published 0.2.0 gateway packages. M3 opc_quality gate checked
and CLEAR: OtOpcUa never writes HistorianHistoricalValue (uses only the
WriteLiveValues path, no opc_quality field), and every OpcQuality=0 is on
the read-side HistorianSample in test fixtures — unaffected by 0.2.0's
HistorianHistoricalValue.opc_quality proto3-optional change. Build clean;
gateway-driver 99/0 and runtime 364/0 tests pass.
2026-07-10 07:50:55 -04:00
Joseph Doherty 3b5ef439d4 docs(xmldoc): fix genuine doc issues surfaced by /fixdocs (17 fixes)
v2-ci / build (push) Successful in 3m11s
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Failing after 3m51s
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Failing after 1m50s
v2-ci / unit-tests (push) Failing after 8m37s
- CapabilityInvoker: 3 methods that implement IDriverCapabilityInvoker now use
  <inheritdoc /> instead of duplicating the interface docs (MissingInheritDoc).
- TwinCAT NativeRegistration ctor: added the 8 missing <param> tags (MissingParam).
- Stripped the arch-review bookkeeping IDs STAB-1 (IS7Plc, S7Driver) + STAB-2
  (TwinCATDriver x4) from comments, keeping the descriptive text.

Left the other 86 flagged items untouched — they are checker false positives on
legitimate technical content: interface-implemented events / IEqualityComparer /
IDisposable members whose <inheritdoc/> IS correct (10 InheritDocMisused), and domain
terms the heuristic misreads as tracking IDs (IEEE-754, S7-1200/300/400/1500, PLC-5,
D2-260, S7/Melsec memory addresses, OPC UA Part 9, phase-N design-doc filename refs,
UNS level-N descriptions, and prose like 'the #1 bug source' / failover 'pick #3').
2026-07-10 06:23:12 -04:00
Joseph Doherty b67bd9e8b5 Merge branch 'docs/archreview-plans-and-drift-fixes'
v2-ci / build (push) Failing after 45s
v2-ci / unit-tests (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
2026-07-09 06:07:34 -04:00
Joseph Doherty 896eef0591 Merge branch 'fix/docker-dev-serverhistorian-stale-keys' 2026-07-09 06:07:34 -04:00
Joseph Doherty 83fdf5fa6c Merge branch 'fix/archreview-s1-s2-ci-coverage' 2026-07-09 06:07:33 -04:00
Joseph Doherty 1d7e7caa41 Merge branch 'fix/archreview-u2-deferred-sink-forwarding-guard' 2026-07-09 06:07:33 -04:00
Joseph Doherty 8f7b0017ef Merge branch 'fix/archreview-crit4-twincat-replay' 2026-07-09 06:07:33 -04:00
Joseph Doherty dfc02af3f0 Merge branch 'fix/archreview-crit3-s7-reconnect' 2026-07-09 06:07:33 -04:00
Joseph Doherty 45455b7677 Merge branch 'fix/archreview-crit2-vt-timeout' 2026-07-09 06:07:33 -04:00
Joseph Doherty 0ed30483b4 Merge branch 'fix/archreview-crit1-split-brain-resolver' 2026-07-09 06:07:33 -04:00
Joseph Doherty 025aac4d58 Merge branch 'fix/archreview-crit13-resilience-config-artifact' 2026-07-09 06:07:33 -04:00
Joseph Doherty dfeb21902d Merge branch 'fix/archreview-crit10-wire-capability-invoker' 2026-07-09 06:07:33 -04:00
Joseph Doherty 7834b3b71f Merge branch 'fix/archreview-c1-wire-analyzer' 2026-07-09 06:07:33 -04:00
Joseph Doherty 357709bc71 docs(archreview): docker-dev ServerHistorian rig bug FIXED (branch fix/docker-dev-serverhistorian-stale-keys 7233e2ba)
Flip the #12 rig-bug note from 'recommend fixing' to fixed: FOLLOWUP-10 + STATUS task #12
+ new branch topology row. central-1 verified booting clean with the committed compose.
2026-07-08 23:43:56 -04:00
Joseph Doherty 7233e2ba08 fix(docker-dev): replace retired Wonderware ServerHistorian keys with the gateway shape (disabled by default)
The historian-gateway cutover made ServerHistorianOptions read Endpoint/ApiKey/UseTls,
but docker-dev compose still set Enabled=true with the retired Wonderware Host/Port/
SharedSecret keys → HistorianGatewayClientAdapter.Create built a Uri from an empty
Endpoint → unhandled UriFormatException crashed the Host at every bring-up, leaving
docker-dev unbootable against current code.

central-1 + central-2 now use the gateway-shape keys, DISABLED by default (the rig has no
historian gateway): Enabled=${OTOPCUA_HISTORIAN_ENABLED:-false}, Endpoint/ApiKey via env,
UseTls + AllowUntrustedServerCertificate. To enable, set OTOPCUA_HISTORIAN_ENABLED=true +
OTOPCUA_HISTORIAN_ENDPOINT + export ServerHistorian__ApiKey (never committed). Verified:
compose config valid, central-1 boots clean (no UriFormatException; OPC UA + AdminUI serving).

Found during the arch-review #10 live-gate rig session; branched off master (independent
of the archreview work).
2026-07-08 23:43:02 -04:00
Joseph Doherty 347446d627 docs(archreview #12): record live-gate outcome (DI wiring confirmed live; behavioral log a disciplined stop)
Ran the #10 live gate: built the Host image from source (crit13 tip; Gitea feed answers
200 anonymously so restore worked), stood up the docker-dev rig, central-1 booted healthy
on the new invoker wiring. Prod DI binds the REAL DriverCapabilityInvokerFactory —
confirmed live + by a new deterministic guard (ResilienceInvokerFactoryRegistrationTests,
crit13 8a8b9ec5). The behavioral retry/breaker log-line was NOT observed: reading the
Modbus driver showed its cheap paths don't throw a wrapped exception (Discover offline,
Subscribe lazy, reads non-wrapped) — only an idempotent write-to-dead-endpoint or S7 does,
not worth the multi-dependency chase since the retry behavior is deterministically proven.

Also found a rig bug: docker-dev compose's retired Wonderware ServerHistorian keys
(Host/Port/SharedSecret, Enabled=true) crash the Host at bring-up (historian-gateway cutover
now reads Endpoint -> empty UriFormatException). Worked around with an override; recommend
fixing the committed compose. FOLLOWUP-10 + STATUS updated; crit13 topology head -> 8a8b9ec5.
2026-07-08 23:25:31 -04:00
Joseph Doherty 8a8b9ec564 test(archreview #10/#13): guard prod DI binds the REAL invoker factory
AddOtOpcUaDriverFactories (the driver-node bootstrap) must bind the real
DriverCapabilityInvokerFactory — not the NullDriverCapabilityInvokerFactory
pass-through — and it must mint a real CapabilityInvoker per instance. This is the
deterministic DI half of the #10 live gate: a refactor dropping back to the
pass-through would make the whole retry/breaker/bulkhead pipeline inert in production
while every pass-through-defaulting unit test stays green. 3 tests (real factory bound,
real invoker minted, shared singletons), non-fixtured. Host.IntegrationTests.
2026-07-08 23:16:23 -04:00
Joseph Doherty 051a59845d docs(archreview): record #13 IMPLEMENTED (ResilienceConfig read-path plumbing)
- New FOLLOWUP-13: the DriverInstance.ResilienceConfig column was authored/persisted
  and already in the artifact; only the driver-node read path dropped it. Read-path
  plumbing + invalidate-on-change + respawn-on-change; deterministic verification.
- FOLLOWUP-10 sub-gap (a) 'tier-defaults only' → RESOLVED by #13 (sub-gap (b) tracker
  reader still open).
- STATUS.md: task #13 → completed; crit13 branch topology row; Completed-table row;
  findings #10 bullet updated; FOLLOWUP-13 linked.
2026-07-08 22:51:15 -04:00
Joseph Doherty 75403caa1a feat(archreview #13): apply per-instance ResilienceConfig from the deploy artifact
The DriverInstance.ResilienceConfig column was authored in AdminUI, persisted to the
entity, and serialized into the deployment artifact by ConfigComposer — but the runtime
read path dropped it: DriverInstanceSpec didn't carry it and the invoker factory always
passed null, so every driver got tier defaults regardless of its configured overrides
(a silent dead-config gap — #10's residual sub-finding).

Read-path plumbing (write side was already complete):
- DriverInstanceSpec gains ResilienceConfig; DeploymentArtifact.TryReadSpec reads the column.
- IDriverCapabilityInvokerFactory.Create takes resilienceConfigJson; DriverHostActor.SpawnChild
  threads spec.ResilienceConfig; the concrete factory parses it (ParseOrDefaults, layering on the
  tier), logs any parse diagnostic (never throws), and builds the invoker with the merged options.
- Invalidate-on-change: the pipeline cache keys on (instance, host, capability) and ignores options
  on a hit, so Create() now Invalidate()s the instance's cached pipelines first (no-op on first
  spawn) — a respawn with changed options rebuilds them.
- DriverSpawnPlanner treats a ResilienceConfig change as a stop+respawn (the invoker/options are
  bound to the child at spawn); a pure DriverConfig change stays an in-place delta (no reconnect).
- Host DI passes a logger to the factory for the parse diagnostic.

Verification (deterministic): factory Create applies a retryCount:0 override to actual execution
(control test proves tier default retries), invalidates the instance's cache on re-create (scoped —
sibling survives), malformed config logs+falls-back; planner respawns on ResilienceConfig change
(incl null→json) and stays delta on a pure config change; artifact parse carries/omits the column.
Core.Tests 243 (+5), Runtime.Tests 363 (+5), Host builds clean.
2026-07-08 22:48:17 -04:00
Joseph Doherty e233db7456 docs(archreview): record #11 CORRECTED (Crit-1 premise) + track residual #12/#13
- FOLLOWUP-11 → IMPLEMENTED with outcome (docs/comment-only, no code revert; the four
  corrected sites; Cluster.Tests 29/29).
- STATUS.md: task #11 → completed; crit1 branch head → eaf78aad; Completed-table row;
  Findings #11 bullet flipped to CORRECTED; added task rows #12 (live gate) + #13
  (ResilienceConfig-in-artifact).
2026-07-08 22:36:10 -04:00
Joseph Doherty eaf78aad90 docs(archreview #11): correct Critical 1 premise — SBR was already active on master, not NoDowning
Akka.Cluster.Hosting's WithClustering enables an SBR downing provider by default
(applies SplitBrainResolverOption.Default when ClusterOptions.SplitBrainResolver is
null), which reads the pre-existing akka.conf keep-oldest block. So the cluster was
NOT running NoDowning before Critical 1 and hard-crash failover already worked — the
typed KeepOldestOption is reinforcing/explicit-in-code, not the sole activator.

Corrects the inaccurate 'HOCON inert / NoDowning / never fails over' framing in:
- ServiceCollectionExtensions.BuildClusterOptions XML comment
- akka.conf split-brain-resolver comment
- docs/Redundancy.md Split-brain section
- SplitBrainResolverActivationTests summary + assertion message (+ method rename)

No code revert (the typed option is correct belt-and-suspenders). Cluster.Tests 29/29.
Surfaced by the #9 hard-kill failover negative control.
2026-07-08 22:34:20 -04:00
Joseph Doherty 877cb9bc56 docs(archreview): record #10 IMPLEMENTED (Core.Abstractions seam) + residual findings
Updates FOLLOWUP-10 + STATUS.md now that #10 is wired:
- The filed "thread CapabilityInvoker into the actor" was infeasible — Runtime is
  deliberately Polly-free (refs Core.Abstractions, not Core). Shipped a seam mirroring
  IDriverFactory (branch fix/archreview-crit10-wire-capability-invoker @ 62556c24).
- Triage corrections: only 6 of 7 sites are real (GenericDriverNodeManager is test-only
  scaffolding, re-annotated); tier-defaults only (ResilienceConfig not in the artifact).
- Observability finding: the resilience-status tracker has no reader (Stream E.2/E.3
  never shipped) → added retry/breaker logging as the interim surface + live-verify enabler.
- Verification: negative-control, recording-invoker routing, real-invoker actor-context,
  analyzer + logging tests; clean solution build; Runtime 358 / Core 238 / Analyzers 32.
- Residual: live behavioral gate (rig recipe recorded) + the ResilienceConfig-in-artifact
  sub-gap follow-up.
2026-07-08 21:42:17 -04:00
Joseph Doherty 62556c245a test(archreview #10): cover actor-context preservation through the REAL CapabilityInvoker
The pass-through NullDriverCapabilityInvoker returns the call-site task directly, so it
never exercises the real invoker's internal `pipeline.ExecuteAsync(...).ConfigureAwait(false)`.
Adds a test-only Core reference to Runtime.Tests (the PRODUCTION Runtime assembly stays
Polly-free via the seam) and a discovery test that drives DriverInstanceActor through a real
CapabilityInvoker over a genuinely-yielding ITagDiscovery driver — proving the invoker's
internal ConfigureAwait(false) does NOT leak to the actor's own await, so the post-await
Context.Parent.Tell still runs (DiscoveredNodesReady arrives). Runtime.Tests 358, Core.Tests 238.
2026-07-08 21:38:14 -04:00
Joseph Doherty bacea1a44f fix(archreview #10): wire CapabilityInvoker into prod dispatch via a Polly-free seam
Remediates RESILIENCE-DISPATCH-GAP (surfaced by the 07/C-1 analyzer): the Phase 6.1
CapabilityInvoker resilience pipeline (retry / breaker / bulkhead / telemetry) was
constructed only in tests — the production dispatch layer called driver-capability
methods directly, bypassing it entirely.

The filed plan (thread CapabilityInvoker directly into DriverInstanceActor) is
INFEASIBLE: Runtime is deliberately Polly-free (references Core.Abstractions, not
Core — the same boundary IDriverFactory documents). So this introduces a seam,
mirroring IDriverFactory exactly:

- Core.Abstractions: IDriverCapabilityInvoker (+ NullDriverCapabilityInvoker
  pass-through) and IDriverCapabilityInvokerFactory (+ null factory).
- Core: CapabilityInvoker now implements the interface; new
  DriverCapabilityInvokerFactory builds a per-instance invoker over the
  process-singleton pipeline builder + status tracker + tier resolver.
- Runtime: DriverInstanceActor takes an IDriverCapabilityInvoker (default =
  pass-through) and routes all 6 dispatch sites (write, alarm-ack, subscribe,
  unsubscribe, alarm-subscribe, discover) through it; per-host key resolved via
  IPerCallHostResolver for single-ref calls, driver-instance key for bulk calls.
  DriverHostActor builds + injects the real invoker per spawned driver.
- Host: DriverFactoryBootstrap registers the tracker, pipeline builder, and
  concrete factory (needs DriverFactoryRegistry.GetTier). Runtime SCE resolves the
  factory from DI like IDriverFactory; pass-through on nodes without it bound.
- Analyzer: IDriverCapabilityInvoker.ExecuteAsync/ExecuteWriteAsync are now
  recognized wrapper homes (interface-typed invoker calls must suppress OTOPCUA0001
  too). All 6 RESILIENCE-DISPATCH-GAP pragmas removed — the analyzer (an error in
  Runtime) is now the standing regression guard.

Also adds retry / breaker-open / breaker-close LOGGING to the pipeline builder —
the operator-facing observability surface (the Admin /hosts reader, Phase 6.1
Stream E.2/E.3, was never built, so the pipeline otherwise runs silently).

Scope notes:
- Tier-DEFAULT policy only: DriverInstanceSpec (the deploy artifact) does not carry
  the per-instance ResilienceConfig JSON, so overrides aren't applied yet (tracked
  follow-up: plumb ResilienceConfig through the composer/artifact).
- GenericDriverNodeManager's call is NOT wired: that class is test-only scaffolding
  (zero production references — the Server has its own address-space path); its
  pragma is re-annotated accordingly, not left as a "tracked gap".

Verification (unit + analyzer; live behavioral gate still pending — see FOLLOWUP-10):
- Negative control: unwrapping any site fails the Runtime build with OTOPCUA0001.
- New Runtime guard (recording invoker): write routes via ExecuteWriteAsync with the
  IPerCallHostResolver host; subscribe via ExecuteAsync — proves runtime routing.
- New analyzer test: the interface is a valid wrapper home.
- New pipeline-builder test: retry events are logged.
- Full solution builds clean (0 errors); Runtime.Tests 357, Core.Tests 238,
  Analyzers.Tests 32 all green; pass-through default keeps existing dispatch tests
  byte-for-byte unchanged.
2026-07-08 21:33:16 -04:00
Joseph Doherty 9cd1f8c4e7 docs(archreview): file follow-up records for open decisions #10 (RESILIENCE-DISPATCH-GAP) + #11 (Crit-1 SBR premise) 2026-07-08 20:06:15 -04:00
Joseph Doherty 14f0cae7d9 docs(archreview): mark guards 07/C-1, 07/S-1,S-2,S-4 + #9 failover done; record findings #10, #11 2026-07-08 18:27:16 -04:00
Joseph Doherty a25c9ed097 test(cluster): hard-kill failover integration test on the 2-node harness (arch-review 03/S1)
The load-bearing integration half of Critical 1. Adds TwoNodeClusterHarness.HardKillNodeAAsync
(canonical in-process crash sim: shut down node A's remoting transport via
IRemoteActorRefProvider.Transport.Shutdown() — associations drop, heartbeats/gossip stop, NO
graceful Cluster.Leave, so node B's failure detector marks A Unreachable) + a
WaitForNodeBSoleDriverLeaderAsync helper (waits for a STABLE takeover: B sole Up member + driver
role-leader). HardKillFailoverTests crashes the oldest node and asserts the survivor takes over —
the exact scenario the graceful StopNodeBAsync path (Cluster.Leave, no downing decision) cannot
exercise.

Live-verified in-process (~35s: acceptable-heartbeat-pause 10s + stable-after 15s + convergence).
Negative control confirmed the test BITES: forcing explicit NoDowning
(akka.cluster.downing-provider-class = "") makes it time out with the node stuck Up-but-Unreachable.

FINDING surfaced by the negative control: removing Critical 1's typed
ClusterOptions.SplitBrainResolver did NOT break failover, because Akka.Cluster.Hosting's
WithClustering applies SplitBrainResolverOption.Default when the option is null — enabling the SBR
downing provider that reads the akka.conf keep-oldest block (present on master BEFORE Critical 1).
So the cluster was NOT running NoDowning before Critical 1; the typed option reinforces the akka.conf
strategy rather than being the sole activator, and Critical 1's 'HOCON inert / NoDowning' premise is
inaccurate. The test guards the failover OUTCOME regardless of activation path. Comments corrected to
reflect this; flagged for review.
2026-07-08 18:24:49 -04:00
Joseph Doherty 10b898305f ci(v2): whole-solution unit leg + fail-on-skip gate + deflake CLI sleeps (arch-review 07/S-1,S-2,S-4)
S-1: replace the hand-maintained 5-project unit-tests matrix (which silently dropped
Client's 388 tests, Analyzers' 31, and every driver + most Core suite) with ONE
whole-solution leg — dotnet test ZB.MOM.WW.OtOpcUa.slnx --filter
"Category!=E2E&Category!=LiveIntegration". Self-maintaining: a new *.Tests project is
covered automatically, matching CLAUDE.md's own guidance.

S-2: emit trx + add scripts/ci/assert-not-all-skipped.sh, a fail-on-skip gate that
turns 'green CI == everything skipped' into a red build. Wired strict (MIN_EXECUTED=1)
on the unit leg; report-only (MIN_EXECUTED=0) on the fixtureless integration leg so
its skip tally is VISIBLE without a false red — with a documented follow-up to start
the one public-image fixture (opc-plc) as a service and flip it strict.

S-4 (paired): the newly-CI'd Client.CLI.Tests had fixed-sleep startup races
(await Task.Delay(100/150) before cancelling a background command) that would flake
under CI load. Added SubscribeInvoked / SubscribeAlarmsInvoked readiness signals
(TaskCompletionSource) to FakeOpcUaClientService and replaced the 11 sleeps across
AlarmsCommandTests / SubscribeCommandTests / EventHandlerLifecycleTests with a
deterministic await-the-signal (10s timeout guard).

Verified: workflow YAML parses; skip-gate proven locally on a real trx (executed=31
=> OK), a synthetic all-skipped trx (executed=0 => exit 1 with diagnostic),
report-only mode (never fails), multi-file sum, and missing-file (exit 2);
Client.CLI.Tests 104/104 green after the deflake. (CI job execution itself is
verifiable only on push — nothing pushed.)
2026-07-08 17:58:42 -04:00
Joseph Doherty f0082af5b9 build(analyzers): wire OTOPCUA0001 tree-wide + triage the surfaced hits (arch-review 07/C-1)
Inject the custom UnwrappedCapabilityCallAnalyzer as an OutputItemType=Analyzer
ProjectReference from Directory.Build.props (excluding the analyzer + its test
project) so OTOPCUA0001 runs on every src/ and tests/ compilation — it previously
enforced its CapabilityInvoker-wrapping rule against nothing but its own 31 unit
tests (the 'built-but-never-wired' failure mode).

Triage of the ~280 surfaced hits, three categories:

1. RESILIENCE-DISPATCH-GAP (7 sites, DriverInstanceActor x6 + GenericDriverNodeManager
   x1): a REAL, previously-untracked gap the analyzer caught on first wiring — the
   Phase 6.1 CapabilityInvoker resilience pipeline (retry/breaker/bulkhead/telemetry)
   is constructed ONLY in tests and was never wired into the production dispatch
   layer. Scoped per-site #pragma with a greppable RESILIENCE-DISPATCH-GAP marker
   explicitly noting these are tracked-but-not-intentional, pending the dispatch-wiring
   remediation (filed as a follow-up). Keeps the analyzer live everywhere else in
   those projects so a NEW unwrapped call still fails the build.

2. Driver-INTERNAL self-calls (3 sites, AbCipAlarmProjection x2 + S7Driver x1):
   a driver's own poll/ack path calling its own capability method. The invoker wraps
   the driver from the dispatch layer OUTWARD; a driver re-wrapping its own internal
   calls would double-wrap. Genuinely intentional — scoped #pragma with that rationale.

3. Wire-level test suites + manual-testing CLIs (12 projects): invoke drivers directly
   by design — the analyzer's own documented intentional case. Project-level NoWarn
   with a comment.

Verified: full solution build green, 0 OTOPCUA0001 hits; analyzer's 31 tests pass;
negative control — dropping one dispatch-gap pragma re-fires OTOPCUA0001 and fails
the Runtime build, proving the analyzer is genuinely live tree-wide, not disabled.
2026-07-08 17:51:19 -04:00
Joseph Doherty a203950848 docs(archreview): mark guard 03/U2 (deferred-sink forwarding test) done 2026-07-08 17:38:58 -04:00
Joseph Doherty a65c2ced19 test(commons): reflection-exhaustive deferred-sink forwarding guard (arch-review 03/U2)
Add a mechanical, reflection-driven guard that every method of every forwarding
interface DeferredAddressSpaceSink implements (IOpcUaAddressSpaceSink +
ISurgicalAddressSpaceSink) actually reaches the inner sink, plus the sibling
DeferredServiceLevelPublisher. A DispatchProxy recording inner auto-implements
future interface members with zero maintenance, so a newly-added sink capability
that the wrapper forgets to forward now fails a test instead of shipping inert on
every driver-role host (the F10b / PR#423 failure mode). A guard-of-the-guard
asserts the wrapper implements exactly the known forwarding-interface set, so a
brand-new capability *interface* trips the test too.

This is the prerequisite gate for 03/P1 (surgical remove methods) and 02/P
(batched WriteValues): both add sink capability members that must not ship inert.

Verified: 3 new tests green (Commons.Tests 57/57); negative control (stubbing
UpdateFolderDisplayName to a non-forwarding no-op) fails with the exact
member-named diagnostic, proving the guard bites.
2026-07-08 17:37:46 -04:00
Joseph Doherty 26c9d28b69 docs(archreview): mark Critical 4 (TwinCAT notification replay / STAB-2) done
All 4 Criticals complete. crit4 branch af318fb4: replayable-intent store +
reconnect-replay + probe-recycle, 4 unit guards + 174/174 (unit-only, no TC3 fixture).
Next: systemic guards (task 6 forwarding test first) + Critical-1 failover harness (task 9).
2026-07-08 17:21:13 -04:00
Joseph Doherty af318fb442 fix(twincat): replay native ADS notifications after reconnect (archreview STAB-2 / Critical 4)
Native ADS notifications (the default subscribe mode) were stored as opaque handles
with no record of the symbol/type/interval/handler needed to replay. On a client swap
(EnsureConnectedAsync building a fresh client after a drop) the notifications were
silently orphaned — no Bad status, no error, pushes just stopped until redeploy.
Compounding: the IsConnected fast-path keys on AMS-port state, not wire liveness, and
a probe failure only transitioned state without recycling the dead client.

Fix:
- Store REPLAYABLE INTENT: NativeRegistration (symbol/type/bit/interval/onChange +
  swappable live handle) hung off DeviceState.NativeRegistrations, populated by
  SubscribeAsync via RegisterNotificationAsync (under ConnectGate).
- Split EnsureConnectedAsync into a gate wrapper + EnsureConnectedUnderGateAsync core;
  when the core installs a NEW client it replays every stored intent onto it and swaps
  the live handle (disposing the dead one). Register + replay both run under ConnectGate
  so they can't race.
- Probe loop: on a wire-probe failure (false or throw) RecycleClientAsync disposes+nulls
  the client so the next tick rebuilds + replays — closes the fast-path-keys-on-port-state
  compounding bug.

No TwinCAT docker fixture exists (integration needs a real TC3 XAR), so the fake-client
unit tests are the authoritative coverage:
- 4 new guards in TwinCATReconnectReplayTests (replay-onto-fresh-client + push reaches
  OnDataChange + old handle disposed; replay-all-tags; unsubscribe-after-reconnect stops
  replaying; probe-failure recycles+rebuilds).
- Full TwinCAT unit suite 174/174 green; full solution builds 0 errors.
2026-07-08 17:20:10 -04:00
Joseph Doherty fcd36bd53d docs(archreview): mark Critical 3 (S7 reconnect / STAB-1) done + live-verified
STATUS.md + 05 plan banner: crit3 branch 25c0c6f6, IS7Plc seam + lazy reconnect,
4 unit guards + full S7 suite, LIVE snap7 container-bounce proof (recovers with no
redeploy). Next: Critical 4 (TwinCAT ADS re-registration, unit-only — no TC3 fixture).
2026-07-08 17:05:51 -04:00
Joseph Doherty 25c0c6f68f fix(s7): add lazy reconnect path (archreview STAB-1 / Critical 3)
The S7 Plc was opened once in InitializeAsync and never re-opened: a transient
PLC reboot / network blip permanently killed the driver until redeploy. Introduce
an IS7Plc / IS7PlcFactory seam (mirrors TwinCAT's ITwinCATClientFactory) and a lazy
EnsureConnectedAsync that disposes a dead handle and re-opens a fresh connection on
the next data call. Reads/writes mark the handle dead on a connection-fatal fault
(socket drop / ErrorCode.ConnectionError) but NOT on a data-address error; the probe
loop routes through EnsureConnectedAsync as a backstop.

The seam also closes the S7 TEST-1 gap — the reconnect state machine is now unit-
testable without a live PLC (S7.Net.Plc is sealed with no in-process fake).

Verification:
- 4 deterministic unit guards (fatal→reopen, data-error→no-reopen, raw socket→reopen,
  reopen-fail→degrade-then-recover); full S7 unit suite 234/234 green.
- LIVE end-to-end proof against real python-snap7 (S7_1500ReconnectTests, double-gated
  on sim reachability + S7_RECONNECT_BOUNCE_CMD): bounced the container, driver observed
  the outage and resumed Good reads with NO redeploy. Baseline smoke 3/3 still green.
2026-07-08 17:04:46 -04:00
Joseph Doherty e31d29c04c docs(archreview): add STATUS.md handoff + per-plan status banners
Track remediation progress: 2 of 4 Criticals done (03/S1 split-brain resolver;
02/U2+U3 VT timeout+ALC) + both CLAUDE.md doc-drifts fixed. STATUS.md is the
single source of truth (branch topology, completed items, task list, next-up,
resume facts). Each plan carries a status banner.
2026-07-08 16:25:39 -04:00
Joseph Doherty 7fd44f0fa7 fix(vtags): real script timeout + ALC-safe compile cache (arch-review 02/U2,U3)
U2 (Critical): RoslynVirtualTagEvaluator ran scripts with a CancellationToken
that a synchronous Roslyn script can never observe, so one CPU-bound/infinite-
loop virtual-tag script hung the owning VirtualTagActor forever (no timeout, no
recovery). Route evaluation through the existing TimedScriptEvaluator (Task.Run
+ WaitAsync) so the wall-clock budget is real; a runaway script now returns Bad
within the timeout instead of wedging the actor.

U3 (High): the live path cached evaluators in a raw ConcurrentDictionary never
cleared on republish, leaking a collectible AssemblyLoadContext per edited
script forever. Swap in the purpose-built CompiledScriptCache (Lazy single-
compile, dispose-on-Clear) and add an apply-boundary clear: a new narrow
IScriptCacheOwner seam the VirtualTagHostActor calls on each ApplyVirtualTags
generation (mirrors ScriptedAlarmEngine's per-generation clear).

Tests: infinite-loop-fails-within-timeout (bounded by WaitAsync so a regression
fails instead of hanging), happy-path-after-wrapping, ClearCompiledScripts-
empties-cache, and a host-actor wiring guard asserting each apply generation
calls ClearCompiledScripts (theme #1 production-caller assertion). Host.Integration
VT suite 18/18, Runtime VT suite 12/12.
2026-07-08 16:21:47 -04:00
Joseph Doherty 9fadead6a6 docs(archreview): remediation plans + fix flagged doc drift
Add the 7 per-domain design+implementation plans (archreview/plans/) with
an index, produced from the 2026-07-08 architecture review.

Fix two confirmed doc drifts the review flagged (theme #5):
- CLAUDE.md KNOWN LIMITATION 2: the continuous-historization historized-ref
  feed IS wired (AddressSpaceApplier.FeedHistorizedRefs -> UpdateHistorizedRefs
  -> recorder); rewrite to reflect that value-capture is code-complete and only
  the live end-to-end + restart-convergence verification remains.
- CLAUDE.md ScriptAnalysis gating: endpoints use Roles=Administrator,Designer
  via RequireAuthorization, not the FleetAdmin policy.
2026-07-08 16:14:37 -04:00
Joseph Doherty a81dea102c fix(cluster): activate split-brain resolver (arch-review 03/S1)
The split-brain-resolver HOCON block in akka.conf was inert — nothing
registered a downing provider, so the cluster ran Akka's default NoDowning:
a hard-crashed node was never downed, cluster singletons + the driver
role-leader never failed over, and a partition left both redundancy sides
at ServiceLevel 240 indefinitely.

Activate the resolver via the typed ClusterOptions.SplitBrainResolver
(KeepOldestOption { DownIfAlone = true }) in a new testable
BuildClusterOptions helper. keep-oldest + down-if-alone is correct for a
2-node warm-redundancy pair. HOCON keeps stable-after (typed option can't
express it) + a cross-reference comment. Deterministic unit guard added;
end-to-end verified the resolved ActorSystem config carries the provider
and stable-after survives the HOCON/Hosting merge. Redundancy.md corrected.

Hard-kill failover integration test tracked as a follow-up (effort-M harness).
2026-07-08 16:14:26 -04:00
Joseph Doherty 9cad9ed0fc docs(xmldoc): fill missing XML docs + strip tracking-ID comments across src
v2-ci / build (push) Failing after 41s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
Adds <summary>/<param>/<returns>/<inheritdoc> where missing and removes
project bookkeeping IDs (task/tracking refs) from shipped code comments,
so the docs read cleanly and CommentChecker is quiet except for known
false positives (PLC/protocol terms, event/IEqualityComparer inheritdoc).
Doc/comment-only; no logic changed; solution builds clean.
2026-07-07 12:38:39 -04:00
Joseph Doherty 384dbd7d36 docs(claude): refresh CLAUDE.md + record historian-gateway integration/dev-rig state
v2-ci / build (push) Failing after 42s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
CLAUDE.md: add scadaproj sister-project index, fix mxaccessgw path to
~/Desktop/MxAccessGateway, note the temporary CVE-2025-6965 NuGetAuditSuppress,
and correct the driver-typed tag-editor list (OpcUaClient is now mapped; only
Galaxy falls back). Also captures docker-dev data-plane GroupToRole + ServerHistorian
live-verify config and the HistorianGateway integration/backlog notes.
2026-07-07 11:46:02 -04:00
Joseph Doherty 4dd47ce707 Merge branch 'fix/historian-gateway-integration'
v2-ci / build (push) Failing after 50s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
Wire the dormant GatewayTagProvisioner (PR #423 shipped it un-registered), make
provisioning observable, document it, and anchor the Serilog log path. Resolves
HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md (Issues 1-4 + the Serilog observation).
2026-06-27 23:26:19 -04:00
Joseph Doherty 257214f775 fix(historian-gateway): wire dormant GatewayTagProvisioner + provisioning observability/docs
PR #423 shipped GatewayTagProvisioner + unit tests but never registered it in
DI nor passed it into the AddressSpaceApplier, so deploying historized tags used
the no-op NullHistorianProvisioning and never called the gateway's EnsureTags
(confirmed live on wonder-app-vd03: zero EnsureTags calls on a historized deploy).
Addresses HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md.

Issue 1 (wire provisioner):
- Runtime: AddHistorianProvisioning extension (gated on ServerHistorian:Enabled,
  mirrors AddServerHistorian) + NullHistorianProvisioning TryAdd default in
  AddOtOpcUaRuntime; WithOtOpcUaRuntimeActors resolves IHistorianProvisioning and
  passes it into the applier.
- Gateway driver: GatewayHistorian.CreateProvisioner factory (mirrors CreateDataSource).
- Host: Program.cs calls AddHistorianProvisioning after AddServerHistorian.
- Tests: AddHistorianProvisioningTests (config-gated registration + the
  register->resolve->applier->EnsureTags chain).

Issue 2 (observability): AddressSpaceApplier logs the provisioning tally on every
successful dispatch (was gated behind Failed/Skipped > 0), including dispatched=N
so a dispatched=N/requested=0 line flags the dormant no-op. +2 tests.

Issue 3 (30s HistoryRead on unprovisioned tags): root coupling fixed by Issue 1;
documented the CallTimeout knob + coupling. Default left at 30s pending the
multi-data-point investigation the issue requests (lowering risks truncating
legitimate large reads).

Issue 4 (docs): docs/Historian.md gains a "Tag auto-provisioning (EnsureTags)"
section and CLAUDE.md a wiring/gating note (both stress ServerHistorian:Enabled).
Sibling scadaproj/CLAUDE.md carries no false claim -> unchanged.

Pre-existing Serilog observation: anchor CWD to AppContext.BaseDirectory before
AddZbSerilog so the relative file sink stops landing in C:\Windows\System32 under
the Windows-service CWD.

Builds 0-error; Runtime.Tests 355, OpcUaServer.Tests 329, Gateway.Tests 99 (+4
live-skipped) all green.
2026-06-27 23:24:29 -04:00
dohertj2 245316d80d Merge pull request 'test(live): assert FU-1 alarm SendEvent->ReadEvents round-trip (gateway C4 fixed)' (#425) from fix/fu1-alarm-source-roundtrip into master
v2-ci / build (push) Failing after 40s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
2026-06-27 16:51:55 -04:00
Joseph Doherty 7a94f7adf0 test(historian-gateway,live): assert FU-1 alarm SendEvent->ReadEvents round-trip (C4 fixed)
v2-ci / build (pull_request) Failing after 41s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (pull_request) Has been skipped
The gateway now populates Runtime.dbo.Events.Source_Object from the event
SourceName (gateway C4 fix: HistorianProtoMapper threads SourceName into the
`source_object` event property). So an ad-hoc alarm SendEvent is now source-
filterable on readback, and Alarm_SendEvent_then_ReadEvents asserts the round-
trip instead of skipping with the old "ad-hoc sends land without Source_Object"
reason. Poll window widened to 60s for the live event-view flush latency.

Live-proven at the gateway level (HistorianGateway EventSourceObjectProbeTests,
2026-06-27); this asserts the same round-trip through the OtOpcUa alarm writer +
data source adapters.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-27 16:48:16 -04:00
dohertj2 53798bbf40 Merge pull request 'feat: HistorianGateway as the OtOpcUa historian backend (read/write/alarms + continuous historization); retire Wonderware' (#423) from feat/historian-gateway-backend into master
v2-ci / build (push) Failing after 41s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
2026-06-27 11:09:02 -04:00