Commit Graph

5103 Commits

Author SHA1 Message Date
Joseph Doherty 169149f1eb Improve admin field names and generation-safe dialog lifecycle 2026-11-13 21:15:58 -05:00
Joseph Doherty 06b1743051 Associate protocol tag fields batch 3 2026-11-13 21:05:33 -05:00
Joseph Doherty 1746b04588 Associate protocol tag fields batch 2 2026-11-13 21:05:33 -05:00
Joseph Doherty 8cc053ceb7 fix(admin-ui): pass contextual names to native Monaco editor 2026-11-13 21:04:30 -05:00
Joseph Doherty 2415db8ca4 Associate AB and Focas tag editor fields 2026-11-13 20:53:58 -05:00
Joseph Doherty 7e57074653 fix(admin-ui): associate native device endpoint fields 2026-11-13 20:49:26 -05:00
Joseph Doherty 7b7fdf0eb7 Associate S7 SQL and TwinCAT driver fields and diagnostics 2026-11-13 20:42:31 -05:00
Joseph Doherty eed546d96d Associate MQTT MTConnect and OPC fields with readable feedback 2026-11-13 20:33:37 -05:00
Joseph Doherty c70bc44cd9 Associate Galaxy Heidenhain and Modbus driver fields 2026-11-13 20:23:31 -05:00
Joseph Doherty 9be717c0d7 Associate fields in AB and Focas driver editors 2026-11-13 20:19:43 -05:00
Joseph Doherty a47ffb9151 test(admin): replace bUnit preview with stable 2.7.2 2026-11-13 15:23:32 -05:00
Joseph Doherty 79371402e4 Merge branches 'worktree-agent-a41f08fe289ebe7a8', 'worktree-agent-a21d19bd14da138e7' and 'worktree-agent-a8d85b6d5ef424006'
v2-ci / unit-tests (push) Failing after 1h32m12s
2026-11-13 12:43:14 -05:00
Joseph Doherty 73132b7d78 fix(opcuaclient): #910 name the per-endpoint connect budget; widen it in the #816 host-name tests
Cause: the client-side PerEndpointConnectTimeout (3 s), not opc-plc. The
validate:null case is the first test the assembly runs, so it pays the
cold-process cost of the first Basic256Sha256 connect: loading the fresh
throwaway certificate, validating the server's, and the asymmetric handshake.
In the G2 full-solution run that local work used up the budget. The opc-plc
log at 13:13:50-53Z shows the discovery channel closing at 50.709, then nothing
from the client for 3 s, then OpenSecureChannel answered at 53.775 and the
client closing the channel 48 ms later. The SDK reports a wait cancelled by
that budget as BadRequestInterrupted. opc-plc auto-accepted the certificate
(--aa, "Trusting certificate ... command line option") with no reject and no
store round-trip.

- Driver: a per-endpoint budget expiry now surfaces as TimeoutException naming
  PerEndpointConnectTimeout instead of BadRequestInterrupted or a bare
  TaskCanceledException. There is no behaviour change. Unit test added
  against a silent TCP listener.
- Test: the ServerHostNameValidationTests secure options use a 15 s budget,
  the same as the browser integration tests' 10 s. Connect latency is not
  what these tests check.
- docs/drivers/OpcUaClient.md: documents what the budget covers.
2026-11-13 12:42:45 -05:00
Joseph Doherty fc08892eb3 fix(modbus-fixture): run pymodbus sim without asyncio debug mode; FC03 healthcheck + watchdog (#909)
Root cause: pymodbus 3.13's `pymodbus.simulator` console script hard-codes
asyncio.run(run_main(), debug=True). Debug mode captures a traceback per
scheduled callback and logs callbacks slower than 100 ms (the "took 0.146
seconds" lines in #909). Side by side on the shared docker host (closed-loop
FC03 clients, 2 s deadline): 100 clients got ~120-170 req/s, p99 1.6-1.9 s
with client timeouts in debug mode vs ~2400 req/s, p99 ~135 ms without it.
The shared sim also carries the docker-dev rig's 49 Modbus drivers on each of
central-1/central-2 (98 sockets, ~220 req/s at idle), which is above the
debug-mode capacity, so a test run on top was answered late ("Cancel send,
because not connected!") and timed out.

- run_simulator.py: same CLI as pymodbus.simulator, debug=False, plus a watchdog
  thread that exits after 3 failed FC03 probes so restart: unless-stopped
  replaces the container (docker has no restart-on-unhealthy).
- fc3_probe.py: stdlib FC03 read with a deadline (MBAP or RTU framing); the
  compose healthcheck.
- docker-compose.yml: shared anchor for every pymodbus service: new entrypoint,
  `project: lmxopcua` label, restart: unless-stopped, healthcheck, bounded
  json-file logs.
- README: why, the numbers, and how to diagnose load.
2026-11-13 11:03:08 -05:00
Joseph Doherty f1da809f3c test(scripted-alarms): wait for the sibling run to finish before asserting InFlightCount (#911)
ScriptRunGuard counts a run on StartedRunCount and InFlightCount together the
moment it is admitted, but releases InFlightCount only in the run's finally.
The runaway test waited on StartedRunCount >= 6 and then asserted
InFlightCount == 1, so the healthy sibling's 4th-push run - admitted but still
executing - was counted alongside the runaway orphan (2). Test race, not an
engine defect: widening only the healthy run reproduced it 5/5.

- wait until StartedRunCount == 6 AND InFlightCount == 1
- healthy predicate spins ~100 ms so the window is open on every run
- positive controls: the final push must be refused once more (the in-flight
  run is the orphan) and the quality-once absence check runs only after the
  healthy Activated event from the later pass has been delivered

Under 28 busy-loops on 14 cores: before 2/60 failed, after 0/60.
2026-11-13 10:46:19 -05:00
Joseph Doherty e299dc090a chore(ga): retire G1 - delete the stale Phase 6 compliance scripts; v2 GA exit criteria all met/waived/retired
v2-ci / unit-tests (push) Has been cancelled
The phase-6-*.ps1 scripts checked v2 Phase 6 code that later commits deliberately
deleted (76310b88 v1 Server/Admin, #522, #524, LiteDB cache, v3 migration squash):
56/56 failures were retired files, so they could not pass. Owner retired G1; G2
plus the per-area suites cover the invariants. phase-7-compliance.ps1 is untouched.
2026-11-13 10:33:43 -05:00
Joseph Doherty f93a1179dd docs(ga): G7 PASS - FOCAS live CNC on vd03 (Makino 31i-B), axis tag added via owner-approved deploy e9fbe63e
v2-ci / unit-tests (push) Waiting to run
2026-11-13 10:32:09 -05:00
Joseph Doherty 1862fab671 docs(deploy): QA vq03/vq04 switched to OpcUa:Redundancy:Mode=Warm; #138 live gate PASS (#907, #717)
v2-ci / unit-tests (push) Failing after 14m56s
Two service-env entries per node (Mode=Warm, PeerApplicationUris__0), pair stopped and
started together. Gates: 250/90, RedundancySupport Warm, standby BadOutOfService,
LocalDb sync, canary Good on the Primary; standby Write could only show BadNotWritable
(QA tags are read-only). ScadaBridge WarmCanary moved off the Standby by ServiceLevel in
0.1 s (connect-time preference); graceful Primary stop still took the 66 s connectivity
path (ScadaBridge#141). End state: Warm, vq04 Primary.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 09:00:35 -05:00
Joseph Doherty 22de358f74 docs(ga): G2 PASS (one defect fixed in f416b883; #909/#910/#911 environmental); G1 + G7 evidence recorded pending owner decision
v2-ci / unit-tests (push) Has been cancelled
2026-11-13 08:59:07 -05:00
Joseph Doherty a157b27628 fix(install): swap waits for the host process to exit before moving the folder (#908)
v2-ci / unit-tests (push) Has been cancelled
On QA (76e73e7d deploy) Invoke-OtOpcUaInstallSwap.ps1's first Move-Item of
app\ failed "file in use" right after the service reported Stopped: SCM
reports Stopped while the host is still exiting (e.g. a log file under
InstallRoot still open).

- After the stop, wait (bounded, -ProcessExitTimeoutSeconds, default 60)
  until the service's host process (PID + start time captured before the
  stop) and every process imaged under InstallRoot have exited; otherwise
  throw naming them, with nothing moved.
- The renames use [IO.Directory]::Move instead of Move-Item. On windev,
  PowerShell 7's Move-Item falls back to an item-by-item move when the
  rename fails because a file under the folder is open, and leaves a partial
  PreviousInstall tree behind (a retry then nests into it).
- The first rename is retried for at most -MoveRetrySeconds (default 10)
  while it fails because something under InstallRoot is open (a backstop
  for a holder whose image is outside InstallRoot).
- -Verify/-Reapply/-Scope semantics unchanged.

Tests: dry-run harness gains fake-process-list tests for the wait and the
retry plus AST order checks (145 passed); the Windows harness gains three
real cases (holder past the budget, holder released 2 s into the wait,
holder outside InstallRoot) - 77/0 on pwsh 7 and Windows PowerShell 5.1 on
windev; with the wait removed the first two fail.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 08:46:27 -05:00
Joseph Doherty f416b88324 fix(717): use RoleParser.Driver in the Warm verdict (RoleAuthorityGuard)
v2-ci / unit-tests (push) Has been cancelled
c569f806 passed the bare "driver" literal to WarmRoleAuthority.OldestUpMember
at OpcUaPublishActor.cs:1875-1876, which RoleAuthorityGuardTests forbids outside
RoleParser. Same value; found by the #615 G2 full-suite run. Cluster.Tests
255/255, Runtime Warm tests 23/23.
2026-11-13 08:33:15 -05:00
Joseph Doherty 16ddddb2eb docs(615): G5 signed off by the Fleet Admin; every #615 gate complete, #615 closed
v2-ci / unit-tests (push) Has been cancelled
2026-11-13 08:09:37 -05:00
Joseph Doherty 5baf05f634 docs(deploy): vd03 + QA deployed c9640f5c -> 76e73e7d (#905 #906 #900 #717, LocalDb 0.3.8)
v2-ci / unit-tests (push) Has been cancelled
Executed 2026-11-13 12:27-12:48Z; target moved from the pack's 47f0b98c to the tip 76e73e7d
(HG Client 1.1.0, behaviour unchanged; note added to the pack).

- vd03: ~14 s downtime (was ~101 s; #905 window 5.7 s), SL 250, FOCAS 21/22, Mode Unset,
  HG Probe 494 -> 536 with no write RPCs (added gate for #900).
- QA: pair outage 36.5 s (was ~91 s), 250/240, both failover legs, no failback, sync
  connected, canary Good. #906: vq04 (sync server) shutdown -> Leaving 30.0 s on c9640f5c,
  21 ms on 76e73e7d; peer at 250 in 0.47 s (leg 1: 0.36 s).
- Deviation: the first Move-Item of app\ failed "file in use" right after the stop on both
  QA nodes; a retry with the service stopped succeeded (lmxopcua#908).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 07:51:01 -05:00
Joseph Doherty c9bc358dc9 feat(cli): print array values; docs: #615 G9 PASS with otopcua-cli, G8 waived
v2-ci / unit-tests (push) Has been cancelled
G9 ran on an isolated 2-node pair (otopcua-g9, 003c7f89, Mode unset as
deployed) with otopcua-cli as the owner-approved client: A1-A3, A5, A7, A8
(graceful, ~11 s gap), B2 (docker kill, failover <1 s, ~2 s gap, 0 Bad,
survivor 250 at +44 s) and B3 (no failback) pass; A4/A5b unmet under Unset,
met under Warm (#717). G8 (CTT) waived by the owner for GA; G5 is the only
open exit criterion.

The run found the CLI printing every array as System.String[] (ServerArray
unreadable); ValueFormatter now renders [a, b] / hex for read, subscribe and
historyread, with tests. Runbook A4/A5b text updated for the #717 modes.
Evidence: docs/plans/evidence/2026-11-13-615-g9/RESULTS.md.
2026-11-13 07:46:56 -05:00
Joseph Doherty a59dbd4f07 docs(heidenhain): driver accepted, #829/#853 closed
The ignitionoee CR capture (6d2aec40: 7 h read-only sample of 34 production
controls on the Java LSV2 driver) and the 2026-11-11 iTNC 530 run did not
change the driver's code: ProgramState 0 = cutting (11,865 of 11,869 motion
pairs), 7 = idle; TNC 426/430 BadNotSupported; overrides BadUserAccessDenied
under INSPECT; capabilities Unknown until first poll; alarm classes 1/7/8 seen,
class 5 not. Owner accepted the driver on that basis; remaining limits stay
documented in Heidenhain.md §Hardware-gated.
2026-11-13 07:46:56 -05:00
Joseph Doherty 76e73e7dbf chore(historian): HistorianGateway Client/Contracts 1.1.0, typed error reason (#900)
v2-ci / unit-tests (push) Has been cancelled
Bump ZB.MOM.WW.HistorianGateway.Client + Contracts 1.0.0 -> 1.1.0 (historiangw#135,
release 6365f34). GatewayErrorReasons now reads Client 1.1.0's typed
HistorianGatewayException.StatusCode / ErrorReason / ErrorDomain (constants from
HistorianGatewayErrorReasons) instead of parsing grpc-status-details-bin itself; the
direct Grpc.StatusProto pin and reference are dropped (the client brings it). The
published client maps every RpcException to a HistorianGatewayException, so the
trailer fallback is not needed. Behaviour unchanged: FailedPrecondition +
AGGREGATE_NOT_SUPPORTED_FOR_TAG (domain historian_gateway.v1) ->
BadAggregateNotSupported, everything else keeps its handling.

Adds GatewayErrorReasonsTests (typed-property-only exceptions, so they fail against
the old trailer implementation); wire tests and docs/Historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 07:17:10 -05:00
Joseph Doherty 97d513f7dc docs(deploy): next vd03 + QA deploy pack c9640f5c -> 47f0b98c (#905 #906 #717 LocalDb 0.3.8), read-only preflight
v2-ci / unit-tests (push) Has been cancelled
Range, settings (no migration; OpcUa:Redundancy:Mode unset everywhere), read-only
ConfigDb/SMB/OPC UA preflight on both tiers, #905 scoped swap with -NoStart on both
tiers, #906 failover timing gate (stop the sync-server node as Primary), Unset
metadata gate, rollback with the vd03 #903 drift note. Not executed.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 07:02:20 -05:00
Joseph Doherty 47f0b98c6e chore(deps): pin ZB.MOM.WW.LocalDb 0.3.8 (scadaproj#57)
v2-ci / unit-tests (push) Has been cancelled
LocalDb 0.3.8 ends open sync streams on ApplicationStopping and refuses
new ones Unavailable while stopping (lmxopcua#906, library side). The
host-side ShutdownStreamCancellationInterceptor stays: it also covers
the mesh telemetry/command streams.

GracefulStopWithOpenSyncStreamTests' positive control started passing
the stop in ~950 ms under 0.3.8, because the library alone now ends
the stream. The control registers LocalDbSyncService without the host
lifetime (the 0.3.7 behaviour), so it still reproduces the 8 s drain.

Release build 0 warnings. Host.Tests 288/288, Host.IntegrationTests
350 passed / 2 skipped, Runtime.Tests 1063 passed / 3 skipped,
Core.AlarmHistorian.Tests 53/53.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 06:55:27 -05:00
Joseph Doherty 003c7f8954 docs(deploy): #780 QA scripted-alarm drill passed (kill x2, quarantine, siblings Good, 250/240); config back to rev 82dd0598
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 06:54:09 -05:00
Joseph Doherty 36eb6d558d docs(717): Mode=Unset re-check after c569f806 (no change); a leaving node reports 0
v2-ci / unit-tests (push) Has been cancelled
T13b: the isolated otopcua-t13b pair with OpcUa:Redundancy:Mode absent, on origin/master f826af8e
(it contains c569f806) and A/B on c569f806^ (687256d6). Every check passes on both builds:
- levels 250/240;
- Good data on both nodes;
- writes, Acknowledge, AddComment and alarm delivery on the Secondary;
- graceful stop: the survivor reaches 250 259 ms after Leaving (pre-fix 270 ms);
- crash: 250 at +43.6 s (pre-fix +42.8 s), with no demotion at auto-down;
- rejoin at 240 with no failback.
The role-decision changes are gated on IsWarm and do not run under Unset.

Plan L5, the design's section 4.3 and the Warm table in Redundancy.md now say a Leaving/Exiting node
reports ServiceLevel 0, not 90 (ComputeWarm returns 0 for any member that is not Up/Joining; T13 note A).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 06:44:46 -05:00
Joseph Doherty f826af8e8a docs(717): T13 live gate evidence on an isolated Warm pair
v2-ci / unit-tests (push) Waiting to run
Every applicable T13 leg passes on 0c19f26c + c569f806. The leg list covers:
- metadata and ServiceLevel bands;
- standby projection;
- alarm suppression with Standby ConditionRefresh and the promotion refresh;
- the three D4 demotion timings, run live;
- standby write and Part 9 refusal (BadInvalidState);
- graceful and crash failover, with no failback;
- subscription transfer;
- the Mode=Unset regression.

The two crash-path defects the gate found are fixed in c569f806. Evidence, rig files and the
raw-client harness are in docs/plans/evidence/2026-11-13-717-t13/.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:39:17 -05:00
Joseph Doherty c569f80694 fix(717): Warm crash survivor stays Active through auto-down; verdict reads its own events
Two defects on the crash-failover path, found by the T13 live gate on an isolated docker pair:

1. When auto-down removed the unreachable Primary (~15 s after the survivor promoted), the
   snapshot's Primary left the unreachable set, so (3') failed and the survivor demoted to
   Standby (BadOutOfService, ServiceLevel 50) for ~15 s, until its own singleton published
   after the hand-over retries. A Primary that is Down, or that this node saw removed, now
   counts as gone (WarmRoleAuthority.SnapshotPrimaryGone). A member that is merely
   Leaving/Exiting, or that this node never saw, is not.

2. The verdict read Cluster.State / SelfMember (Akka's ClusterReadView, a separate subscriber
   to the same events), which can lag the event being handled. Live, the survivor stayed
   Standby 4.3 s after its own UnreachableMember event (promotion waited for the next health
   tick). The Warm verdict now uses a membership view built from this actor's own
   member/reachability subscription, falling back to the read view only before the first event.

Tests: WarmRoleAuthorityTests.A_snapshot_Primary_that_auto_down_removed_is_still_gone_and_the_survivor_stays_Active,
OpcUaPublishActorWarmTests.The_crash_survivor_promotes_on_its_own_unreachable_event_and_stays_Active_through_the_removal
(synthetic events the read view never sees). Mutations: read-view-only verdict and the removed
arm switched off each fail the actor test. Runtime.Tests 1058 passed, 3 skipped.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:36:58 -05:00
Joseph Doherty 687256d68e test(717): deferred Warm coverage - 2-node handover, DataEncoding, deleted/reused items, clone matrix, ACL stand-down, per-counter metrics
v2-ci / unit-tests (push) Has been cancelled
- T3 WarmSingletonHandoverTests: real 2-node cluster, singleton under ClusterSingletonManager on both nodes; the old
  incarnation's held-back publish (older revision, higher sequence) reaches a real Warm OpcUaPublishActor after the
  handover and never demotes the new Primary. Pin: a removed node cannot rejoin from its old ActorSystem; a new one
  rejoins younger (higher up-number).
- T6 DataEncoding: Binary/XML encoded reads of a structured tag projected on the Standby; scalar encoding error kept;
  a DefaultXml item is projected, then promoted through its own encoding (TypeId i=888).
- T7 item deleted between queue and publish; ClientHandle reused by i=2267 on a payload recycled through the SDK 2.0.1
  pool (precondition asserted); WarmStandbyCloneMatrixTests over DataChange+DiagnosticInfos/EventNotificationList/
  StatusChange, clone/cached immutability, encode round-trip, real-pool stale marker.
- T8 ACL-stood-down item receives nothing from demotion or promotion; restored grant reaches it again.
- T10 per-counter MeterListener tests (by instrument instance): warm_transition, warm_role gauge, snapshot_rejected
  (publish actor both reasons + DriverHostActor's ordering gate, first actor test of it), projected_notifications,
  suppressed_alarm_events. Process-wide state tests run in non-parallel collections.
- Every test mutation-checked (26 mutations killed); no product bug found. Plan status updated.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:31:11 -05:00
Joseph Doherty 0c19f26c81 fix(install): cut the swap's stopped window to the swapped-in folder (#905)
v2-ci / unit-tests (push) Has been cancelled
Root cause: the protected-DACL coverage scan, not the ACL writes. Both -Reapply and
-Verify walked every denied folder (all of E:\ApiInstall on vd03, nested roots again)
with Get-ChildItem -Recurse + Get-Acl per item + a pipeline per item for -Skip, about
3.4 ms an item on PS 5.1, all while the service was stopped. An unchanged deny is read,
not rewritten; the re-applied install-folder deny propagates over ~1,400 files in 0.5 s.

- Find-OtOpcUaDenyCoverageGap: .NET enumeration, -Skip folders pruned, DACL-only
  GetAccessControl(Access) (about 0.15 ms an item); reparse points not followed.
- Get-OtOpcUaCoverageWalkRoots: each denied folder walked once (outermost only).
- Install-ScriptWorkerIsolation.ps1 -Scope All|InstallRoot|Outside (with -Reapply,
  -Verify, -PlanOnly). InstallRoot = the folder ACEs a swap drops; Outside = the rest,
  including the firewall rule (its query costs 3-5 s; its exe path does not change).
  The two partition the plan. Prints the scan time.
- Invoke-OtOpcUaInstallSwap.ps1: -Reapply/-Verify -Scope Outside before the stop (a
  failure stops before anything moves), -Scope InstallRoot inside the window; preflight
  checks the candidate carries the worker exe; prints the stopped window and its parts.
  -NoStart, -NoCarry and the pre-#901 refusal unchanged.

windev, vd03-like tree (17,851 items): stopped window 153.1 s -> 6.0 s (PS 5.1),
101.6 s -> 6.2 s (PS 7); the Outside part takes ~23 s with the old build serving.
Dry-run harness 101 -> 123 passed; Windows test 72 passed on PS 5.1 and PS 7.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:51:42 -05:00
Joseph Doherty f817c77e02 test(717): deferred-event boundary, ACL-filtered promotion refresh, native-ack refusal, effective-URI XML load; docs + T13 overlay
v2-ci / unit-tests (push) Has been cancelled
- Wire: an event reported while Active but deferred (#838) is delivered after demotion; the promotion refresh
  withholds an ACL-denied condition (positive control alongside); ConditionRefresh2 and model-change events on the
  Standby; a native Acknowledge is refused and never routed; Warm validates and advertises the EFFECTIVE
  ApplicationUri from an ApplicationConfigPath XML load.
- docs/Redundancy.md: Warm section, the Unset/None i=11314 behaviour (orphan readable by NodeId under Unset,
  deleted under None), instruments; OpcUaServer.md, Configuration.md, security.md, cluster-topology rule.
- docker-dev/docker-compose.warm.yml for the T13 gate (central pair, Mode=Warm).
- Plan: implementation status and the test gaps left for T13/follow-up.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:39:50 -05:00
Joseph Doherty 7829b937d1 feat(717): Warm non-transparent redundancy, default-off (T1-T10)
OpcUa:Redundancy:Mode (Unset default = today's server exactly; None; Warm; Hot and unknown values fail start).
- T1 options + validator + effective-URI check + boot banner; Warm needs the driver role.
- T2 metadata in OnNodeManagerStartedAsync: Warm -> NonTransparentRedundancyType, read-only
  ServerUriArray [self, peers], RedundancySupport=Warm, boot ServiceLevel 1; None -> RedundancySupport=None
  and the SDK's orphan i=11314 deleted; Unset untouched.
- T3 RedundancyStateChanged.Revision (up-number, incarnation tick, sequence); subscribers drop older
  revisions under Warm (OpcUaPublishActor, DriverHostActor); unrevisioned never makes a node Active.
- T4 WarmRoleAuthority + ComputeWarm bands (Standby 90/50, Active 250/200/100), PromoteWhilePrimaryUnreachable.
- T5 IStandbyProjection + DeferredStandbyProjection; OpcUaPublishActor runs promotion/demotion in the spec
  4.7 order (Active level last and retried; Standby level first, data degraded regardless).
- T6 EquipmentVariableState read projection (BadOutOfService/null); Warm Standby refuses writes BadInvalidState.
- T7 D4 mechanism adopted after T0: ordinary-tag marker on MonitoredItemNotification.Message validated by
  (ClientHandle, DataValue); first-delivery decision keyed by (SubscriptionId, SequenceNumber); the SDK's
  cached copy is never mutated.
- T8 queue current/BadOutOfService to live ordinary-tag items ignoring filters.
- T9 alarm stamp/deliver split (suppressed on Standby), Part 9 methods refused on Standby, promotion
  refresh via ISubscription.ConditionRefresh2Async on every live event item.
- T10 deploy-time RedundancyMode cross-check + six instruments.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:39:50 -05:00
Joseph Doherty dcff5ae994 fix(host): leave the cluster before Kestrel drains and end open streams on stop (#906)
v2-ci / unit-tests (push) Waiting to run
A planned stop of the Primary took ~30 s on QA: WebApplication stops the web
server before Akka.Hosting's service, and Kestrel's graceful drain waited the
whole ShutdownTimeout for the peer's open LocalDb sync stream (the library's
LocalDbSyncService only observes the call token, which fires at the END of the
drain). Only then did CoordinatedShutdown run, so the stopping node held the
Primary role and the peer sat at ServiceLevel 240 for the whole window.

- GracefulClusterLeaveService (IHostedLifecycleService): StoppingAsync runs
  before any hosted service's StopAsync; it raises ApplicationStopping, then
  runs and awaits CoordinatedShutdown with ClrExitReason (not Cluster.Leave,
  whose ClusterLeavingReason makes Akka.Hosting set ExitCode -1). The cluster
  is left while Kestrel is still up.
- ShutdownStreamCancellationInterceptor (innermost on the shared gRPC
  pipeline): every streaming call gets a context whose token is linked with
  ApplicationStopping; streams arriving once stopping are refused Unavailable.
  Host-side mitigation until LocalDb 0.3.8 is published and pinned
  (scadaproj 3ddb083, scadaproj#57); it also covers the mesh streams.

Test GracefulStopWithOpenSyncStreamTests (real WebApplication + Kestrel h2c +
MapZbLocalDbSync + single-node Akka cluster + real peer initiator, 8 s drain
budget): stop 895 ms, cluster left at 892 ms, drain began at 894 ms. Positive
control without the fix: stop 8863 ms, cluster left at 8841 ms after the drain
began at 0 ms. Mutations: no CoordinatedShutdown in StoppingAsync -> order
assertion fails (drain at 1 ms, removed at 893 ms); interceptor passes the
original context -> stop 8006 ms, fails.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:32:13 -05:00
Joseph Doherty 2688bd5cfd docs(deploy): #780 QA ScriptedAlarm seam applied (drill pending) and vd03 runner flip to OutOfProcess
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:15:34 -05:00
Joseph Doherty 8667ce58aa docs(717): adopt the T0-proven first-delivery mechanism in spec §4.6/§10 and plan T2/T7/T13
v2-ci / unit-tests (push) Has been cancelled
The decision record is keyed by (SubscriptionId, SequenceNumber); the ordinary-tag marker rides
MonitoredItemNotification.Message, validated by ClientHandle and DataValue. The SDK's cached copy is
never mutated. None deletes the SDK's orphan i=11314. T13 gains transfer and durable-restore legs.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:03:56 -05:00
Joseph Doherty 2238da6504 docs(717): T0 SDK 2.0.1 re-proof — metadata and read projection pass; provenance and first-delivery choice fail as designed
v2-ci / unit-tests (push) Has been cancelled
The throwaway loopback spike (11/11 over the wire) shows SDK 2.0.1 clones the cached NotificationMessage on every
Publish, queued send and Republish, and pools notification payloads, so design §4.6's object-keyed provenance and
first-delivery ledger never match a delivered object (0 hits; Good leaks after demotion). A ledger keyed by
(SubscriptionId, SequenceNumber) plus a provenance token on the clone-copied MonitoredItemNotification.Message helper
passed the same matrix. Per the plan's T0 exit rule this re-opens D4's mechanism; T1+ wait for the owner's go/no-go.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:54 -05:00
Joseph Doherty a42c9d7f7c docs(717): record the owner's 2026-11-13 answers in the Warm redundancy spec and plan
Standby writes and Part 9 methods refused BadInvalidState under Warm, promote while the Primary is
unreachable (OpcUa:Redundancy:PromoteWhilePrimaryUnreachable, default on), standby status code
BadOutOfService, no ConfigDb migration, bands as drafted. ScadaBridge#138 is the filed prerequisite.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:54 -05:00
Joseph Doherty c0ba450d46 docs(deploy): vd03 + QA vq03/vq04 deployed to c9640f5c (#901 #902 #903 #834 #872 #904)
v2-ci / unit-tests (push) Has been cancelled
DEV vd03 6c4d2990 -> c9640f5c (~101 s), QA pair fea7de0a -> c9640f5c (~91 s pair
outage). #901 settings record written once per node, swaps gated on -Verify,
all gates pass (SL 250 / 250-240, failover no failback, LocalDb sync, canary,
no historian writes). #903 edit-and-revert leaves drift: RowVersion is in the
revision hash.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:06 -05:00
Joseph Doherty 6e67250e74 docs(deploy): next vd03 (6c4d2990) + QA (fea7de0a) deploy pack to c9640f5c, read-only ConfigDb preflight
v2-ci / unit-tests (push) Has been cancelled
No migrations in either range; no #834/#898/#899/#897/#896 findings on DEV or QA.
QA isolation record holds 12 deny paths (OTel collector added 2026-11-12), not 11.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:20:19 -05:00
Joseph Doherty c9640f5ced fix(galaxy): picker-form alarm ack waits for the feed snapshot instead of dialling bare tag.Attr (#904)
v2-ci / unit-tests (push) Has been cancelled
A picker-form native alarm (attributeRef tag.Attr) only learns the provider-qualified reference
Galaxy!<Area>.tag.Attr from the alarm feed. After a host restart the condition can be restored from
LocalDb before the feed's first snapshot reached the new driver, and an ack in that window dialled the
bare tag.Attr, which the gateway's AcknowledgeAlarm refuses.

The feed now exposes InitialSnapshotCompleted (set on snapshot_complete, after every snapshot entry has
been raised). AcknowledgeAsync resolves the reference through ResolveAlarmAckReferenceAsync: when a
picker-form tag has learned nothing yet and the feed is running, it waits for that snapshot (2 s, and the
caller's token) and dials the reference the snapshot names, so the area always comes from the feed and a
re-areaed object acks under its current area. No reference is guessed: if the snapshot does not name the
alarm, or does not arrive in time, the authored reference is dialled as before and refused visibly.

Chosen over the alternatives in the issue: a GR-browse area lookup can be stale against the runtime
area and is not loaded for authored tags, and a gateway-side tag.Attr resolution needs a cross-repo
deploy and would resolve from the same monitor cache the snapshot already carries.

Tests: 5 driver cases + 1 feed case (fail with the call site / sentinel reverted), and a skip-gated live
gate GalaxyPickerAckBeforeSnapshotLiveTests, run against windev's TEST Galaxy (TestMachine_015.TestAlarm003):
an ack issued before the snapshot dials Galaxy!TestArea.TestMachine_015.TestAlarm003 and is confirmed; with
the call site reverted it fails with the gateway's InvalidRequest refusal.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:07:38 -05:00
Joseph Doherty f0c6cf9640 fix(security): running drivers refuse out-of-namespace credential refs at resolve time (#834)
v2-ci / unit-tests (push) Has been cancelled
The #815 warn-only compatibility window is over. A running Galaxy driver
(InitializeAsync, every gateway ReopenAsync, the ack fallback password) and a
running OpcUaClient driver (session open) now refuse an out-of-namespace
credential reference with the same CredentialReferenceException that save,
deploy (CredentialReferenceOutOfScope), Test Connect and browse already raise,
before any store / environment / file read and before any endpoint is dialled.
The one existing scope check (CredentialReferences.Parse/ResolveAsync) is now
the only path; nothing new was added.

Removed:
- LegacyCredentialReferenceTracker (Core.Abstractions), its rate-limited WARN
  and the credential-compat:<field> diagnostics entry.
- The per-driver Legacy* resolve methods and every `legacy` parameter
  (GalaxySecretRef, GalaxyAckCredentialResolver, OpcUaUserIdentityFactory,
  OpcUaUserCertificatePaths, OpcUaClientSecretResolution).
- IDriverDiagnostics on GalaxyDriver / OpcUaClientDriver (it only carried the
  tracker's entries).
- The tracker also kept a pre-#836 absolute UserCertificatePath loading on a
  running OpcUaClient driver; that is refused now too.

Preflight / banner decision: KEPT, as a hard error. The Running source
(driver-health credential-compat entries) and CredentialReferenceBannerModel
are gone with the tracker, but the ConfigDb scan still tells an operator
something no other surface does: a Deployed finding is now a driver that is
not connecting, a Draft finding is a deploy the gate will refuse - named
before anyone deploys, e.g. after restoring an old ConfigDb backup or
importing a pre-#815 config. It also carries the #835 cross-instance and
#836/#850 path checks. CredentialReferencePreflightService now logs one Error
per finding (was Warning) with no warn-only wording, and the /raw banner is
alert-danger.

DEV and QA ConfigDbs hold no Galaxy/OpcUaClient driver (checked 2026-11-13);
no production install exists.

Tests: GalaxyDriverLegacyCredentialTests replaced by
GalaxyDriverCredentialScopeTests; new OpcUaClientDriverCredentialScopeTests;
tracker tests in OpcUaClientSecretResolutionTests /
OpcUaClientUserCertificatePathTests deleted; AdminUI preflight tests drop the
Running/banner-model cases and assert Error level. Against the old code the
new refusal tests fail (Galaxy 5/6, OpcUaClient 4/5; the two positive controls
pass on both).

docs/security.md: the migration table now says running drivers refuse; the
warn-only section is replaced by "The compatibility window is over".

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:06:43 -05:00
Joseph Doherty 2cb8a2c317 fix(install): worker isolation survives a folder swap: recorded settings, -Reapply, -Verify, swap script (#901)
v2-ci / unit-tests (push) Has been cancelled
An upgrade that swaps the install folder dropped the explicit ScriptWorker isolation ACEs on script-worker\, the
install folder and a copied-back pki\ (vd03 2026-11-13). Re-applying needed the operator to repeat the arguments.

- Install-ScriptWorkerIsolation.ps1 records its effective settings (InstallRoot, HostServiceAccount, PeCacheDir,
  HostServiceName, ConfigureHostService) beside DenyPaths under ...\Services\OtOpcUaScriptWorker\Parameters (new plan
  step SettingsRecord) and locks that key to SYSTEM + Administrators (protected DACL; no secrets recorded).
- -Reapply: re-apply from the record, no arguments; prints a summary line (changed/unchanged).
- -Verify: check only, [ok]/[MISSING] per step, exit 1 when anything is missing. -Uninstall also works from the record.
  A node isolated by a pre-#901 installer (DenyPaths only) must run the full-argument install once; both modes say so.
- Invoke-OtOpcUaInstallSwap.ps1: preflight -> stop -> move live/candidate -> carry appsettings/pki/logs ->
  -Reapply -> -Verify -> start; a failed -Verify leaves the service stopped. Also the rollback (-NoCarry).
- Tests: dry-run harness 101/101 (record + argument rules; record-key SDDL mutation turns 2 red). The Windows test adds
  swap -> Verify fails (exactly the 3 lost ACEs) -> Reapply (3 changed) -> Verify ok -> Reapply 12/12 unchanged, PE-cache
  and firewall negative controls, the swap script, and the pre-#901 refusal; 52/52 on windev under 5.1 and 7, with a
  scratch local host account and with SYSTEM. Evidence: scripts/validation/opcua-windows-service/evidence/2026-11-13-windev-901-reapply-verify.
- Docs: ServiceHosting isolation section; vd03 pack (swap step, rollback, post-deploy gate, one-time full-argument run),
  the 2026-11-13 vd03 record, and the QA pack (QA swap preceded the isolation install; ACLs checked intact over SMB on
  vq03/vq04 2026-11-13; next-deploy procedure).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:06:10 -05:00
Joseph Doherty 2470f82eea fix(galaxy): Pending/Warning status rows map to Uncertain; detail 8001 to BadTypeMismatch (#872)
v2-ci / unit-tests (push) Has been cancelled
Owner decision 2026-11-13 on #872 item 1. StatusCodeMap.FromMxStatus now maps a
Pending or Warning row to Uncertain (0x40000000) for any detail, ahead of the
detail vocabulary, keeping the once-per-(detail, category) warning. MXAccess sets
success = -1 on Pending (accepted, in progress); neither category is a failure.

Detail 8001 (automation-object ConfigurationError, write rejected for type
conversion, live-captured on windev 2026-11-13) maps to BadTypeMismatch. 8018
stays BadUserAccessDenied via its SecurityError category, now pinned by a test.

Data change: the value is published Uncertain. Write reply: the result is
Uncertain, never Good, so the server (which commits only Good) reports the write
not committed and reverts the node; a later MXAccess commit arrives as a data
change. Host status is unchanged (probe decode treats Uncertain and Bad alike,
quality < 192 -> Unknown); no alarm path calls the map.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:01:07 -05:00
Joseph Doherty 35576485b1 fix(adminui): deployments drift chip never serves a stale "In sync" (#903)
v2-ci / unit-tests (push) Has been cancelled
Root cause: DeploymentDriftProbe (a singleton shared by every circuit) cached its verdict until
the ConfigEdit audit watermark (MAX(EditedAtUtc), COUNT(*)) moved, assuming every configuration
write appends a ConfigEdit row. Only deployments and driver restart/reconnect do; the AdminUI
editors (RawTreeService, UnsTreeService, ScriptEdit, cluster/node/ACL pages) write the config
tables directly. So after a draft change the page kept showing the cached "In sync" until
something unrelated moved the watermark.

- DeploymentDriftProbe: the watermark cache is gone; every verdict comes from a composition that
  started after the caller asked. Compositions stay serialised behind the gate, and callers queued
  behind one share a verdict only when that composition started after they arrived (start
  sequence number) for the same sealed hash, so sharing is exact, never stale.
- Deployments page: the drift check runs in OnAfterRenderAsync (interactive render only, so a
  visit costs one composition, not two with prerender); the first render shows a neutral
  "Checking for changes..." chip, never a verdict; a failed check shows "Drift check failed" and
  logs a warning instead of crashing the page. A deploy re-runs the check.
- Tests: probe tests for an un-audited draft change (real ConfigComposer over InMemory), fresh
  composition per call, and queued callers not reusing an older composition; bUnit page tests that
  a second page load after a direct draft change shows "Configuration drift" and that the first
  render shows "Checking" until the comparison answers. All fail against the previous code.
  DeploymentDriftProbeSqlServerTests is removed: it pinned the SQL translation of the watermark
  query, which no longer exists.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 01:50:37 -05:00
Joseph Doherty b85e004bdb fix(adminui): Monaco dispose on a gone circuit is expected, not an error (#902)
Closing a tab with the script editor open ended the circuit; MonacoEditor.DisposeAsync then
called MonacoBlazor.dispose through SafeInvokeAsync, which caught only JSDisconnectedException,
so the TaskCanceledException from the torn-down circuit escaped and was logged at [ERR].

DisposeAsync now treats JSDisconnectedException, OperationCanceledException (incl.
TaskCanceledException) and ObjectDisposedException as the expected gone-circuit outcome and logs
them at Debug; a real JSException from a live page is still logged (Warning); the
DotNetObjectReference is released in a finally. MonacoEditor is the only component in the repo
that uses JS interop, so there is nothing else to align.

bUnit tests: dispose throwing each of the three gone-circuit exceptions neither escapes nor logs
above Debug (with a positive control that the failing call ran), and a JSException still warns.
They fail against the previous DisposeAsync.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 01:50:37 -05:00
Joseph Doherty d528838864 fix(galaxy): #895 review - re-areaed, subtag-provider and pre-subscription alarm references resolve
v2-ci / unit-tests (push) Has been cancelled
Review of e113faf5 found three gaps in the alarm-feed reverse map:

- A tag hand-authored as Galaxy!<Area>.tag.Attr stopped resolving once Galaxy moved the object to
  another area: the feed's full reference named the new area and none of the fallbacks looked at the
  authored provider form. Provider-form attributeRefs are now also indexed by their area-free
  tag.Attr, and the ack dials the reference the feed now uses.
- A reference composed with an empty area (Galaxy!tag.Attr, the gateway's ComposeFullReference when
  the area is blank) with the subtag provider's object-only source reference stripped "tag." and
  missed. The part after the '!' is now also tried.
- Resolution ran after the subscription gate, so a transition seen with no handle registered taught
  nothing for the ack. It now resolves first; fan-out is still gated.

A picker tag and a provider-form tag for the same alarm resolve deterministically and log a warning.
New tests cover re-area for both forms, subtag shapes, H6-style dotted refs, unmatched/malformed
refs, the collision, pre-subscription learning, and the #869 confirmation of a picker-form ack;
reverted-code runs fail them.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 01:49:07 -05:00