Commit Graph

5060 Commits

Author SHA1 Message Date
Joseph Doherty 1fe4dbf51c Standardize cluster anatomy and gate authoring destinations 2026-11-13 21:43:30 -05:00
Joseph Doherty 78a39a1095 Standardize administrative page and table grammar 2026-11-13 21:43:30 -05:00
Joseph Doherty 75ff38103f Standardize remaining runtime page and table grammar 2026-11-13 21:43:15 -05:00
Joseph Doherty cb3400c13c fix(admin-ui): align task orientation and navigation with page policies 2026-11-13 21:42:30 -05:00
Joseph Doherty 28add408c4 Name common identity resilience and static builder fields 2026-11-13 21:38:19 -05:00
Joseph Doherty 27bca683c1 Complete protocol field associations and verify rendered inventory 2026-11-13 21:29:03 -05:00
Joseph Doherty ee8ac73e8c Standardize first runtime page and table anatomy 2026-11-13 21:24:07 -05:00
Joseph Doherty 169149f1eb Improve admin field names and generation-safe dialog lifecycle 2026-11-13 21:15:58 -05:00
Joseph Doherty 06b1743051 Associate protocol tag fields batch 3 2026-11-13 21:05:33 -05:00
Joseph Doherty 1746b04588 Associate protocol tag fields batch 2 2026-11-13 21:05:33 -05:00
Joseph Doherty 8cc053ceb7 fix(admin-ui): pass contextual names to native Monaco editor 2026-11-13 21:04:30 -05:00
Joseph Doherty 2415db8ca4 Associate AB and Focas tag editor fields 2026-11-13 20:53:58 -05:00
Joseph Doherty 7e57074653 fix(admin-ui): associate native device endpoint fields 2026-11-13 20:49:26 -05:00
Joseph Doherty 7b7fdf0eb7 Associate S7 SQL and TwinCAT driver fields and diagnostics 2026-11-13 20:42:31 -05:00
Joseph Doherty eed546d96d Associate MQTT MTConnect and OPC fields with readable feedback 2026-11-13 20:33:37 -05:00
Joseph Doherty c70bc44cd9 Associate Galaxy Heidenhain and Modbus driver fields 2026-11-13 20:23:31 -05:00
Joseph Doherty 9be717c0d7 Associate fields in AB and Focas driver editors 2026-11-13 20:19:43 -05:00
Joseph Doherty a47ffb9151 test(admin): replace bUnit preview with stable 2.7.2 2026-11-13 15:23:32 -05:00
Joseph Doherty 79371402e4 Merge branches 'worktree-agent-a41f08fe289ebe7a8', 'worktree-agent-a21d19bd14da138e7' and 'worktree-agent-a8d85b6d5ef424006'
v2-ci / unit-tests (push) Failing after 1h32m12s
2026-11-13 12:43:14 -05:00
Joseph Doherty 73132b7d78 fix(opcuaclient): #910 name the per-endpoint connect budget; widen it in the #816 host-name tests
Cause: the client-side PerEndpointConnectTimeout (3 s), not opc-plc. The
validate:null case is the first test the assembly runs, so it pays the
cold-process cost of the first Basic256Sha256 connect: loading the fresh
throwaway certificate, validating the server's, and the asymmetric handshake.
In the G2 full-solution run that local work used up the budget. The opc-plc
log at 13:13:50-53Z shows the discovery channel closing at 50.709, then nothing
from the client for 3 s, then OpenSecureChannel answered at 53.775 and the
client closing the channel 48 ms later. The SDK reports a wait cancelled by
that budget as BadRequestInterrupted. opc-plc auto-accepted the certificate
(--aa, "Trusting certificate ... command line option") with no reject and no
store round-trip.

- Driver: a per-endpoint budget expiry now surfaces as TimeoutException naming
  PerEndpointConnectTimeout instead of BadRequestInterrupted or a bare
  TaskCanceledException. There is no behaviour change. Unit test added
  against a silent TCP listener.
- Test: the ServerHostNameValidationTests secure options use a 15 s budget,
  the same as the browser integration tests' 10 s. Connect latency is not
  what these tests check.
- docs/drivers/OpcUaClient.md: documents what the budget covers.
2026-11-13 12:42:45 -05:00
Joseph Doherty fc08892eb3 fix(modbus-fixture): run pymodbus sim without asyncio debug mode; FC03 healthcheck + watchdog (#909)
Root cause: pymodbus 3.13's `pymodbus.simulator` console script hard-codes
asyncio.run(run_main(), debug=True). Debug mode captures a traceback per
scheduled callback and logs callbacks slower than 100 ms (the "took 0.146
seconds" lines in #909). Side by side on the shared docker host (closed-loop
FC03 clients, 2 s deadline): 100 clients got ~120-170 req/s, p99 1.6-1.9 s
with client timeouts in debug mode vs ~2400 req/s, p99 ~135 ms without it.
The shared sim also carries the docker-dev rig's 49 Modbus drivers on each of
central-1/central-2 (98 sockets, ~220 req/s at idle), which is above the
debug-mode capacity, so a test run on top was answered late ("Cancel send,
because not connected!") and timed out.

- run_simulator.py: same CLI as pymodbus.simulator, debug=False, plus a watchdog
  thread that exits after 3 failed FC03 probes so restart: unless-stopped
  replaces the container (docker has no restart-on-unhealthy).
- fc3_probe.py: stdlib FC03 read with a deadline (MBAP or RTU framing); the
  compose healthcheck.
- docker-compose.yml: shared anchor for every pymodbus service: new entrypoint,
  `project: lmxopcua` label, restart: unless-stopped, healthcheck, bounded
  json-file logs.
- README: why, the numbers, and how to diagnose load.
2026-11-13 11:03:08 -05:00
Joseph Doherty f1da809f3c test(scripted-alarms): wait for the sibling run to finish before asserting InFlightCount (#911)
ScriptRunGuard counts a run on StartedRunCount and InFlightCount together the
moment it is admitted, but releases InFlightCount only in the run's finally.
The runaway test waited on StartedRunCount >= 6 and then asserted
InFlightCount == 1, so the healthy sibling's 4th-push run - admitted but still
executing - was counted alongside the runaway orphan (2). Test race, not an
engine defect: widening only the healthy run reproduced it 5/5.

- wait until StartedRunCount == 6 AND InFlightCount == 1
- healthy predicate spins ~100 ms so the window is open on every run
- positive controls: the final push must be refused once more (the in-flight
  run is the orphan) and the quality-once absence check runs only after the
  healthy Activated event from the later pass has been delivered

Under 28 busy-loops on 14 cores: before 2/60 failed, after 0/60.
2026-11-13 10:46:19 -05:00
Joseph Doherty e299dc090a chore(ga): retire G1 - delete the stale Phase 6 compliance scripts; v2 GA exit criteria all met/waived/retired
v2-ci / unit-tests (push) Has been cancelled
The phase-6-*.ps1 scripts checked v2 Phase 6 code that later commits deliberately
deleted (76310b88 v1 Server/Admin, #522, #524, LiteDB cache, v3 migration squash):
56/56 failures were retired files, so they could not pass. Owner retired G1; G2
plus the per-area suites cover the invariants. phase-7-compliance.ps1 is untouched.
2026-11-13 10:33:43 -05:00
Joseph Doherty f93a1179dd docs(ga): G7 PASS - FOCAS live CNC on vd03 (Makino 31i-B), axis tag added via owner-approved deploy e9fbe63e
v2-ci / unit-tests (push) Waiting to run
2026-11-13 10:32:09 -05:00
Joseph Doherty 1862fab671 docs(deploy): QA vq03/vq04 switched to OpcUa:Redundancy:Mode=Warm; #138 live gate PASS (#907, #717)
v2-ci / unit-tests (push) Failing after 14m56s
Two service-env entries per node (Mode=Warm, PeerApplicationUris__0), pair stopped and
started together. Gates: 250/90, RedundancySupport Warm, standby BadOutOfService,
LocalDb sync, canary Good on the Primary; standby Write could only show BadNotWritable
(QA tags are read-only). ScadaBridge WarmCanary moved off the Standby by ServiceLevel in
0.1 s (connect-time preference); graceful Primary stop still took the 66 s connectivity
path (ScadaBridge#141). End state: Warm, vq04 Primary.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 09:00:35 -05:00
Joseph Doherty 22de358f74 docs(ga): G2 PASS (one defect fixed in f416b883; #909/#910/#911 environmental); G1 + G7 evidence recorded pending owner decision
v2-ci / unit-tests (push) Has been cancelled
2026-11-13 08:59:07 -05:00
Joseph Doherty a157b27628 fix(install): swap waits for the host process to exit before moving the folder (#908)
v2-ci / unit-tests (push) Has been cancelled
On QA (76e73e7d deploy) Invoke-OtOpcUaInstallSwap.ps1's first Move-Item of
app\ failed "file in use" right after the service reported Stopped: SCM
reports Stopped while the host is still exiting (e.g. a log file under
InstallRoot still open).

- After the stop, wait (bounded, -ProcessExitTimeoutSeconds, default 60)
  until the service's host process (PID + start time captured before the
  stop) and every process imaged under InstallRoot have exited; otherwise
  throw naming them, with nothing moved.
- The renames use [IO.Directory]::Move instead of Move-Item. On windev,
  PowerShell 7's Move-Item falls back to an item-by-item move when the
  rename fails because a file under the folder is open, and leaves a partial
  PreviousInstall tree behind (a retry then nests into it).
- The first rename is retried for at most -MoveRetrySeconds (default 10)
  while it fails because something under InstallRoot is open (a backstop
  for a holder whose image is outside InstallRoot).
- -Verify/-Reapply/-Scope semantics unchanged.

Tests: dry-run harness gains fake-process-list tests for the wait and the
retry plus AST order checks (145 passed); the Windows harness gains three
real cases (holder past the budget, holder released 2 s into the wait,
holder outside InstallRoot) - 77/0 on pwsh 7 and Windows PowerShell 5.1 on
windev; with the wait removed the first two fail.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 08:46:27 -05:00
Joseph Doherty f416b88324 fix(717): use RoleParser.Driver in the Warm verdict (RoleAuthorityGuard)
v2-ci / unit-tests (push) Has been cancelled
c569f806 passed the bare "driver" literal to WarmRoleAuthority.OldestUpMember
at OpcUaPublishActor.cs:1875-1876, which RoleAuthorityGuardTests forbids outside
RoleParser. Same value; found by the #615 G2 full-suite run. Cluster.Tests
255/255, Runtime Warm tests 23/23.
2026-11-13 08:33:15 -05:00
Joseph Doherty 16ddddb2eb docs(615): G5 signed off by the Fleet Admin; every #615 gate complete, #615 closed
v2-ci / unit-tests (push) Has been cancelled
2026-11-13 08:09:37 -05:00
Joseph Doherty 5baf05f634 docs(deploy): vd03 + QA deployed c9640f5c -> 76e73e7d (#905 #906 #900 #717, LocalDb 0.3.8)
v2-ci / unit-tests (push) Has been cancelled
Executed 2026-11-13 12:27-12:48Z; target moved from the pack's 47f0b98c to the tip 76e73e7d
(HG Client 1.1.0, behaviour unchanged; note added to the pack).

- vd03: ~14 s downtime (was ~101 s; #905 window 5.7 s), SL 250, FOCAS 21/22, Mode Unset,
  HG Probe 494 -> 536 with no write RPCs (added gate for #900).
- QA: pair outage 36.5 s (was ~91 s), 250/240, both failover legs, no failback, sync
  connected, canary Good. #906: vq04 (sync server) shutdown -> Leaving 30.0 s on c9640f5c,
  21 ms on 76e73e7d; peer at 250 in 0.47 s (leg 1: 0.36 s).
- Deviation: the first Move-Item of app\ failed "file in use" right after the stop on both
  QA nodes; a retry with the service stopped succeeded (lmxopcua#908).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 07:51:01 -05:00
Joseph Doherty c9bc358dc9 feat(cli): print array values; docs: #615 G9 PASS with otopcua-cli, G8 waived
v2-ci / unit-tests (push) Has been cancelled
G9 ran on an isolated 2-node pair (otopcua-g9, 003c7f89, Mode unset as
deployed) with otopcua-cli as the owner-approved client: A1-A3, A5, A7, A8
(graceful, ~11 s gap), B2 (docker kill, failover <1 s, ~2 s gap, 0 Bad,
survivor 250 at +44 s) and B3 (no failback) pass; A4/A5b unmet under Unset,
met under Warm (#717). G8 (CTT) waived by the owner for GA; G5 is the only
open exit criterion.

The run found the CLI printing every array as System.String[] (ServerArray
unreadable); ValueFormatter now renders [a, b] / hex for read, subscribe and
historyread, with tests. Runbook A4/A5b text updated for the #717 modes.
Evidence: docs/plans/evidence/2026-11-13-615-g9/RESULTS.md.
2026-11-13 07:46:56 -05:00
Joseph Doherty a59dbd4f07 docs(heidenhain): driver accepted, #829/#853 closed
The ignitionoee CR capture (6d2aec40: 7 h read-only sample of 34 production
controls on the Java LSV2 driver) and the 2026-11-11 iTNC 530 run did not
change the driver's code: ProgramState 0 = cutting (11,865 of 11,869 motion
pairs), 7 = idle; TNC 426/430 BadNotSupported; overrides BadUserAccessDenied
under INSPECT; capabilities Unknown until first poll; alarm classes 1/7/8 seen,
class 5 not. Owner accepted the driver on that basis; remaining limits stay
documented in Heidenhain.md §Hardware-gated.
2026-11-13 07:46:56 -05:00
Joseph Doherty 76e73e7dbf chore(historian): HistorianGateway Client/Contracts 1.1.0, typed error reason (#900)
v2-ci / unit-tests (push) Has been cancelled
Bump ZB.MOM.WW.HistorianGateway.Client + Contracts 1.0.0 -> 1.1.0 (historiangw#135,
release 6365f34). GatewayErrorReasons now reads Client 1.1.0's typed
HistorianGatewayException.StatusCode / ErrorReason / ErrorDomain (constants from
HistorianGatewayErrorReasons) instead of parsing grpc-status-details-bin itself; the
direct Grpc.StatusProto pin and reference are dropped (the client brings it). The
published client maps every RpcException to a HistorianGatewayException, so the
trailer fallback is not needed. Behaviour unchanged: FailedPrecondition +
AGGREGATE_NOT_SUPPORTED_FOR_TAG (domain historian_gateway.v1) ->
BadAggregateNotSupported, everything else keeps its handling.

Adds GatewayErrorReasonsTests (typed-property-only exceptions, so they fail against
the old trailer implementation); wire tests and docs/Historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 07:17:10 -05:00
Joseph Doherty 97d513f7dc docs(deploy): next vd03 + QA deploy pack c9640f5c -> 47f0b98c (#905 #906 #717 LocalDb 0.3.8), read-only preflight
v2-ci / unit-tests (push) Has been cancelled
Range, settings (no migration; OpcUa:Redundancy:Mode unset everywhere), read-only
ConfigDb/SMB/OPC UA preflight on both tiers, #905 scoped swap with -NoStart on both
tiers, #906 failover timing gate (stop the sync-server node as Primary), Unset
metadata gate, rollback with the vd03 #903 drift note. Not executed.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 07:02:20 -05:00
Joseph Doherty 47f0b98c6e chore(deps): pin ZB.MOM.WW.LocalDb 0.3.8 (scadaproj#57)
v2-ci / unit-tests (push) Has been cancelled
LocalDb 0.3.8 ends open sync streams on ApplicationStopping and refuses
new ones Unavailable while stopping (lmxopcua#906, library side). The
host-side ShutdownStreamCancellationInterceptor stays: it also covers
the mesh telemetry/command streams.

GracefulStopWithOpenSyncStreamTests' positive control started passing
the stop in ~950 ms under 0.3.8, because the library alone now ends
the stream. The control registers LocalDbSyncService without the host
lifetime (the 0.3.7 behaviour), so it still reproduces the 8 s drain.

Release build 0 warnings. Host.Tests 288/288, Host.IntegrationTests
350 passed / 2 skipped, Runtime.Tests 1063 passed / 3 skipped,
Core.AlarmHistorian.Tests 53/53.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 06:55:27 -05:00
Joseph Doherty 003c7f8954 docs(deploy): #780 QA scripted-alarm drill passed (kill x2, quarantine, siblings Good, 250/240); config back to rev 82dd0598
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 06:54:09 -05:00
Joseph Doherty 36eb6d558d docs(717): Mode=Unset re-check after c569f806 (no change); a leaving node reports 0
v2-ci / unit-tests (push) Has been cancelled
T13b: the isolated otopcua-t13b pair with OpcUa:Redundancy:Mode absent, on origin/master f826af8e
(it contains c569f806) and A/B on c569f806^ (687256d6). Every check passes on both builds:
- levels 250/240;
- Good data on both nodes;
- writes, Acknowledge, AddComment and alarm delivery on the Secondary;
- graceful stop: the survivor reaches 250 259 ms after Leaving (pre-fix 270 ms);
- crash: 250 at +43.6 s (pre-fix +42.8 s), with no demotion at auto-down;
- rejoin at 240 with no failback.
The role-decision changes are gated on IsWarm and do not run under Unset.

Plan L5, the design's section 4.3 and the Warm table in Redundancy.md now say a Leaving/Exiting node
reports ServiceLevel 0, not 90 (ComputeWarm returns 0 for any member that is not Up/Joining; T13 note A).

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 06:44:46 -05:00
Joseph Doherty f826af8e8a docs(717): T13 live gate evidence on an isolated Warm pair
v2-ci / unit-tests (push) Waiting to run
Every applicable T13 leg passes on 0c19f26c + c569f806. The leg list covers:
- metadata and ServiceLevel bands;
- standby projection;
- alarm suppression with Standby ConditionRefresh and the promotion refresh;
- the three D4 demotion timings, run live;
- standby write and Part 9 refusal (BadInvalidState);
- graceful and crash failover, with no failback;
- subscription transfer;
- the Mode=Unset regression.

The two crash-path defects the gate found are fixed in c569f806. Evidence, rig files and the
raw-client harness are in docs/plans/evidence/2026-11-13-717-t13/.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:39:17 -05:00
Joseph Doherty c569f80694 fix(717): Warm crash survivor stays Active through auto-down; verdict reads its own events
Two defects on the crash-failover path, found by the T13 live gate on an isolated docker pair:

1. When auto-down removed the unreachable Primary (~15 s after the survivor promoted), the
   snapshot's Primary left the unreachable set, so (3') failed and the survivor demoted to
   Standby (BadOutOfService, ServiceLevel 50) for ~15 s, until its own singleton published
   after the hand-over retries. A Primary that is Down, or that this node saw removed, now
   counts as gone (WarmRoleAuthority.SnapshotPrimaryGone). A member that is merely
   Leaving/Exiting, or that this node never saw, is not.

2. The verdict read Cluster.State / SelfMember (Akka's ClusterReadView, a separate subscriber
   to the same events), which can lag the event being handled. Live, the survivor stayed
   Standby 4.3 s after its own UnreachableMember event (promotion waited for the next health
   tick). The Warm verdict now uses a membership view built from this actor's own
   member/reachability subscription, falling back to the read view only before the first event.

Tests: WarmRoleAuthorityTests.A_snapshot_Primary_that_auto_down_removed_is_still_gone_and_the_survivor_stays_Active,
OpcUaPublishActorWarmTests.The_crash_survivor_promotes_on_its_own_unreachable_event_and_stays_Active_through_the_removal
(synthetic events the read view never sees). Mutations: read-view-only verdict and the removed
arm switched off each fail the actor test. Runtime.Tests 1058 passed, 3 skipped.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:36:58 -05:00
Joseph Doherty 687256d68e test(717): deferred Warm coverage - 2-node handover, DataEncoding, deleted/reused items, clone matrix, ACL stand-down, per-counter metrics
v2-ci / unit-tests (push) Has been cancelled
- T3 WarmSingletonHandoverTests: real 2-node cluster, singleton under ClusterSingletonManager on both nodes; the old
  incarnation's held-back publish (older revision, higher sequence) reaches a real Warm OpcUaPublishActor after the
  handover and never demotes the new Primary. Pin: a removed node cannot rejoin from its old ActorSystem; a new one
  rejoins younger (higher up-number).
- T6 DataEncoding: Binary/XML encoded reads of a structured tag projected on the Standby; scalar encoding error kept;
  a DefaultXml item is projected, then promoted through its own encoding (TypeId i=888).
- T7 item deleted between queue and publish; ClientHandle reused by i=2267 on a payload recycled through the SDK 2.0.1
  pool (precondition asserted); WarmStandbyCloneMatrixTests over DataChange+DiagnosticInfos/EventNotificationList/
  StatusChange, clone/cached immutability, encode round-trip, real-pool stale marker.
- T8 ACL-stood-down item receives nothing from demotion or promotion; restored grant reaches it again.
- T10 per-counter MeterListener tests (by instrument instance): warm_transition, warm_role gauge, snapshot_rejected
  (publish actor both reasons + DriverHostActor's ordering gate, first actor test of it), projected_notifications,
  suppressed_alarm_events. Process-wide state tests run in non-parallel collections.
- Every test mutation-checked (26 mutations killed); no product bug found. Plan status updated.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 04:31:11 -05:00
Joseph Doherty 0c19f26c81 fix(install): cut the swap's stopped window to the swapped-in folder (#905)
v2-ci / unit-tests (push) Has been cancelled
Root cause: the protected-DACL coverage scan, not the ACL writes. Both -Reapply and
-Verify walked every denied folder (all of E:\ApiInstall on vd03, nested roots again)
with Get-ChildItem -Recurse + Get-Acl per item + a pipeline per item for -Skip, about
3.4 ms an item on PS 5.1, all while the service was stopped. An unchanged deny is read,
not rewritten; the re-applied install-folder deny propagates over ~1,400 files in 0.5 s.

- Find-OtOpcUaDenyCoverageGap: .NET enumeration, -Skip folders pruned, DACL-only
  GetAccessControl(Access) (about 0.15 ms an item); reparse points not followed.
- Get-OtOpcUaCoverageWalkRoots: each denied folder walked once (outermost only).
- Install-ScriptWorkerIsolation.ps1 -Scope All|InstallRoot|Outside (with -Reapply,
  -Verify, -PlanOnly). InstallRoot = the folder ACEs a swap drops; Outside = the rest,
  including the firewall rule (its query costs 3-5 s; its exe path does not change).
  The two partition the plan. Prints the scan time.
- Invoke-OtOpcUaInstallSwap.ps1: -Reapply/-Verify -Scope Outside before the stop (a
  failure stops before anything moves), -Scope InstallRoot inside the window; preflight
  checks the candidate carries the worker exe; prints the stopped window and its parts.
  -NoStart, -NoCarry and the pre-#901 refusal unchanged.

windev, vd03-like tree (17,851 items): stopped window 153.1 s -> 6.0 s (PS 5.1),
101.6 s -> 6.2 s (PS 7); the Outside part takes ~23 s with the old build serving.
Dry-run harness 101 -> 123 passed; Windows test 72 passed on PS 5.1 and PS 7.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:51:42 -05:00
Joseph Doherty f817c77e02 test(717): deferred-event boundary, ACL-filtered promotion refresh, native-ack refusal, effective-URI XML load; docs + T13 overlay
v2-ci / unit-tests (push) Has been cancelled
- Wire: an event reported while Active but deferred (#838) is delivered after demotion; the promotion refresh
  withholds an ACL-denied condition (positive control alongside); ConditionRefresh2 and model-change events on the
  Standby; a native Acknowledge is refused and never routed; Warm validates and advertises the EFFECTIVE
  ApplicationUri from an ApplicationConfigPath XML load.
- docs/Redundancy.md: Warm section, the Unset/None i=11314 behaviour (orphan readable by NodeId under Unset,
  deleted under None), instruments; OpcUaServer.md, Configuration.md, security.md, cluster-topology rule.
- docker-dev/docker-compose.warm.yml for the T13 gate (central pair, Mode=Warm).
- Plan: implementation status and the test gaps left for T13/follow-up.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:39:50 -05:00
Joseph Doherty 7829b937d1 feat(717): Warm non-transparent redundancy, default-off (T1-T10)
OpcUa:Redundancy:Mode (Unset default = today's server exactly; None; Warm; Hot and unknown values fail start).
- T1 options + validator + effective-URI check + boot banner; Warm needs the driver role.
- T2 metadata in OnNodeManagerStartedAsync: Warm -> NonTransparentRedundancyType, read-only
  ServerUriArray [self, peers], RedundancySupport=Warm, boot ServiceLevel 1; None -> RedundancySupport=None
  and the SDK's orphan i=11314 deleted; Unset untouched.
- T3 RedundancyStateChanged.Revision (up-number, incarnation tick, sequence); subscribers drop older
  revisions under Warm (OpcUaPublishActor, DriverHostActor); unrevisioned never makes a node Active.
- T4 WarmRoleAuthority + ComputeWarm bands (Standby 90/50, Active 250/200/100), PromoteWhilePrimaryUnreachable.
- T5 IStandbyProjection + DeferredStandbyProjection; OpcUaPublishActor runs promotion/demotion in the spec
  4.7 order (Active level last and retried; Standby level first, data degraded regardless).
- T6 EquipmentVariableState read projection (BadOutOfService/null); Warm Standby refuses writes BadInvalidState.
- T7 D4 mechanism adopted after T0: ordinary-tag marker on MonitoredItemNotification.Message validated by
  (ClientHandle, DataValue); first-delivery decision keyed by (SubscriptionId, SequenceNumber); the SDK's
  cached copy is never mutated.
- T8 queue current/BadOutOfService to live ordinary-tag items ignoring filters.
- T9 alarm stamp/deliver split (suppressed on Standby), Part 9 methods refused on Standby, promotion
  refresh via ISubscription.ConditionRefresh2Async on every live event item.
- T10 deploy-time RedundancyMode cross-check + six instruments.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:39:50 -05:00
Joseph Doherty dcff5ae994 fix(host): leave the cluster before Kestrel drains and end open streams on stop (#906)
v2-ci / unit-tests (push) Waiting to run
A planned stop of the Primary took ~30 s on QA: WebApplication stops the web
server before Akka.Hosting's service, and Kestrel's graceful drain waited the
whole ShutdownTimeout for the peer's open LocalDb sync stream (the library's
LocalDbSyncService only observes the call token, which fires at the END of the
drain). Only then did CoordinatedShutdown run, so the stopping node held the
Primary role and the peer sat at ServiceLevel 240 for the whole window.

- GracefulClusterLeaveService (IHostedLifecycleService): StoppingAsync runs
  before any hosted service's StopAsync; it raises ApplicationStopping, then
  runs and awaits CoordinatedShutdown with ClrExitReason (not Cluster.Leave,
  whose ClusterLeavingReason makes Akka.Hosting set ExitCode -1). The cluster
  is left while Kestrel is still up.
- ShutdownStreamCancellationInterceptor (innermost on the shared gRPC
  pipeline): every streaming call gets a context whose token is linked with
  ApplicationStopping; streams arriving once stopping are refused Unavailable.
  Host-side mitigation until LocalDb 0.3.8 is published and pinned
  (scadaproj 3ddb083, scadaproj#57); it also covers the mesh streams.

Test GracefulStopWithOpenSyncStreamTests (real WebApplication + Kestrel h2c +
MapZbLocalDbSync + single-node Akka cluster + real peer initiator, 8 s drain
budget): stop 895 ms, cluster left at 892 ms, drain began at 894 ms. Positive
control without the fix: stop 8863 ms, cluster left at 8841 ms after the drain
began at 0 ms. Mutations: no CoordinatedShutdown in StoppingAsync -> order
assertion fails (drain at 1 ms, removed at 893 ms); interceptor passes the
original context -> stop 8006 ms, fails.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:32:13 -05:00
Joseph Doherty 2688bd5cfd docs(deploy): #780 QA ScriptedAlarm seam applied (drill pending) and vd03 runner flip to OutOfProcess
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:15:34 -05:00
Joseph Doherty 8667ce58aa docs(717): adopt the T0-proven first-delivery mechanism in spec §4.6/§10 and plan T2/T7/T13
v2-ci / unit-tests (push) Has been cancelled
The decision record is keyed by (SubscriptionId, SequenceNumber); the ordinary-tag marker rides
MonitoredItemNotification.Message, validated by ClientHandle and DataValue. The SDK's cached copy is
never mutated. None deletes the SDK's orphan i=11314. T13 gains transfer and durable-restore legs.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 03:03:56 -05:00
Joseph Doherty 2238da6504 docs(717): T0 SDK 2.0.1 re-proof — metadata and read projection pass; provenance and first-delivery choice fail as designed
v2-ci / unit-tests (push) Has been cancelled
The throwaway loopback spike (11/11 over the wire) shows SDK 2.0.1 clones the cached NotificationMessage on every
Publish, queued send and Republish, and pools notification payloads, so design §4.6's object-keyed provenance and
first-delivery ledger never match a delivered object (0 hits; Good leaks after demotion). A ledger keyed by
(SubscriptionId, SequenceNumber) plus a provenance token on the clone-copied MonitoredItemNotification.Message helper
passed the same matrix. Per the plan's T0 exit rule this re-opens D4's mechanism; T1+ wait for the owner's go/no-go.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:54 -05:00
Joseph Doherty a42c9d7f7c docs(717): record the owner's 2026-11-13 answers in the Warm redundancy spec and plan
Standby writes and Part 9 methods refused BadInvalidState under Warm, promote while the Primary is
unreachable (OpcUa:Redundancy:PromoteWhilePrimaryUnreachable, default on), standby status code
BadOutOfService, no ConfigDb migration, bands as drafted. ScadaBridge#138 is the filed prerequisite.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:54 -05:00
Joseph Doherty c0ba450d46 docs(deploy): vd03 + QA vq03/vq04 deployed to c9640f5c (#901 #902 #903 #834 #872 #904)
v2-ci / unit-tests (push) Has been cancelled
DEV vd03 6c4d2990 -> c9640f5c (~101 s), QA pair fea7de0a -> c9640f5c (~91 s pair
outage). #901 settings record written once per node, swaps gated on -Verify,
all gates pass (SL 250 / 250-240, failover no failback, LocalDb sync, canary,
no historian writes). #903 edit-and-revert leaves drift: RowVersion is in the
revision hash.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:56:06 -05:00
Joseph Doherty 6e67250e74 docs(deploy): next vd03 (6c4d2990) + QA (fea7de0a) deploy pack to c9640f5c, read-only ConfigDb preflight
v2-ci / unit-tests (push) Has been cancelled
No migrations in either range; no #834/#898/#899/#897/#896 findings on DEV or QA.
QA isolation record holds 12 deny paths (OTel collector added 2026-11-12), not 11.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
2026-11-13 02:20:19 -05:00