Review of 167a48bf (FIX-FIRST): making native Confirmed/Shelving method-owned produced over-the-wire
regressions — a native TimedShelve never expired, a native OneShotShelve was silently undone by
SetActiveState, an OPC UA ack left a native alarm Unconfirmed forever (ScadaBridge maps ConfirmedState),
and any local state was per-node, never reaching the redundant peer.
- Reverted: the driver projection owns native Confirmed/Shelving again.
- Native Confirm / OneShotShelve / TimedShelve / Unshelve return BadNotSupported with no event, as native
Enable/Disable already did. Scripted alarms keep every method.
- A native Acknowledge re-asserts ConfirmedState=true and the projection's Retain rule (Active || !Acked)
before its single report. The native condition starts Confirmed.
- The method path reports only when the state actually moved (before/after). A re-ack is silent and
restores the Confirmed/Retain that the SDK flips, so a scripted node never drifts from its engine.
AddComment always reports.
- Docs: native alarms support Acknowledge/AddComment only; the Secondary-node native-ack flap is documented.
Tests: refused native methods (5), native ack stays Confirmed through echo/clear/reactivate, scripted
TimedShelve expires with exactly one unshelve event, re-ack silent (native + scripted). Each fix falsified.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
#587 — a client Acknowledge/Confirm/AddComment/Shelve/Enable/Disable emitted no condition event: the SDK's
ReportStateChange is gated on AutoReportStateChanges (never set), and the engine re-projection found the node
already in the new state and was suppressed by the delta-gate. WireMethodTransitionReporting now wraps each
method's SDK handler and, on Good, reports exactly once via ReportConditionEvent, under Lock so the routed
re-projection cannot race in ahead of the SDK applying the state. Independent of the SDK's imprecise
AreEventsMonitored bookkeeping. The shelve methods' state is applied by the wrapper (the SDK never applies
it). Native conditions' Confirmed/Shelving are method-owned (NativeAlarmProjector's constants would re-report
every ack and undo every shelve). MaxTimeShelved set to int.MaxValue ms at materialise — Create() left it 0
and every TimedShelve failed BadShelvingTimeOutOfRange. Wrong "SDK auto-fires (E2)" comments fixed.
#588 — with ACL on, GeneralModelChange and AuditWriteUpdate events no longer go to Server.ReportEvent
(ungated to every i=2253 item): ReportToServerObject delivers them per item, a model change narrowed to the
Affected nodes the session may Browse (dropped when none), an audit write only with Read on its source plus
the SDK audit/encryption guard. Flag off: unchanged.
#586 review: materialise raises only the new condition's monitored unit (was a full O(N) reconcile per
condition ⇒ O(N²) deploys under Lock).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Review of 75e559f6:
- OnConditionReportEvent now mirrors MonitoredNode2's AuditEventState guard (only while Server.Auditing,
over SignAndEncrypt / https-uabinary). Conditions report their Part 9 method audit events through
themselves, so the first cut queued AuditConditionAcknowledge (i=8944) etc. to every i=2253 item with
auditing off -- the removed folder->Server forward used to drop them.
- ReconcileAllEventsMonitoring keeps one AreEventsMonitored unit on every materialised condition while any
i=2253 item exists (called from SubscribeToAllEvents, materialise, removal, rebuild), so
ConditionState.EventsMonitored() is true on a server whose only subscribers are Server-object items.
A protected SubscribeToEvents override no-ops all-events items, which stops the SDK's AddRootNotifier
(unsubscribe: true bug) taking that unit back from conditions rebuilt under a live item.
Flag-off over-the-wire coverage: Acknowledge/AddComment with and without a folder co-subscriber (no audit
event, transitions once each), EventsMonitored on/after-rebuild/off, rebuild survival with one copy per
event, native + scripted conditions once each, refresh once per condition with RefreshStart/End; plus an
ACL-on native-vs-scripted notifier-folder decision. Falsified: removing the audit guard reds both audit
cases; removing the reconcile reds the monitored test.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Server-object (all-events) items bypassed the ACL entirely: the SDK attached them to every root
notifier's fan-out via SubscribeToAllEvents, and a root notifier with no direct subscriber forwarded
to the core Server object, which this node manager never sees.
- SubscribeToAllEvents override records the item and attaches it to nothing.
- Each condition's own OnReportEvent (fires once per event) queues the event to each recorded item
whose session holds Subscribe on the condition (RawPath) or any notifier folder it reports through
-- the per-folder path's question; otherwise mirrors MonitoredNode2.OnReportEvent.
- The root-notifier forward to the core Server object (OnReportEvent override) is dropped.
- ConditionRefresh on an i=2253 item is filtered the same way (once per condition).
- Revalidation is implicit: every event re-asks the authorizer (per-generation cache, key expiry and
revocation re-checked per call). Inert with Security:Acl:Enabled=false.
Over-the-wire tests: no-grant vs granted (positive control), created-before-deploy, revoke/restore by
deployment, key expiry with no tick, refresh, and flag-off. Falsified: disabling the filter, or
restoring the forward, turns the enforcement tests red.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
#562: direct event (alarm) monitored items are now tracked in a second map
(_eventItems, fed by the SubscribeToEvents override) and re-authorized by
RevalidateMonitoredItems on both triggers (DataPlaneAuthorityState.Changed and
AclRevalidationTimer). A denied item is detached via the SDK's own protected
SubscribeToEvents(unsubscribe: true) - never MonitoringMode.Disabled - and
re-attached on restore; a client delete of a stood-down item is absorbed.
Over-the-wire tests: revoke -> stop (positive control on a still-granted
notifier) -> restore -> resume; clock-driven expiry via the real timer with a
control session; delete-while-stood-down is not resurrected; unexpired
negative control. Server-object (all-events) subscriptions are documented as
outside the ACL (observed over the wire).
#563: DataPlaneAuthorizer.AuthorizeKeyUsable runs the FindUsableKey probe on
the WriteOperate and AlarmAck role gates regardless of Security:Acl:Enabled,
so an expired or deployment-disabled key loses its activation-time roles on
its open session. Anonymous (no key id) is unaffected; reads/subscriptions
stay ungated with the flag off. No new flag.
Docs: docs/security.md and the CLAUDE.md API-key/ACL section.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
AbCip: node-level Drivers:AbCip:DefaultTransport and the working pin
recipe. AbLegacy: default libplctag, Native opt-in, exact wire evidence.
Type-mismatch and Unsupported are refusals that never evict or degrade.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Native is the default, so a locally refused write must not look like a comms fault.
- AbLegacy: an AbTransportException with AbStatus.Unsupported (status-bit sub-element writes, PLC-5
non-N reads, all PLC-5 writes; libplctag's ErrorUnsupported maps here too) now returns
BadNotSupported on both the read and the write path (incl. the bit-RMW path). The tag handle is
not evicted and driver health is not touched. A native runtime whose init read is refused locally
is CACHED rather than disposed, so a poll of a refused tag does not re-create (and re-lease the
pooled connection) every cycle.
- AbCip: the same mapping on the write path and both read paths (single + whole-UDT).
- Uses the existing named BadNotSupported constants (StatusCodeParityTests guard).
- Tests (falsified by disabling the mapping, and separately the keep-on-refusal): status-bit
writes and PLC-5 read/write through the driver over the native runtime — BadNotSupported, zero
wire requests, one create, health unchanged; AbCip refused read/write via a refusing fake plus a
control that a non-refusal failure still evicts and degrades.
- Docs/CLAUDE.md: BadNotSupported; #571 known-gap wording removed.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
- I1: the upgrade restart itself moves every no-key AbCip/AbLegacy driver to Native (every process
start constructs drivers, incl. boot-from-cache). Docs + CLAUDE.md now say so and tell operators
to set "transport": "Libplctag" BEFORE upgrading (older builds ignore the unknown key).
- I2: AbLegacy release note leads with the status-bit write refusal (T4:0.DN, C5:0.CU, R6:0.EN, ...)
on defaulted SLC/MicroLogix drivers and the fallback. The real client status is
BadCommunicationError, not BadNotSupported (the driver's write catch-all maps the runtime's
AbTransportException like any transport failure, as #571 does for PLC-5), with handle eviction
and Degraded health; pinned by a driver-level test (no 0xAA write on the wire, with a positive
control). ST string byte order stated as unverified. Also corrects the docs' "T4:0/DN" examples:
the driver's address grammar is "T4:0.DN" ("/DN" is a malformed address).
- I3: DriverProbeHandshakeE2eTests' two AbCip probe tests are a [Theory] over Native and
Libplctag. Run against ab_server controllogix: all 4 rows pass with identical exact messages.
- M1: documented (docs + resolver XML doc): the AdminUI merges only the probed device, so a
mixed-family defaulted AbLegacy driver can probe Native while running Libplctag.
- M2: factory ParseEnum rejects undefined integer strings ("7") via Enum.IsDefined (both drivers).
- M3: AbCip driver-level bad value reads "AB CIP config in '<id>' has unknown Transport ..."; the
named constant is gone (tagName: null now means driver-level).
- M4: the explicit-Native limited-family Warning is once per instance, not per (re)init.
- Nit: EffectiveTransport docs note that injected factories bypass it.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Task 22 (spec section 8 P5), with the controller rulings that exclude PLC-5 and LogixPccc.
- Transport is nullable on both option records; null (key absent) means "use the default" and is
never serialized. Factory DTO parse: absent -> null; a bad value still fails via ParseEnum.
- One resolver per driver (AbCipTransportResolver / AbLegacyTransportResolver), used by the driver
constructor, the Test Connect probe and (through the driver) the CLI.
AbCip: null -> Native. AbLegacy: null -> Native unless any device is Plc5 or LogixPccc, then the
whole driver resolves to Libplctag. Explicit values always win; explicit Native with a
Plc5/LogixPccc device logs one Warning at init.
- Init log shows configured vs effective, e.g. transport=Native (default).
- AdminUI forms: "Default (...)" / "Native (managed)" / "libplctag (fallback)". Default omits the
key on save (reverses Task 19's explicit re-emit, which would have pinned every edited driver).
- CLIs: --transport default|native|libplctag, default "default" via the resolver, so
--plc-type Plc5 with no --transport runs libplctag.
- Tests pinned to Libplctag where a default-path runtime was created: found by instrumenting the
native factories with the current xUnit test name (one unit test, AbLegacyReinitConfigAdoption;
plus the two fixture-gated AbCip probe E2E tests whose exact messages were proven on libplctag).
TransportMatrix and the differential harness already pass explicit transports.
- Switch/resolver/CLI/form tests are falsified by mutating the resolver (default -> Libplctag,
exclusion removed, AbLegacy default -> Libplctag): each mutation turns them red.
- Replaces the inline role-name literal in AbCipDriverFactoryExtensions' ParseEnum call with a named
constant; RoleAuthorityGuardTests was already red on the base commit for that line.
- Docs: AbCip.md / AbLegacy.md default + release note; CLAUDE.md AB section.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Additive/wire-compatible releases across all three families; Directory.Packages.props
is the only pin surface (no csproj-level overrides, no lock files). Auth 0.3.0 adds
IApiKeyStoreMigrator and the optional Auth.ApiKeys.SqlServer package, neither needed
here. LocalDb 0.3.4's on-disk bookkeeping schema goes v2->v3 (upgrades in place on
open); 0.3.0 made the snapshot-resync flag per-table instead of per-database; 0.3.2
added validated Replication:KeepAlivePingDelay/KeepAlivePingTimeout config (defaults
60s/20s, left unset here) and a public LocalDbOplogGapException; 0.3.3 makes
RegisterReplicated throw LocalDbRegistrationException for a table that is the parent
of a row-mutating ON DELETE FK, which does not apply to this repo's four replicated
tables (no FKs). MxGateway.Client/Contracts 0.2.1 publishes the alarm snapshot_status
truncation frame and live alarm watch-list refresh; vendored the new nupkgs into
nuget-packages/ (the local-mxgw source MxGateway.* is exclusively package-source-mapped
to), matching the existing pattern from the 0.1.0->0.1.1 and 0.1.1->0.2.0 bumps.
Full solution builds clean (0 errors, 908 pre-existing analyzer warnings, unchanged
count). Full test suite: one failure, ClusterRoleInfoRoleLeaderCaseTests.
RoleLeaderChanged_for_driver_then_Driver_keeps_distinct_entries, reproduces
identically on the unbumped 0.2.1/0.2.1/0.2.0 tree under the same full-suite parallel
run (and passes standalone in isolation in 2s on the bumped tree) - a pre-existing
timing/parallelism flake, not a bump regression, so left as-is.
Updated CLAUDE.md and two docs (Configuration.md, operations/2026-07-20-localdb-pair-
replication.md) where they stated the old pin numbers and the now-superseded
per-database snapshot-resync behavior as current fact; dated historical records
(archreview snapshots, dated plan docs) were left untouched.
Claude-Session: https://claude.ai/code/session_01Wzg6NUCWHipSexLs2cgRFv
CLAUDE.md's "Browse needs ancestor visibility" bullet and docs/security.md's
ancestor-visibility bullet now name the topmost ancestor: the single shared
root folder (s=OtOpcUa, raw namespace) that both realm trees hang under, its
mapping onto the trie's zero-segment realm-root scope, the either-realm transit
(it fronts both trees), and the realm qualification.
Also corrects the adjacent security.md bullet that claimed "the two namespace
root folders remain visible" — there is one anchor, not one per namespace, and
what an ungranted session sees is the ns=0 Objects listing of it rather than a
browse into it. Flagged rather than silently left, since it contradicts the
mechanism the new bullet documents.
LocalDbSetup.OnReady registered all four replicated tables unconditionally on
every driver node, replicating or not. That was not merely wasteful (an
unreplicated node paid two INSERTs plus a json_object of the whole row inside
every write transaction, for an oplog nothing ever drained) — it made
baselineExistingRows unusable, so enabling replication on a long-running
default-OFF node converged on SUBSEQUENT WRITES ONLY and the rows already in the
file stayed invisible to the peer, silently and forever.
Registration now follows one predicate, LocalDbSetup.ReplicationIsConfigured:
LocalDb:Replication:PeerAddress OR :ApiKey non-blank. The OR is load-bearing —
only the dialling half of a pair sets PeerAddress, so keying on it alone would
strip capture from every passive node and the pair would converge in one
direction only. SyncListenPort deliberately does not join it: the passive
endpoint is fail-closed on the key, so a port with no ApiKey syncs with nobody.
configured => RegisterReplicated(table, baselineExistingRows: true), per
table, in the unchanged load-bearing DDL->register order
unconfigured => DeregisterReplicated(table), an idempotent boot self-heal that
drops the capture triggers an earlier build left behind and
prunes that table's oplog/row-version rows. One Information
line, only when it actually cleaned something.
PAIR SYMMETRY: the sync handshake compares registered-table digests fail-closed,
so flipping either key is a both-nodes-together change — set it on both and
restart both. Documented in CLAUDE.md, docs/Configuration.md and the replication
runbook (which claimed "this repo never calls DeregisterReplicated").
Verified against the pinned LocalDb 0.2.1: RegisterReplicated(string, bool) and
DeregisterReplicated(string) -> bool both exist; baselining seeds
__localdb_row_version at HLC 0 and the deregister return value distinguishes
"cleaned" from "nothing to clean". No pin bump needed.
Tests: LocalDbSetupTests gains pins for both branches (configured via
PeerAddress alone and via ApiKey alone; unconfigured => no tables, zero
triggers; late-enable => an HLC-0 baseline ledger entry; the log line fires once
and only when something was cleaned) — four went red against the old
unconditional code, and all four go red again under a constant-true predicate.
LocalDbWiringTests splits its exact-set pin into a replicating graph and a
default-OFF one. LocalDbPairHarness and AlarmSfLegacyMigratorTests built their
database from LocalDb:Path alone while configuring the replication engine from a
second config — production reads one configuration, so the two are now aligned.
Host.IntegrationTests: 224 passed, 0 failed, 5 skipped.
CLAUDE.md — a new "How a deploy reaches that address space" block under the v3
Batch 4 address-space section. The blanket ChangedRawTags/ChangedUnsReferenceVariables
=> Rebuild rule is gone, so the doc now names both halves explicitly: what still
rebuilds (alarm-bearing raw-tag deltas, raw-tag identity changes, raw container
renames/removals, changed equipment/alarms) and what is now surgical (non-alarm
attribute deltas, every UnsReferenceDelta). Also records the three semantics that
are easy to break by accident: mirror repair rides UnsReferenceVariable's inherited
backing fields (omitting a newly-inherited field silently reopens the staleness),
a degraded non-rebuild apply deliberately does not advance _lastApplied, and the
raw-realm announce gap is no longer neutral now that surgical adds keep
subscriptions alive.
CLAUDE.md ConfigSource — the RevisionHash == SHA-256(artifact bytes) paragraph is
still true and stays; it gains the clause that the blob is gzip-compressed since
2026-08-15 and the hash covers the COMPRESSED bytes, which is precisely why the
fetcher, cache and serve path were untouched.
docs/Uns.md + docs/Raw.md — one section each on what a deploy now tears down, from
the authoring surface's point of view, including the mirror repair semantics
(identity-preserving backing edits update in place and keep their MonitoredItems;
re-points and effective-name edits remove+recreate that one node).
docs/Configuration.md — "Artifact compression (2026-08-15)" under ConfigSource/
ConfigServe: gzip inside the hashed bytes, magic-sniffed on read, pre-upgrade
uncompressed rows still parse, plus the two operational facts an operator needs.
The first deploy after upgrade re-hashes every revision (same config, new bytes,
new RevisionHash) so it is a one-time full re-dispatch rather than a no-op; and
rollout is ordered — nodes must run artifact-sniffing code BEFORE central starts
emitting compressed blobs, since a pre-upgrade node handed a gzip artifact fails
the apply and keeps last-known-good until it is upgraded.
Plan doc — a "Follow-ups (recorded, not implemented)" section covering the seven
items surfaced during implementation and review: the raw-realm announce gap, the
RemoveEquipmentSubtree dangling-inverse-edge exposure, the missing end-to-end test
for a degraded remove-bearing apply's ratcheted rebuild, the scripted-alarm
thin-plan-ids-against-rich-rows blindness, the absent monotonic dispatch sequence,
LocalDb store ordering under concurrent StoreAsync, and threading the in-hand blob
through RestoreApplied.
tasks.json — all nine tasks completed.
Docs only; no source changes.
Closes the pin skew against ScadaBridge (which already pins 0.2.1). Pure pin
bump — no code change was required:
- `RegisterReplicated(string)` still delegates to `baselineExistingRows: false`,
so `LocalDbSetup.OnReady` is byte-identical in behaviour.
- Everything 0.2.x added is additive (`DeregisterReplicated`, `MaxBatchBytes`,
the `baselineExistingRows` overload); none of it is called here.
- The wire protocol is unchanged (0.1.x peers still sync) and the on-disk
bookkeeping schema v1 -> v2 upgrades in place on open, so no rig, volume or
operator action is needed.
The one thing that DID need reconciling is documentation. 0.2.0 made batching
byte-budgeted: `MaxBatchBytes` (2 MB default, under the 4 MB gRPC cap) now
bounds the wire message, and `MaxBatchSize` is demoted to a secondary row cap
plus the DB read-page size. The rig's `MaxBatchSize = 16` was chosen as a
hand-computed byte proxy ("row-count-only against gRPC's 4 MB cap"), and that
rationale is now wrong everywhere it appeared — appsettings, Configuration.md,
the pair-replication runbook, and both docker-dev site-a services.
The VALUE stays 16, for a different and still-real reason: OplogStore /
SnapshotStreamer materialise a whole read page in memory *before* the
byte-budget split runs, so 16 x ~171 KB artifact chunk rows bounds that
transient allocation at ~2.7 MB. Documented as such, with `MaxBatchBytes`
added to both config tables.
CLAUDE.md records the pin and two deliberate non-adoptions: this repo does not
take ScadaBridge's conditional-registration + boot-time `DeregisterReplicated`
self-heal (OnReady registers all four tables unconditionally, and
LocalDbSetupTests pins that set), and consequently does not use
`baselineExistingRows` either — so enabling replication on a long-running
default-OFF node converges on subsequent writes, not on rows already in the
file. Both are behaviour changes, out of scope for a pin bump.
Verification: solution builds clean; full suite run (57 projects). All 66
LocalDb tests in Host.IntegrationTests green. One unrelated failure,
RoslynVirtualTagEvaluatorTests.Evaluate_racing_ClearCompiledScripts_never_
fails_with_disposed — a Roslyn evaluator disposal race with no LocalDb in its
path, whose own comment documents that its "at most one clear per evaluation"
assumption depends on thread-pool pressure. It passed 3/3 re-runs in isolation
and fails only under full-solution parallel load; pre-existing test-design
fragility, not a regression from this bump.
#542 filed FleetStatusBroadcaster as blind to site nodes since the Phase 6 per-cluster mesh
split: it derives "the fleet" from central's own Akka gossip ring, which after the split is
just central's own pair. Scoping the cross-mesh rewrite found the deeper truth — the whole
chain is dead end to end, so this deletes it rather than reviving it.
- Zero consumers. FleetStatusBroadcaster published FleetStatusChanged on the DPS
"fleet-status" topic; FleetStatusSignalRBridge forwarded it to FleetStatusHub
("/hubs/fleet-status"); nothing anywhere connected. The AdminUI bans self-connecting
HubConnections, no browser JS subscribes, and /fleet (Fleet.razor) reads
NodeDeploymentState from the ConfigDb plus IClusterRoleInfo on a 5 s poll instead.
- Its second input never existed. DriverHostStatusHeartbeat had no production sender —
only the broadcaster's own test told it — so CurrentRevision was always null and the
30 s staleness -> Degraded flip could never fire.
- Post-Phase-6 DPS is pair-local, so even the central-only view it did produce was an
accident of topology rather than a design.
The fleet visibility that actually works is unchanged and already cross-mesh-correct:
/fleet over NodeDeploymentState, and the four Phase 5 per-node telemetry dials.
Precedent for deleting inert machinery rather than activating it: #522 (driver stability
tiers), #507 (discovered-node injection), #525 (GetMemoryFootprint).
Deleted: FleetStatusBroadcaster (+ its nested DriverHostStatusHeartbeat), its admin
singleton registration and FleetStatusBroadcasterKey, FleetStatusSignalRBridge and its
key, FleetStatusHub and its route mapping, and the FleetStatusChanged / FleetNodeStatus /
FleetNodeHealth messages. Both Fleet/ directories survive — ControlPlane keeps
ClusterNodeAddressReconciler, Commons keeps GetDiagnostics. Docs swept where they claimed
a fleet-status channel exists (Telemetry.md's Phase 5 non-migrated channels,
Configuration.md, CLAUDE.md); completed-program plan docs under docs/plans and docs/v2 are
left as the historical record they are.
Build 0 errors, 868 warnings — byte-identical to master's rebuild baseline.
ControlPlane.Tests 231/231, AdminUI.Tests 1109/1109.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Task 3.2 of the ClusterClient->gRPC migration. Sweeps the docs that still
described the Akka ClusterClient transport as live, following its Phase 3
deletion (this branch): MeshTransport:Mode is now Dps (pair-local DPS) or
Grpc (cross-mesh unary calls) only, ClusterClient fails validation loudly,
CentralContactPoints is deleted (not superseded), and the split-topology
validator requires exactly Grpc rather than ClusterClient.
- docs/Configuration.md: rewrite the MeshTransport section — table,
dark-switch/PSK-seeding prose, listener sharing (:4056 driver-role,
:4055 admin-role), remove the "shipped dark, cutover pending" framing.
- docs/Redundancy.md: replace the Dps/ClusterClient transport table with
Dps/Grpc, fix the Phase 6 CentralCommunicationActor + validator bullets.
- docs/Telemetry.md: fix stale ClusterClient cross-refs, add a note that
the driver-role telemetry listener also hosts NodeCommandService.
- docs/v2/config-db-schema.md: one-line "(retired: ...)" annotation on the
historical AkkaPort/GrpcPort note — left as a historical record otherwise.
- CLAUDE.md: rewrite "## Mesh command transport (MeshTransport)" for the
post-deletion world (no ClusterClient/receptionist, per-node gRPC dial,
fire-and-forget semantics unchanged) and fix two Phase 6 bullets.
deferment.md has no mesh-transport mentions; left untouched.
docs-only change; `dotnet build ZB.MOM.WW.OtOpcUa.slnx` stays green (0
errors) after the edit.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Ran on the docker-dev central pair against the local HistorianGateway container talking to the real
AVEVA Historian on wonder-sql-vd03 (VPN up), with the host image rebuilt at e7fbe977.
Two things about the setup matter more than the results:
The value source had to be DRIVEN. Both the project Modbus fixture and oee-modbus-sim hold static
registers, and a static source produces no dependency-mux fan-out at all — the recorder captures
once at deploy and then goes quiet, which reads exactly like a broken recorder. Hand-written
Modbus FC6 writes put the value stream under the gate's control, which is what makes the rate
measurements below mean anything. The project fixture was also brought back up on the docker host
and MAIN-modbus-dev repointed to it, undoing the temporary :1502 repoint the read-back leg made.
The poison tag is the real shape, not a synthetic fault: GATE491-B's historianTagname was CHANGED
on an EXISTING tag, which Issue 5 guarantees is never provisioned, so its writes fail permanently
against a historian that has no such tag. That is how #538 was reached in the first place.
#537 — both nodes up: the Secondary logged the suppression line once and its outbox file froze
while the Primary's kept growing; 6 value changes x 2 tags gave WriteLiveValues{ok} +12 exactly,
where the pre-fix measurement was twice that. Failover: stopping the Primary made the survivor log
"now the recording Primary ... resuming value capture" and pick up every subsequent change. Return:
the restarted node was demoted, logged suppression, and the rate stayed at one node.
#538 — the quarantine fired with the tag named; 5 value changes over 30s gave ok +5 and error +5,
exactly one write per tag per change against a 2s drain interval.
The decisive leg is the IDLE one: 40s with no value changes left both counters unchanged and the
outbox byte size flat. The driven-rate legs only show a rate that happens to look right; a pre-fix
build re-writes its queued healthy entries on every drain pass whether or not anything new arrives,
so its counters climb during a quiet window with no input at all. Flat counters over idle is the
observation the bug cannot produce.
Incidental finding, recorded against Issue 5 rather than treated as a fix: a node RESTART cures an
unprovisioned tag. central-2's boot logged provisioning (dispatched=2, requested=2, ensured=2,
failed=0) and the poison tag started succeeding, because a full address-space rebuild treats every
historized tag as added and re-runs EnsureTags — where the incremental deploy that introduced the
renamed tag did not. So an operator who restarts a node while diagnosing "my tag isn't historizing"
makes the symptom disappear without learning the cause.
Docs only; no production code changed.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Both defects were found by #491's live gate against a real HistorianGateway and filed rather than
fixed at the time, because each one's fix is a design decision rather than a patch.
#537 — both nodes of a redundant pair captured every value, so the historian received duplicate
live samples for every historized tag (measured: the gateway's write counter advanced twice per
value and halved when one central was stopped).
The recorder now reads a capture gate per value, fed from the new
PrimaryGatePolicy.ShouldRecordHistorizedValues through IHistorizationRoleView — a third verdict on
the same RedundancyRoleView object, registered unconditionally on driver nodes rather than
alongside the alarm section's IRedundancyRoleView (which only exists when that section is enabled;
piggybacking is the #511 registration accident).
Two things are deliberate and easy to get backwards:
* It gates the CAPTURE, not the drain — the opposite of the alarm sink. That queue is in LocalDb
and replicates, so the Secondary already holds the Primary's rows. The historization outbox is
a node-local FasterLog replicating to nobody, so a drain-only gate would leave the standby
appending forever to a queue no one empties: unbounded disk, then a flood of stale duplicates
the moment it is promoted. Entries a node already captured are still drained by that node.
* The peer it stands down for must RECORD THE SAME REFS, not merely hold the same rows — a
Primary in this node's own per-cluster mesh. A node carrying no cluster-scoped role cannot
establish that and never stands down: under a fleet-wide election the elected Primary is
usually running a different cluster's configuration, and deferring to it would end this
cluster's historization silently, with the drain reporting success on an empty outbox forever.
That is the scope mismatch that once suspended the alarm drain on every node of the rig.
An unknown role captures, matching the alarm drain — a duplicate sample is recoverable, a hole in
the trend is not.
#538 — the drain writes every per-tag group and then acks all-or-nothing (deliberate: RemoveAsync
truncates THROUGH an id, so partial acking could drop older unwritten entries). One permanently
failing tag therefore pinned the FIFO head while every healthy tag in the batch was re-submitted
to the historian on every pass, indefinitely — duplicate samples and an outbox that never
truncated, observed live at 8.3 MB from a single unprovisionable tag.
The drain now tracks consecutive failures per historian tag and abandons that tag's entries once
it exceeds MaxWriteAttempts (new, default 5), so the head advances. The counter only advances on a
pass where some OTHER tag wrote successfully: that is what separates "this tag is poison" from
"the backend is down". A count-only quarantine looks identical on the poison-tag case and silently
discards the whole backlog during an ordinary gateway restart, turning a duplicate-samples bug
into a lost-history one. Abandonment is also deferred until every failing tag is over budget, so
nothing is discarded on a pass that could not have advanced the head anyway. Each abandonment logs
a Warning naming the tag; the total rides RecorderStatus.AbandonedEntries.
Capacity also gains a non-zero default (500,000 entries) as the disk backstop for the one case the
quarantine deliberately cannot resolve — a historian refusing every write, where retaining is
correct. Set 0 to restore unbounded retention.
Every new test was falsified by reverting the production change it guards, and each one reddens
alone:
* drop the backend-reachable evidence -> only A_backend_that_fails_every_tag_abandons_nothing
* never abandon -> only A_permanently_failing_tag_is_abandoned_so_healthy_tags_stop_being_rewritten
* latch the capture gate at construction -> only The_capture_gate_is_re_read_per_value...
* treat any Primary as a scope peer -> only the two scope-mismatch cases
Full suite green. (Four OpcUaClient.Browser failures during the run were the opc-plc fixture being
down on the shared Docker host; brought it back up and they pass.)
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
The 2026-08-01 run closed every leg except read-back, which no reachable historian could
serve that day (the LAN instance accepts writes but serves no reads under a licence tag-count
violation; the licensed one needed the VPN). With the VPN up the run was repeated against
wonder-sql-vd03 and the remaining legs pass:
- EnsureTags provisioning: ensured=2, failed=0 — the path the LAN box could not exercise at
all (it is RemoteGrpc-only and that CAP was not listening).
- Read-back: two Modbus registers carrying DISTINCT NON-ZERO constants were historized, and
each read back as its own value (HR20 -> 8224, HR3 -> 120, quality 192). A first attempt
read back 0, which proves nothing — the same trap #536 recorded, so the proof was rebuilt
on values a crossed or fabricated answer could not fake.
Found during the run and FILED rather than fixed:
- #538 — one permanently-failing tag re-writes every healthy tag on every drain pass, forever.
DrainOnceAsync writes EVERY per-tag group and then acks all-or-nothing, so a tag whose
historian tag does not exist stops the FIFO head advancing while the healthy tags' values
are re-submitted every drain interval; the outbox never truncates (Capacity defaults to 0 =
unbounded, observed at 8.3 MB). The all-or-nothing ack is deliberate — RemoveAsync truncates
THROUGH an id, so partial acking could drop older unwritten entries — so the fix changes the
outbox contract and does not belong in this commit.
Also re-confirmed live: EnsureTags provisioning fires only for newly-ADDED historized tags, so
flipping an existing tag's historianTagname provisions nothing (Issue 5 of
2026-06-27-historian-gateway-integration-issues.md).
No production code changes — the recorder fix shipped in ad744734.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Ran #491's live gate against a real ZB.MOM.WW.HistorianGateway talking to a real AVEVA
Historian, and it found a production-fatal defect on the first value it ever saw.
ContinuousHistorizationRecorder.OnValueChangedAsync touched `Self` after awaiting
IHistorizationOutbox.AppendAsync(...).ConfigureAwait(false). ConfigureAwait(false) discards
Akka's ActorTaskScheduler synchronization context, so `Self` — which reads ActorBase.Context —
threw NotSupportedException. The throw escaped into supervision, which RESTARTED the actor;
because production seeds an EMPTY ref set (the deployed set arrives later via
UpdateHistorizedRefs), the restart silently discarded every historized ref. Net effect: one
value captured, then nothing, with no further error to show for it. Fixed by capturing
`IActorRef self = Self` before the await — the idiom the GetStatus handler already applied to
Sender.
Thirteen unit tests missed it because InMemoryOutbox.AppendAsync returned
ValueTask.CompletedTask: a completed ValueTask never yields, so the continuation ran inline
with the context still installed. The production FasterLog outbox fsyncs per entry and does
yield. The first regression test was itself vacuous — `await Task.Yield()` posts back to the
CAPTURED context, so the append still completed on the actor thread and reverting the fix left
the test green. The fake must complete on a POOL thread (Task.Delay(1).ConfigureAwait(false)),
as a real fsync does; with that, reverting the fix reddens exactly one test.
The rig could not run this gate at all: docker-dev carried ServerHistorian__* but no
ContinuousHistorization__* keys, so the recorder could never spawn. Both centrals now carry
them (default OFF — the recorder needs ContinuousHistorization:Enabled AND
ServerHistorian:Enabled), with the outbox on the per-node named volume so it survives a
container recreate. ServerHistorian__UseTls was hard-coded true and is now parameterized; a
plaintext h2c gateway was unreachable without it.
Gate results: outbox append, drain, WriteLiveValues accepted by a real historian, and restart
convergence (capture resumes after a restart with no new deployment) all pass. The read-back
hop is NOT proven — the LAN historian accepts writes but serves no reads under a "License
violation: Database tag count exceeds license tag count", and the licensed corporate one needs
the VPN. EnsureTags provisioning is likewise unexercised there (RemoteGrpc-only, and that CAP
is not listening). #491 stays open for the read-back hop alone.
Also recorded, not fixed: the recorder has NO Primary gate, unlike the alarm store-and-forward
sink, so both nodes of a redundant pair send duplicate live values. Gating it would introduce a
failover window with no capture — that is a design decision, not a cleanup.
Evidence: docs/plans/2026-08-01-continuous-historization-live-gate.md
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
The last stage of #520. Enforcement landed in stages 5-6; this closes the
authoring end, where a grant could be saved, deployed and sealed green while
resolving to nothing.
Scope kind and scope target are now chosen TOGETHER from a picker over the
cluster's raw and UNS trees (AclScopePickerModal -> AclScopeTree, backed by
IAclScopeService). They were previously an unrelated dropdown and a free-text
box, so "RawDevice + a TagId" was a well-formed row. Because the trees are
rooted at the grant's own cluster and each node supplies its own kind,
NodeAclScopeCrossCluster, NodeAclScopeIdUnexpected, NodeAclScopeIdMissing and
NodeAclScopeKindRetired all become unauthorable by construction.
CheckScopeAsync mirrors DraftValidator.ValidateNodeAclScopes rule for rule and
blocks the save when a scope resolves to nothing - re-checked on submit, not
trusted from form load, because a row whose entity was deleted since is exactly
where the operator never touches the picker. The list page marks such rows
unresolved with the reason. The principal field suggests known principals and
reports how many live keys reach the one typed; reaching nobody warns rather
than blocks, since authoring ahead of minting is legitimate.
Deliberate departures from the plan line, all recorded in the design doc:
- The banners are reshaped, not removed. "Not enforced" stopped being true at
stage 6, but enforcement is still a per-node flag the AdminUI cannot see, so
a narrow grant really does confer nothing on a default-configured node.
- A Cluster row produces a NULL ScopeId. The deploy gate rejects one that
carries an id, so filling in the cluster id would author undeployable rows.
- AclScopeTree is a new component rather than a third mode on RawTree, which
already carries the /raw editing tree and the equipment reference picker.
- UNS tags are not offered: a UNS signal is a UnsTagReference, and a Tag grant
already covers the raw node and every UNS projection of it.
Also fixed: the ACL editor had a DataAnnotationsValidator and no
ValidationSummary, so an invalid NodeAclId made Create a silent no-op.
Tests: 30 in AdminUI.Tests/Acls, falsified with six production mutations -
each turned exactly the intended tests red and no others.
Live gate PASSED on docker-dev, 11 legs; both central containers asserted on
the same image by container id. The pair that matters is legs 8 and 11: the
page and the deploy gate reject the same row with the same reason. Legs 5 and
9 are the positive controls - without them the gate passes against a page that
refuses everything. Not covered: UNS-realm scope selection (SITE-A has no UNS
areas). Rig restored to zero ACL rows.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Completes data-plane enforcement behind Security:Acl:Enabled. Read (every
attribute, not just Value), Browse, TranslateBrowsePath, CreateMonitoredItems,
SubscribeToEvents and SetMonitoringMode now consult scoped NodeAcl grants.
With the flag off nothing changes.
Four things went beyond the stage as written, each because leaving it out
would have made the feature look like it worked:
- Browse needed ancestor visibility. The trie inherits downward, and a client
reaches a deep node by browsing the nodes above it, so without this every
grant narrower than the cluster was authored, shipped and unreachable. It is
a separate non-inheriting marker, NOT a synthesized Browse grant on each
ancestor: the additive downward walk would carry such a grant from Plant onto
Plant/Siemens/dev9 and every other sibling subtree. Pinned by a test that
goes through the evaluator — probing the trie helper directly passes against
the broken design, because a sibling is absent from the trie either way.
- Revocation had to reach LIVE subscriptions. Every other operation re-asks per
request; a subscription is authorized once and then streams, and streaming is
how machine consumers actually read this server — so stopping only the
re-asking operations would have made revocation close to cosmetic.
IDataPlaneAuthoritySource gained a Changed event.
A denied item is queued BadUserAccessDenied and then DETACHED from its
MonitoredNode2 fan-out, not set MonitoringMode.Disabled. Disabling discards
the item's queued notifications, so the client is silenced without ever being
told why. The over-the-wire test caught this; the unit-level path reports the
item as stood down either way. Detaching is also reversible, so a restored
grant resumes the feed with no client action.
- The decision cache holds the grant WALK, not the decision. Grants and the
trie are fixed for a generation; a key's ExpiresUtc passes with the clock, so
caching the whole decision would let an expired key keep working until the
next deploy — the stage-5 open-session hole in a subtler form.
- TranslateBrowsePath is gated, without which browse filtering is decorative: a
client refused a browse asks TranslateBrowsePathsToNodeIds for the same path.
Found en route: UserAuthorizationState's staleness ceiling couples two clocks.
MembershipResolvedUtc is stamped from DataPlaneAuthorizer's TimeProvider while
IsStale is evaluated against TriePermissionEvaluator's; they are the same
instance in production and diverge the moment anything injects a test clock, at
which point the evaluator denies EVERYTHING with a verdict indistinguishable
from "no grant matched". Stage 5's claim that these predicates cannot fire was
accidentally true; they are now disabled explicitly.
Also fixed: ContinuationPoint.NodeToBrowse is the manager HANDLE, not a
NodeState. The declared type is object, so the wrong cast compiled cleanly and
silently yielded null — skipping the browse source gate entirely.
Falsified: disabling the transit marking reddens exactly the three tests that
assert transit works; implementing transit as an ancestor grant reddens the
sibling-leak test; skipping the fan-out detach reddens the revocation test.
Full solution suite green.
The stage 8 live gate has NOT run yet.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
`Security:Acl:Enabled` (default false) makes scoped `NodeAcl` grants decide
data-plane authorization at the three seams that have one, and gives HistoryRead
a check it never had. Read, Browse and Subscribe stay ungated — stage 6.
The ACL decision REPLACES the server-wide role check rather than chaining with
it. Chained, a grant would have to confer the role as well as pass the scope
check, and only Cluster-scoped grants confer a role — so every narrow grant
would have stayed inert and the flag would have looked inert too. The
load-bearing test is an ALLOW: a session carrying no roles at all writing
successfully on the strength of its grant. A deny-only test passes against a
build where the ACL is never consulted, because the role gate denies too.
Found en route: the principal never reached the session. The authenticator
passed it as OpcUaUserAuthResult.DisplayName, but RoleCarryingUserIdentity
derives DisplayName from the presented token — for a key-bearing client that is
the static username `apikey`, identical fleet-wide. So the trie's matching key
was dropped at the identity boundary, and the audit ClientUserId on a rejected
write recorded `apikey` for every consuming system. Principal and ApiKeyId are
now carried explicitly.
Beyond the stage as sketched:
- Revocation reaches an OPEN session. Every decision re-checks the
authenticating key against the current snapshot; without it a key revoked by a
deployment kept its authority until the client chose to reconnect, which for a
machine-to-machine consumer can be indefinitely.
- UserAuthorizationState is built per CALL, not per session. The per-session
shape exists so an LDAP session can re-resolve membership that changed out of
band; nothing can change out of band under API keys, so the generation stamp is
the whole invalidation signal and a per-call record picks it up for free.
IsStale/NeedsRefresh therefore cannot fire here and are documented as vestigial.
- The trie is built inside DataPlaneAuthorityState.Apply, under the same
generation as the snapshot, rather than by a second wiring at the publish
actor. An uninstalled trie denies everything — indistinguishable from working
enforcement. Deleting the install turns 13 of the 22 new tests red.
- The deploy log reports what the BUILD dropped instead of re-deriving it. The
old count agreed with the builder on one of its four drop reasons; a rule with
a blank ScopeId, a retired scope kind, or a path not starting at a realm
vanished with the log reading zero.
Both IApiKeyStore implementations widened to the full grant set. The narrowing
was redundant — ApiKeyInterimRoleMapper applies the same filter itself — and
became a trap once scope was enforced. Pinned by asserting the roles are
unchanged, with the two grants carrying different flags so it cannot pass by
accident.
Falsified by mutation: enforcement never engaging, the write seam ignoring the
ACL, the realm dropped from the scope, the revocation check removed, and the
trie never installed each turn the intended tests red and no others.
Full solution suite green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
A driver-only node has had no ConfigDb since per-cluster mesh Phase 4, so it
registered NullOpcUaUserAuthenticator and denied every UserName token. Its keys
now ride the deployment artifact instead.
- ConfigComposer snapshots ApiKeys alongside the NodeAcls it already carried.
Only the peppered HMAC ships, never a secret, and the pepper is node config
that is deliberately not in the artifact.
- DeploymentArtifact.ParseDataPlaneAuthority decodes both node-side. Grants are
scoped to the node's own cluster; keys are not, because ApiKey carries no
ClusterId — a key is a fleet-wide credential and the ACL rows are what bind it.
- OpcUaPublishActor installs them on DataPlaneAuthorityState on every apply,
BEFORE its plan.IsEmpty early-return: an artifact that only revokes a key
produces an empty address-space plan, so feeding after that return would leave
the revoked key working until an unrelated structural change redeployed.
- ArtifactApiKeyStore serves them to session activation; a node with a ConfigDb
keeps ConfigDbApiKeyStore, whose per-request read is what makes revocation
immediate there.
- OtOpcUaNodeManager gains DataPlaneAuthority — the cluster identity blocker #4
named. NodeScope.ClusterId is required and the evaluator fails closed on a
mismatch, so until now no scope it built could have evaluated. Nothing reads it
until stages 5/6.
The generation is a node-local COUNTER driven by the RevisionHash, not derived
from its bits. PermissionTrieCache treats the highest GenerationId as current, so
a generation taken from hash bytes would install a lower number about half the
time and the cache would go on serving the previous deployment's ACLs, silently.
Sessions are not stamped with it yet — that belongs with the first seam that
reads a UserAuthorizationState, in stage 5, rather than being built now and
consulted by nothing.
Live-gated on docker-dev against site-a-1 (driver-only, no ConfigDb): key
authenticates, wrong secret denied, both halves of the revocation asymmetry
observed, and a SITE-A grant confirmed not to reach site-b-1. Key minted with an
independent Python HMAC so the verifier was checked against something other than
itself. Rig left clean.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Anonymous sessions used to short-circuit in HandleImpersonation — "let the SDK's
default validation stand" — so they always connected and always carried no roles.
Now admission and authority are separated:
admission -> OpcUa:AllowAnonymous, the HOST's decision (it owns token policies)
authority -> the authenticator, resolving the reserved principal "anonymous"
through the SAME grant path a key's principal goes through
So anonymous can be granted WRITE, not merely tolerated as read-only, and it
arrives at the downstream gates as the same RoleCarryingUserIdentity a key
produces — which is the real payoff: no gate carries an anonymous-shaped branch,
so there is no bypass path to get wrong.
AllowAnonymous defaults to TRUE on purpose. Defaulting it false would silently
deny every client without a key the moment this deploys, including ScadaBridge's
Data Connection Layer in another repo. Making keys mandatory is a rollout
decision, not an upgrade side effect. When false, the Anonymous token policy is
withheld so a client is refused at DISCOVERY rather than only at activation, and
activation is rejected independently in case a client attempts a token type the
endpoint never advertised.
Two fail-closed choices, both falsified:
- A failure to read anonymous's grants DENIES rather than admitting with no
roles. Admitting is the pre-#520 behaviour, and defaulting to it exactly when
the server cannot determine what anonymous may do stops being harmless once
reads are gated. Making it admit reds Anonymous_denies_when_grants_cannot_be_read.
- A key may not authenticate as the reserved "anonymous" principal, or a
credentialled client silently inherits whatever anonymous was granted.
AuthenticateAnonymousAsync is a DEFAULT interface implementation returning
admit-with-no-roles, i.e. exactly the pre-#520 behaviour, so
NullOpcUaUserAuthenticator and every test double keep working rather than being
forced to reimplement it — important because driver-only nodes run the Null one
and would otherwise black out. Note a default interface method is only reachable
through the interface-typed reference, which is how OpcUaApplicationHost holds it.
An existing test asserted the old fall-through and correctly went red; it is
rewritten to pin the new contract rather than deleted, with the reason recorded.
OpcUaServer 366, Host 42 green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Stage 2b. LdapOpcUaUserAuthenticator was left registered nowhere by stage 2a;
this removes it, its three test files, and the config surface that existed only
to serve it.
Deleted: LdapOpcUaUserAuthenticator, LdapOpcUaUserAuthenticatorTests,
LdapAuthResilienceTests, LdapGlauthOutageLiveTests (19 tests).
Also removed four Security:Ldap keys — AuthTimeoutMs,
MaxConcurrentAuthentications, OutageFailureThreshold, OutageCooldownSeconds —
plus their validator rules and binding/validator test coverage. Verified first
that the ONLY consumers were the deleted class and the validator rules asserting
them; none appear in any checked-in appsettings. Leaving them would have created
exactly the dead-config surface #521/#524/#525 were filed to clear up.
Worth stating why that resilience machinery does not need relocating rather than
just deleting. It existed because the OPC UA SDK raises impersonation
SYNCHRONOUSLY under a SessionManager-wide event lock, so a hung LDAP bind there
serialized every session activation server-wide — Anonymous and X509 included.
Key verification is one indexed lookup plus one HMAC with no network I/O, so the
hazard the wall-clock bound, concurrency cap and outage circuit guarded against
no longer exists on that path. LDAP now runs only on AdminUI login, an ordinary
async request path where a slow directory affects the one request making it.
ConnectionTimeoutMs is RETAINED: it is projected into the shared library and is
not authenticator-specific.
Docs corrected rather than left to rot:
- CLAUDE.md's "LDAP Authentication" section described the deleted file as the
data-plane authenticator. Rewritten, including a warning not to "restore" it on
seeing the LDAP wiring that AdminUI legitimately still needs.
- infra/README §5b (the GLAuth outage live gate) marked RETIRED with the reason,
rather than silently dropped — the harness GLAuth is still used by
OTOPCUA_HARNESS_USE_LDAP=1 and is unaffected.
- docs/security.md's five-key resilience table cut to the one surviving key, and
a new API-key identity section added.
Historical archreview/ records are left as-is; they are dated accounts.
Host.IntegrationTests 211 tests (was 230, i.e. exactly the 19 removed) with the
same 3 pre-existing failures verified against clean master. Host 37, Security 89.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
DriverInstanceActor.DisconnectObserved had no production sender. The only paths
into Reconnecting were a failed *initial* connect and an operator's Force
Reconnect, so a driver that connected and then lost its device stayed Connected
indefinitely: publishing Bad for every tag, never re-initialising, and NOT
recovering when the device returned. PublishHealthSnapshot read GetHealth() for
reporting only and never acted on a driver reporting itself unhealthy.
Both comms-loss quality paths -- #477's native-condition annotation and #481's
scripted-alarm input fan-out -- hang off ConnectivityChanged, so both were
correctly wired to a signal that never fired.
The Connected health tick now evaluates a watchdog and self-sends
DisconnectObserved when the driver reports a non-Healthy state AND its
LastSuccessfulRead has been frozen for DefaultCommsLossTimeout (90s).
The conjunction is the design, and the issue's own proposal -- "non-Healthy for
N consecutive polls" -- is wrong. Degraded means *connected but some data is
bad*; several drivers write health per tag inside the read loop
(ModbusDriver.ReadAsync), so one unreadable address latches Degraded forever
against a perfectly reachable device, and a state-only watchdog would reconnect
that driver every 90s indefinitely -- worse than the bug. LastSuccessfulRead is
the honest evidence: preserved across Degraded/Faulted writes, advanced by any
success. The state check remains the second conjunct so an idle Healthy driver
with nothing subscribed is not declared dead for not reading.
Verified across all 12 drivers that a sustained failure does reach a non-Healthy
state -- nine poll drivers via Degraded, MQTT via live connection state, Galaxy
via its reconnect supervisor. CalculationDriver reports Healthy while running
and so can never trip, which is correct: it has no transport.
Also: the Reconnecting retry now calls ShutdownAsync before InitializeAsync.
ModbusDriver (_transport) and MqttDriver (_connection) overwrite an
IAsyncDisposable field inside InitializeAsync without disposing the previous
one, so re-initialising without a teardown leaks a socket per attempt -- latent
while only a manual Force Reconnect reached this path, but the watchdog makes it
automatic and unbounded (~360 leaks for an hour-long outage at the 10s retry).
Tests: 8 new in DriverInstanceActorCommsLossWatchdogTests, each absence
assertion carrying a positive control (a proven count of processed health-poll
ticks). Falsified five ways -- dropping either conjunct reds exactly the test
written for it, and removing the teardown, the null baseline, or the watchdog
call each reds its own guard.
Live gate PASSED on docker-dev (Modbus + otopcua-pymodbus-standard), run twice:
- fixture stopped -> watchdog fires at ~108s, "declaring comms lost"
- fan-out observed at Debug: "pushed BadNoCommunication for 2 mux ref(s)"
(#481 Layer 4) and the #477 native-condition annotation on a driver
owning conditions
- fixture restarted -> reconnected in ~8s with NO node restart; both the raw
(ns=2) and UNS (ns=3) NodeIds read Good with fresh timestamps
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Removes the tag-set drift detector shipped in e77c8a35. The design was sound
-- the tautology drivers were correctly gated out, and the structurally
undetectable "vanished" direction was declared rather than faked -- but none of
that is the objection. The feature browsed real plant equipment on a recurring
timer, at a frequency no operator could configure (the interval was a
constructor parameter with no appsettings key), and that outbound traffic was
never once observed against a real PLC, CNC or MTConnect Agent.
Removed: TagSetDrift/TagSetDriftDetector; DriverInstanceActor's drift timer,
tick message, Props/ctor parameter, gate, handler and leaf-collector;
ITagDiscovery.AuthoredDiscoveryRefs + .DiscoveryStreamIncludesAuthoredTags and
their four implementations; 14 tests.
Kept: ITagDiscovery.SupportsOnlineDiscovery (it predates this and drives the
universal browse picker), and the whole IRediscoverable consumption from
09a401b8 -- the driver tells us, we do not poll.
CONSEQUENCE, deliberately accepted: AbCip and FOCAS browse a live backend but
implement no IRediscoverable, so they now have NO tag-change signal at all -- a
PLC re-download is invisible until someone re-browses by hand. Recorded in
CLAUDE.md so the next reader does not have to rediscover it, along with the
shape to reach for if that coverage is wanted back (event-driven, or triggered
by a reconnect/deploy, not a wall-clock timer).
Runtime.Tests 476 pass / 13 skipped (skip set unchanged); FOCAS 275, TwinCAT
192, MTConnect 491, AbCip 342, Core.Abstractions 266 all green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Joseph Doherty