Commit Graph

82 Commits

Author SHA1 Message Date
Joseph Doherty f1c188114b refactor(fleet): #542 delete the dead fleet-status chain (broadcaster, SignalR bridge, hub, messages)
#542 filed FleetStatusBroadcaster as blind to site nodes since the Phase 6 per-cluster mesh
split: it derives "the fleet" from central's own Akka gossip ring, which after the split is
just central's own pair. Scoping the cross-mesh rewrite found the deeper truth — the whole
chain is dead end to end, so this deletes it rather than reviving it.

  - Zero consumers. FleetStatusBroadcaster published FleetStatusChanged on the DPS
    "fleet-status" topic; FleetStatusSignalRBridge forwarded it to FleetStatusHub
    ("/hubs/fleet-status"); nothing anywhere connected. The AdminUI bans self-connecting
    HubConnections, no browser JS subscribes, and /fleet (Fleet.razor) reads
    NodeDeploymentState from the ConfigDb plus IClusterRoleInfo on a 5 s poll instead.
  - Its second input never existed. DriverHostStatusHeartbeat had no production sender —
    only the broadcaster's own test told it — so CurrentRevision was always null and the
    30 s staleness -> Degraded flip could never fire.
  - Post-Phase-6 DPS is pair-local, so even the central-only view it did produce was an
    accident of topology rather than a design.

The fleet visibility that actually works is unchanged and already cross-mesh-correct:
/fleet over NodeDeploymentState, and the four Phase 5 per-node telemetry dials.

Precedent for deleting inert machinery rather than activating it: #522 (driver stability
tiers), #507 (discovered-node injection), #525 (GetMemoryFootprint).

Deleted: FleetStatusBroadcaster (+ its nested DriverHostStatusHeartbeat), its admin
singleton registration and FleetStatusBroadcasterKey, FleetStatusSignalRBridge and its
key, FleetStatusHub and its route mapping, and the FleetStatusChanged / FleetNodeStatus /
FleetNodeHealth messages. Both Fleet/ directories survive — ControlPlane keeps
ClusterNodeAddressReconciler, Commons keeps GetDiagnostics. Docs swept where they claimed
a fleet-status channel exists (Telemetry.md's Phase 5 non-migrated channels,
Configuration.md, CLAUDE.md); completed-program plan docs under docs/plans and docs/v2 are
left as the historical record they are.

Build 0 errors, 868 warnings — byte-identical to master's rebuild baseline.
ControlPlane.Tests 231/231, AdminUI.Tests 1109/1109.

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
2026-08-06 17:21:30 -04:00
Joseph Doherty 406b1ae5ac docs(mesh): sweep for ClusterClient deletion — Dps|Grpc only, CentralContactPoints retired
Task 3.2 of the ClusterClient->gRPC migration. Sweeps the docs that still
described the Akka ClusterClient transport as live, following its Phase 3
deletion (this branch): MeshTransport:Mode is now Dps (pair-local DPS) or
Grpc (cross-mesh unary calls) only, ClusterClient fails validation loudly,
CentralContactPoints is deleted (not superseded), and the split-topology
validator requires exactly Grpc rather than ClusterClient.

- docs/Configuration.md: rewrite the MeshTransport section — table,
  dark-switch/PSK-seeding prose, listener sharing (:4056 driver-role,
  :4055 admin-role), remove the "shipped dark, cutover pending" framing.
- docs/Redundancy.md: replace the Dps/ClusterClient transport table with
  Dps/Grpc, fix the Phase 6 CentralCommunicationActor + validator bullets.
- docs/Telemetry.md: fix stale ClusterClient cross-refs, add a note that
  the driver-role telemetry listener also hosts NodeCommandService.
- docs/v2/config-db-schema.md: one-line "(retired: ...)" annotation on the
  historical AkkaPort/GrpcPort note — left as a historical record otherwise.
- CLAUDE.md: rewrite "## Mesh command transport (MeshTransport)" for the
  post-deletion world (no ClusterClient/receptionist, per-node gRPC dial,
  fire-and-forget semantics unchanged) and fix two Phase 6 bullets.

deferment.md has no mesh-transport mentions; left untouched.

docs-only change; `dotnet build ZB.MOM.WW.OtOpcUa.slnx` stays green (0
errors) after the edit.

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
2026-08-06 16:23:23 -04:00
Joseph Doherty 4bdce91002 docs(historian): live gate for #537 and #538 — both PASSED
v2-ci / build (push) Successful in 4m29s
v2-ci / unit-tests (push) Failing after 17m6s
Ran on the docker-dev central pair against the local HistorianGateway container talking to the real
AVEVA Historian on wonder-sql-vd03 (VPN up), with the host image rebuilt at e7fbe977.

Two things about the setup matter more than the results:

The value source had to be DRIVEN. Both the project Modbus fixture and oee-modbus-sim hold static
registers, and a static source produces no dependency-mux fan-out at all — the recorder captures
once at deploy and then goes quiet, which reads exactly like a broken recorder. Hand-written
Modbus FC6 writes put the value stream under the gate's control, which is what makes the rate
measurements below mean anything. The project fixture was also brought back up on the docker host
and MAIN-modbus-dev repointed to it, undoing the temporary :1502 repoint the read-back leg made.

The poison tag is the real shape, not a synthetic fault: GATE491-B's historianTagname was CHANGED
on an EXISTING tag, which Issue 5 guarantees is never provisioned, so its writes fail permanently
against a historian that has no such tag. That is how #538 was reached in the first place.

#537 — both nodes up: the Secondary logged the suppression line once and its outbox file froze
while the Primary's kept growing; 6 value changes x 2 tags gave WriteLiveValues{ok} +12 exactly,
where the pre-fix measurement was twice that. Failover: stopping the Primary made the survivor log
"now the recording Primary ... resuming value capture" and pick up every subsequent change. Return:
the restarted node was demoted, logged suppression, and the rate stayed at one node.

#538 — the quarantine fired with the tag named; 5 value changes over 30s gave ok +5 and error +5,
exactly one write per tag per change against a 2s drain interval.

The decisive leg is the IDLE one: 40s with no value changes left both counters unchanged and the
outbox byte size flat. The driven-rate legs only show a rate that happens to look right; a pre-fix
build re-writes its queued healthy entries on every drain pass whether or not anything new arrives,
so its counters climb during a quiet window with no input at all. Flat counters over idle is the
observation the bug cannot produce.

Incidental finding, recorded against Issue 5 rather than treated as a fix: a node RESTART cures an
unprovisioned tag. central-2's boot logged provisioning (dispatched=2, requested=2, ensured=2,
failed=0) and the poison tag started succeeding, because a full address-space rebuild treats every
historized tag as added and re-runs EnsureTags — where the incremental deploy that introduced the
renamed tag did not. So an operator who restarts a node while diagnosing "my tag isn't historizing"
makes the symptom disappear without learning the cause.

Docs only; no production code changed.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-04 12:36:19 -04:00
Joseph Doherty e7fbe97759 fix(historian): gate historization capture on the Primary, quarantine a poison tag (#537, #538)
Both defects were found by #491's live gate against a real HistorianGateway and filed rather than
fixed at the time, because each one's fix is a design decision rather than a patch.

#537 — both nodes of a redundant pair captured every value, so the historian received duplicate
live samples for every historized tag (measured: the gateway's write counter advanced twice per
value and halved when one central was stopped).

The recorder now reads a capture gate per value, fed from the new
PrimaryGatePolicy.ShouldRecordHistorizedValues through IHistorizationRoleView — a third verdict on
the same RedundancyRoleView object, registered unconditionally on driver nodes rather than
alongside the alarm section's IRedundancyRoleView (which only exists when that section is enabled;
piggybacking is the #511 registration accident).

Two things are deliberate and easy to get backwards:

  * It gates the CAPTURE, not the drain — the opposite of the alarm sink. That queue is in LocalDb
    and replicates, so the Secondary already holds the Primary's rows. The historization outbox is
    a node-local FasterLog replicating to nobody, so a drain-only gate would leave the standby
    appending forever to a queue no one empties: unbounded disk, then a flood of stale duplicates
    the moment it is promoted. Entries a node already captured are still drained by that node.

  * The peer it stands down for must RECORD THE SAME REFS, not merely hold the same rows — a
    Primary in this node's own per-cluster mesh. A node carrying no cluster-scoped role cannot
    establish that and never stands down: under a fleet-wide election the elected Primary is
    usually running a different cluster's configuration, and deferring to it would end this
    cluster's historization silently, with the drain reporting success on an empty outbox forever.
    That is the scope mismatch that once suspended the alarm drain on every node of the rig.

An unknown role captures, matching the alarm drain — a duplicate sample is recoverable, a hole in
the trend is not.

#538 — the drain writes every per-tag group and then acks all-or-nothing (deliberate: RemoveAsync
truncates THROUGH an id, so partial acking could drop older unwritten entries). One permanently
failing tag therefore pinned the FIFO head while every healthy tag in the batch was re-submitted
to the historian on every pass, indefinitely — duplicate samples and an outbox that never
truncated, observed live at 8.3 MB from a single unprovisionable tag.

The drain now tracks consecutive failures per historian tag and abandons that tag's entries once
it exceeds MaxWriteAttempts (new, default 5), so the head advances. The counter only advances on a
pass where some OTHER tag wrote successfully: that is what separates "this tag is poison" from
"the backend is down". A count-only quarantine looks identical on the poison-tag case and silently
discards the whole backlog during an ordinary gateway restart, turning a duplicate-samples bug
into a lost-history one. Abandonment is also deferred until every failing tag is over budget, so
nothing is discarded on a pass that could not have advanced the head anyway. Each abandonment logs
a Warning naming the tag; the total rides RecorderStatus.AbandonedEntries.

Capacity also gains a non-zero default (500,000 entries) as the disk backstop for the one case the
quarantine deliberately cannot resolve — a historian refusing every write, where retaining is
correct. Set 0 to restore unbounded retention.

Every new test was falsified by reverting the production change it guards, and each one reddens
alone:

  * drop the backend-reachable evidence -> only A_backend_that_fails_every_tag_abandons_nothing
  * never abandon -> only A_permanently_failing_tag_is_abandoned_so_healthy_tags_stop_being_rewritten
  * latch the capture gate at construction -> only The_capture_gate_is_re_read_per_value...
  * treat any Primary as a scope peer -> only the two scope-mismatch cases

Full suite green. (Four OpcUaClient.Browser failures during the run were the opc-plc fixture being
down on the shared Docker host; brought it back up and they pass.)

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-04 12:12:16 -04:00
Joseph Doherty e77ea4aa73 docs(historian): close #491 — read-back and EnsureTags proven against wonder-sql-vd03
v2-ci / build (push) Successful in 4m11s
v2-ci / unit-tests (push) Failing after 14m17s
The 2026-08-01 run closed every leg except read-back, which no reachable historian could
serve that day (the LAN instance accepts writes but serves no reads under a licence tag-count
violation; the licensed one needed the VPN). With the VPN up the run was repeated against
wonder-sql-vd03 and the remaining legs pass:

- EnsureTags provisioning: ensured=2, failed=0 — the path the LAN box could not exercise at
  all (it is RemoteGrpc-only and that CAP was not listening).
- Read-back: two Modbus registers carrying DISTINCT NON-ZERO constants were historized, and
  each read back as its own value (HR20 -> 8224, HR3 -> 120, quality 192). A first attempt
  read back 0, which proves nothing — the same trap #536 recorded, so the proof was rebuilt
  on values a crossed or fabricated answer could not fake.

Found during the run and FILED rather than fixed:

- #538 — one permanently-failing tag re-writes every healthy tag on every drain pass, forever.
  DrainOnceAsync writes EVERY per-tag group and then acks all-or-nothing, so a tag whose
  historian tag does not exist stops the FIFO head advancing while the healthy tags' values
  are re-submitted every drain interval; the outbox never truncates (Capacity defaults to 0 =
  unbounded, observed at 8.3 MB). The all-or-nothing ack is deliberate — RemoveAsync truncates
  THROUGH an id, so partial acking could drop older unwritten entries — so the fix changes the
  outbox contract and does not belong in this commit.

Also re-confirmed live: EnsureTags provisioning fires only for newly-ADDED historized tags, so
flipping an existing tag's historianTagname provisions nothing (Issue 5 of
2026-06-27-historian-gateway-integration-issues.md).

No production code changes — the recorder fix shipped in ad744734.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-04 11:29:15 -04:00
Joseph Doherty ad74473452 fix(historian): the recorder died on its first value — live gate for continuous historization (#491)
v2-ci / build (push) Successful in 4m54s
v2-ci / unit-tests (push) Failing after 16m42s
Ran #491's live gate against a real ZB.MOM.WW.HistorianGateway talking to a real AVEVA
Historian, and it found a production-fatal defect on the first value it ever saw.

ContinuousHistorizationRecorder.OnValueChangedAsync touched `Self` after awaiting
IHistorizationOutbox.AppendAsync(...).ConfigureAwait(false). ConfigureAwait(false) discards
Akka's ActorTaskScheduler synchronization context, so `Self` — which reads ActorBase.Context —
threw NotSupportedException. The throw escaped into supervision, which RESTARTED the actor;
because production seeds an EMPTY ref set (the deployed set arrives later via
UpdateHistorizedRefs), the restart silently discarded every historized ref. Net effect: one
value captured, then nothing, with no further error to show for it. Fixed by capturing
`IActorRef self = Self` before the await — the idiom the GetStatus handler already applied to
Sender.

Thirteen unit tests missed it because InMemoryOutbox.AppendAsync returned
ValueTask.CompletedTask: a completed ValueTask never yields, so the continuation ran inline
with the context still installed. The production FasterLog outbox fsyncs per entry and does
yield. The first regression test was itself vacuous — `await Task.Yield()` posts back to the
CAPTURED context, so the append still completed on the actor thread and reverting the fix left
the test green. The fake must complete on a POOL thread (Task.Delay(1).ConfigureAwait(false)),
as a real fsync does; with that, reverting the fix reddens exactly one test.

The rig could not run this gate at all: docker-dev carried ServerHistorian__* but no
ContinuousHistorization__* keys, so the recorder could never spawn. Both centrals now carry
them (default OFF — the recorder needs ContinuousHistorization:Enabled AND
ServerHistorian:Enabled), with the outbox on the per-node named volume so it survives a
container recreate. ServerHistorian__UseTls was hard-coded true and is now parameterized; a
plaintext h2c gateway was unreachable without it.

Gate results: outbox append, drain, WriteLiveValues accepted by a real historian, and restart
convergence (capture resumes after a restart with no new deployment) all pass. The read-back
hop is NOT proven — the LAN historian accepts writes but serves no reads under a "License
violation: Database tag count exceeds license tag count", and the licensed corporate one needs
the VPN. EnsureTags provisioning is likewise unexercised there (RemoteGrpc-only, and that CAP
is not listening). #491 stays open for the read-back hop alone.

Also recorded, not fixed: the recorder has NO Primary gate, unlike the alarm store-and-forward
sink, so both nodes of a redundant pair send duplicate live values. Gating it would introduce a
failover window with no capture — that is a design decision, not a cleanup.

Evidence: docs/plans/2026-08-01-continuous-historization-live-gate.md

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-01 19:57:37 -04:00
Joseph Doherty d542ab5b7c feat(security): author ACL grants from the real tree, not free text (#520 stage 7)
The last stage of #520. Enforcement landed in stages 5-6; this closes the
authoring end, where a grant could be saved, deployed and sealed green while
resolving to nothing.

Scope kind and scope target are now chosen TOGETHER from a picker over the
cluster's raw and UNS trees (AclScopePickerModal -> AclScopeTree, backed by
IAclScopeService). They were previously an unrelated dropdown and a free-text
box, so "RawDevice + a TagId" was a well-formed row. Because the trees are
rooted at the grant's own cluster and each node supplies its own kind,
NodeAclScopeCrossCluster, NodeAclScopeIdUnexpected, NodeAclScopeIdMissing and
NodeAclScopeKindRetired all become unauthorable by construction.

CheckScopeAsync mirrors DraftValidator.ValidateNodeAclScopes rule for rule and
blocks the save when a scope resolves to nothing - re-checked on submit, not
trusted from form load, because a row whose entity was deleted since is exactly
where the operator never touches the picker. The list page marks such rows
unresolved with the reason. The principal field suggests known principals and
reports how many live keys reach the one typed; reaching nobody warns rather
than blocks, since authoring ahead of minting is legitimate.

Deliberate departures from the plan line, all recorded in the design doc:
- The banners are reshaped, not removed. "Not enforced" stopped being true at
  stage 6, but enforcement is still a per-node flag the AdminUI cannot see, so
  a narrow grant really does confer nothing on a default-configured node.
- A Cluster row produces a NULL ScopeId. The deploy gate rejects one that
  carries an id, so filling in the cluster id would author undeployable rows.
- AclScopeTree is a new component rather than a third mode on RawTree, which
  already carries the /raw editing tree and the equipment reference picker.
- UNS tags are not offered: a UNS signal is a UnsTagReference, and a Tag grant
  already covers the raw node and every UNS projection of it.

Also fixed: the ACL editor had a DataAnnotationsValidator and no
ValidationSummary, so an invalid NodeAclId made Create a silent no-op.

Tests: 30 in AdminUI.Tests/Acls, falsified with six production mutations -
each turned exactly the intended tests red and no others.

Live gate PASSED on docker-dev, 11 legs; both central containers asserted on
the same image by container id. The pair that matters is legs 8 and 11: the
page and the deploy gate reject the same row with the same reason. Legs 5 and
9 are the positive controls - without them the gate passes against a page that
refuses everything. Not covered: UNS-realm scope selection (SITE-A has no UNS
areas). Rig restored to zero ACL rows.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-01 10:53:31 -04:00
Joseph Doherty 00aa486677 feat(security): enforce node ACLs on read, browse and subscribe (#520 stage 6)
Completes data-plane enforcement behind Security:Acl:Enabled. Read (every
attribute, not just Value), Browse, TranslateBrowsePath, CreateMonitoredItems,
SubscribeToEvents and SetMonitoringMode now consult scoped NodeAcl grants.
With the flag off nothing changes.

Four things went beyond the stage as written, each because leaving it out
would have made the feature look like it worked:

- Browse needed ancestor visibility. The trie inherits downward, and a client
  reaches a deep node by browsing the nodes above it, so without this every
  grant narrower than the cluster was authored, shipped and unreachable. It is
  a separate non-inheriting marker, NOT a synthesized Browse grant on each
  ancestor: the additive downward walk would carry such a grant from Plant onto
  Plant/Siemens/dev9 and every other sibling subtree. Pinned by a test that
  goes through the evaluator — probing the trie helper directly passes against
  the broken design, because a sibling is absent from the trie either way.

- Revocation had to reach LIVE subscriptions. Every other operation re-asks per
  request; a subscription is authorized once and then streams, and streaming is
  how machine consumers actually read this server — so stopping only the
  re-asking operations would have made revocation close to cosmetic.
  IDataPlaneAuthoritySource gained a Changed event.

  A denied item is queued BadUserAccessDenied and then DETACHED from its
  MonitoredNode2 fan-out, not set MonitoringMode.Disabled. Disabling discards
  the item's queued notifications, so the client is silenced without ever being
  told why. The over-the-wire test caught this; the unit-level path reports the
  item as stood down either way. Detaching is also reversible, so a restored
  grant resumes the feed with no client action.

- The decision cache holds the grant WALK, not the decision. Grants and the
  trie are fixed for a generation; a key's ExpiresUtc passes with the clock, so
  caching the whole decision would let an expired key keep working until the
  next deploy — the stage-5 open-session hole in a subtler form.

- TranslateBrowsePath is gated, without which browse filtering is decorative: a
  client refused a browse asks TranslateBrowsePathsToNodeIds for the same path.

Found en route: UserAuthorizationState's staleness ceiling couples two clocks.
MembershipResolvedUtc is stamped from DataPlaneAuthorizer's TimeProvider while
IsStale is evaluated against TriePermissionEvaluator's; they are the same
instance in production and diverge the moment anything injects a test clock, at
which point the evaluator denies EVERYTHING with a verdict indistinguishable
from "no grant matched". Stage 5's claim that these predicates cannot fire was
accidentally true; they are now disabled explicitly.

Also fixed: ContinuationPoint.NodeToBrowse is the manager HANDLE, not a
NodeState. The declared type is object, so the wrong cast compiled cleanly and
silently yielded null — skipping the browse source gate entirely.

Falsified: disabling the transit marking reddens exactly the three tests that
assert transit works; implementing transit as an ancestor grant reddens the
sibling-leak test; skipping the fan-out detach reddens the revocation test.
Full solution suite green.

The stage 8 live gate has NOT run yet.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-01 09:22:12 -04:00
Joseph Doherty f692f6afe4 feat(security): enforce node ACLs on write, alarms and HistoryRead (#520 stage 5)
`Security:Acl:Enabled` (default false) makes scoped `NodeAcl` grants decide
data-plane authorization at the three seams that have one, and gives HistoryRead
a check it never had. Read, Browse and Subscribe stay ungated — stage 6.

The ACL decision REPLACES the server-wide role check rather than chaining with
it. Chained, a grant would have to confer the role as well as pass the scope
check, and only Cluster-scoped grants confer a role — so every narrow grant
would have stayed inert and the flag would have looked inert too. The
load-bearing test is an ALLOW: a session carrying no roles at all writing
successfully on the strength of its grant. A deny-only test passes against a
build where the ACL is never consulted, because the role gate denies too.

Found en route: the principal never reached the session. The authenticator
passed it as OpcUaUserAuthResult.DisplayName, but RoleCarryingUserIdentity
derives DisplayName from the presented token — for a key-bearing client that is
the static username `apikey`, identical fleet-wide. So the trie's matching key
was dropped at the identity boundary, and the audit ClientUserId on a rejected
write recorded `apikey` for every consuming system. Principal and ApiKeyId are
now carried explicitly.

Beyond the stage as sketched:

- Revocation reaches an OPEN session. Every decision re-checks the
  authenticating key against the current snapshot; without it a key revoked by a
  deployment kept its authority until the client chose to reconnect, which for a
  machine-to-machine consumer can be indefinitely.
- UserAuthorizationState is built per CALL, not per session. The per-session
  shape exists so an LDAP session can re-resolve membership that changed out of
  band; nothing can change out of band under API keys, so the generation stamp is
  the whole invalidation signal and a per-call record picks it up for free.
  IsStale/NeedsRefresh therefore cannot fire here and are documented as vestigial.
- The trie is built inside DataPlaneAuthorityState.Apply, under the same
  generation as the snapshot, rather than by a second wiring at the publish
  actor. An uninstalled trie denies everything — indistinguishable from working
  enforcement. Deleting the install turns 13 of the 22 new tests red.
- The deploy log reports what the BUILD dropped instead of re-deriving it. The
  old count agreed with the builder on one of its four drop reasons; a rule with
  a blank ScopeId, a retired scope kind, or a path not starting at a realm
  vanished with the log reading zero.

Both IApiKeyStore implementations widened to the full grant set. The narrowing
was redundant — ApiKeyInterimRoleMapper applies the same filter itself — and
became a trap once scope was enforced. Pinned by asserting the roles are
unchanged, with the two grants carrying different flags so it cannot pass by
accident.

Falsified by mutation: enforcement never engaging, the write seam ignoring the
ACL, the realm dropped from the scope, the revocation check removed, and the
trie never installed each turn the intended tests red and no others.

Full solution suite green.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-01 05:00:28 -04:00
Joseph Doherty a438ac1640 feat(acl): keys + ACLs reach driver-only nodes via the artifact (#520 stage 4)
A driver-only node has had no ConfigDb since per-cluster mesh Phase 4, so it
registered NullOpcUaUserAuthenticator and denied every UserName token. Its keys
now ride the deployment artifact instead.

- ConfigComposer snapshots ApiKeys alongside the NodeAcls it already carried.
  Only the peppered HMAC ships, never a secret, and the pepper is node config
  that is deliberately not in the artifact.
- DeploymentArtifact.ParseDataPlaneAuthority decodes both node-side. Grants are
  scoped to the node's own cluster; keys are not, because ApiKey carries no
  ClusterId — a key is a fleet-wide credential and the ACL rows are what bind it.
- OpcUaPublishActor installs them on DataPlaneAuthorityState on every apply,
  BEFORE its plan.IsEmpty early-return: an artifact that only revokes a key
  produces an empty address-space plan, so feeding after that return would leave
  the revoked key working until an unrelated structural change redeployed.
- ArtifactApiKeyStore serves them to session activation; a node with a ConfigDb
  keeps ConfigDbApiKeyStore, whose per-request read is what makes revocation
  immediate there.
- OtOpcUaNodeManager gains DataPlaneAuthority — the cluster identity blocker #4
  named. NodeScope.ClusterId is required and the evaluator fails closed on a
  mismatch, so until now no scope it built could have evaluated. Nothing reads it
  until stages 5/6.

The generation is a node-local COUNTER driven by the RevisionHash, not derived
from its bits. PermissionTrieCache treats the highest GenerationId as current, so
a generation taken from hash bytes would install a lower number about half the
time and the cache would go on serving the previous deployment's ACLs, silently.

Sessions are not stamped with it yet — that belongs with the first seam that
reads a UserAuthorizationState, in stage 5, rather than being built now and
consulted by nothing.

Live-gated on docker-dev against site-a-1 (driver-only, no ConfigDb): key
authenticates, wrong secret denied, both halves of the revocation asymmetry
observed, and a SITE-A grant confirmed not to reach site-b-1. Key minted with an
independent Python HMAC so the verifier was checked against something other than
itself. Rig left clean.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 14:37:18 -04:00
Joseph Doherty dd939d3c97 feat(acl): anonymous becomes a normal grantable principal (#520 D3)
Anonymous sessions used to short-circuit in HandleImpersonation — "let the SDK's
default validation stand" — so they always connected and always carried no roles.
Now admission and authority are separated:

  admission  -> OpcUa:AllowAnonymous, the HOST's decision (it owns token policies)
  authority  -> the authenticator, resolving the reserved principal "anonymous"
                through the SAME grant path a key's principal goes through

So anonymous can be granted WRITE, not merely tolerated as read-only, and it
arrives at the downstream gates as the same RoleCarryingUserIdentity a key
produces — which is the real payoff: no gate carries an anonymous-shaped branch,
so there is no bypass path to get wrong.

AllowAnonymous defaults to TRUE on purpose. Defaulting it false would silently
deny every client without a key the moment this deploys, including ScadaBridge's
Data Connection Layer in another repo. Making keys mandatory is a rollout
decision, not an upgrade side effect. When false, the Anonymous token policy is
withheld so a client is refused at DISCOVERY rather than only at activation, and
activation is rejected independently in case a client attempts a token type the
endpoint never advertised.

Two fail-closed choices, both falsified:
- A failure to read anonymous's grants DENIES rather than admitting with no
  roles. Admitting is the pre-#520 behaviour, and defaulting to it exactly when
  the server cannot determine what anonymous may do stops being harmless once
  reads are gated. Making it admit reds Anonymous_denies_when_grants_cannot_be_read.
- A key may not authenticate as the reserved "anonymous" principal, or a
  credentialled client silently inherits whatever anonymous was granted.

AuthenticateAnonymousAsync is a DEFAULT interface implementation returning
admit-with-no-roles, i.e. exactly the pre-#520 behaviour, so
NullOpcUaUserAuthenticator and every test double keep working rather than being
forced to reimplement it — important because driver-only nodes run the Null one
and would otherwise black out. Note a default interface method is only reachable
through the interface-typed reference, which is how OpcUaApplicationHost holds it.

An existing test asserted the old fall-through and correctly went red; it is
rewritten to pin the new contract rather than deleted, with the reason recorded.

OpcUaServer 366, Host 42 green.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 13:20:14 -04:00
Joseph Doherty 65fda7237f refactor(acl): delete the LDAP data-plane authenticator and its tests (#520)
Stage 2b. LdapOpcUaUserAuthenticator was left registered nowhere by stage 2a;
this removes it, its three test files, and the config surface that existed only
to serve it.

Deleted: LdapOpcUaUserAuthenticator, LdapOpcUaUserAuthenticatorTests,
LdapAuthResilienceTests, LdapGlauthOutageLiveTests (19 tests).

Also removed four Security:Ldap keys — AuthTimeoutMs,
MaxConcurrentAuthentications, OutageFailureThreshold, OutageCooldownSeconds —
plus their validator rules and binding/validator test coverage. Verified first
that the ONLY consumers were the deleted class and the validator rules asserting
them; none appear in any checked-in appsettings. Leaving them would have created
exactly the dead-config surface #521/#524/#525 were filed to clear up.

Worth stating why that resilience machinery does not need relocating rather than
just deleting. It existed because the OPC UA SDK raises impersonation
SYNCHRONOUSLY under a SessionManager-wide event lock, so a hung LDAP bind there
serialized every session activation server-wide — Anonymous and X509 included.
Key verification is one indexed lookup plus one HMAC with no network I/O, so the
hazard the wall-clock bound, concurrency cap and outage circuit guarded against
no longer exists on that path. LDAP now runs only on AdminUI login, an ordinary
async request path where a slow directory affects the one request making it.

ConnectionTimeoutMs is RETAINED: it is projected into the shared library and is
not authenticator-specific.

Docs corrected rather than left to rot:
- CLAUDE.md's "LDAP Authentication" section described the deleted file as the
  data-plane authenticator. Rewritten, including a warning not to "restore" it on
  seeing the LDAP wiring that AdminUI legitimately still needs.
- infra/README §5b (the GLAuth outage live gate) marked RETIRED with the reason,
  rather than silently dropped — the harness GLAuth is still used by
  OTOPCUA_HARNESS_USE_LDAP=1 and is unaffected.
- docs/security.md's five-key resilience table cut to the one surviving key, and
  a new API-key identity section added.
Historical archreview/ records are left as-is; they are dated accounts.

Host.IntegrationTests 211 tests (was 230, i.e. exactly the 19 removed) with the
same 3 pre-existing failures verified against clean master. Host 37, Security 89.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 13:08:50 -04:00
Joseph Doherty 5a7a08deca fix(drivers): detect mid-run comms loss and recover from it (#527)
DriverInstanceActor.DisconnectObserved had no production sender. The only paths
into Reconnecting were a failed *initial* connect and an operator's Force
Reconnect, so a driver that connected and then lost its device stayed Connected
indefinitely: publishing Bad for every tag, never re-initialising, and NOT
recovering when the device returned. PublishHealthSnapshot read GetHealth() for
reporting only and never acted on a driver reporting itself unhealthy.

Both comms-loss quality paths -- #477's native-condition annotation and #481's
scripted-alarm input fan-out -- hang off ConnectivityChanged, so both were
correctly wired to a signal that never fired.

The Connected health tick now evaluates a watchdog and self-sends
DisconnectObserved when the driver reports a non-Healthy state AND its
LastSuccessfulRead has been frozen for DefaultCommsLossTimeout (90s).

The conjunction is the design, and the issue's own proposal -- "non-Healthy for
N consecutive polls" -- is wrong. Degraded means *connected but some data is
bad*; several drivers write health per tag inside the read loop
(ModbusDriver.ReadAsync), so one unreadable address latches Degraded forever
against a perfectly reachable device, and a state-only watchdog would reconnect
that driver every 90s indefinitely -- worse than the bug. LastSuccessfulRead is
the honest evidence: preserved across Degraded/Faulted writes, advanced by any
success. The state check remains the second conjunct so an idle Healthy driver
with nothing subscribed is not declared dead for not reading.

Verified across all 12 drivers that a sustained failure does reach a non-Healthy
state -- nine poll drivers via Degraded, MQTT via live connection state, Galaxy
via its reconnect supervisor. CalculationDriver reports Healthy while running
and so can never trip, which is correct: it has no transport.

Also: the Reconnecting retry now calls ShutdownAsync before InitializeAsync.
ModbusDriver (_transport) and MqttDriver (_connection) overwrite an
IAsyncDisposable field inside InitializeAsync without disposing the previous
one, so re-initialising without a teardown leaks a socket per attempt -- latent
while only a manual Force Reconnect reached this path, but the watchdog makes it
automatic and unbounded (~360 leaks for an hour-long outage at the 10s retry).

Tests: 8 new in DriverInstanceActorCommsLossWatchdogTests, each absence
assertion carrying a positive control (a proven count of processed health-poll
ticks). Falsified five ways -- dropping either conjunct reds exactly the test
written for it, and removing the teardown, the null baseline, or the watchdog
call each reds its own guard.

Live gate PASSED on docker-dev (Modbus + otopcua-pymodbus-standard), run twice:
  - fixture stopped -> watchdog fires at ~108s, "declaring comms lost"
  - fan-out observed at Debug: "pushed BadNoCommunication for 2 mux ref(s)"
    (#481 Layer 4) and the #477 native-condition annotation on a driver
    owning conditions
  - fixture restarted -> reconnected in ~8s with NO node restart; both the raw
    (ns=2) and UNS (ns=3) NodeIds read Good with fresh timestamps

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 10:58:38 -04:00
Joseph Doherty 5fff597324 revert(drivers): remove the periodic 5-minute device re-browse (#523)
Removes the tag-set drift detector shipped in e77c8a35. The design was sound
-- the tautology drivers were correctly gated out, and the structurally
undetectable "vanished" direction was declared rather than faked -- but none of
that is the objection. The feature browsed real plant equipment on a recurring
timer, at a frequency no operator could configure (the interval was a
constructor parameter with no appsettings key), and that outbound traffic was
never once observed against a real PLC, CNC or MTConnect Agent.

Removed: TagSetDrift/TagSetDriftDetector; DriverInstanceActor's drift timer,
tick message, Props/ctor parameter, gate, handler and leaf-collector;
ITagDiscovery.AuthoredDiscoveryRefs + .DiscoveryStreamIncludesAuthoredTags and
their four implementations; 14 tests.

Kept: ITagDiscovery.SupportsOnlineDiscovery (it predates this and drives the
universal browse picker), and the whole IRediscoverable consumption from
09a401b8 -- the driver tells us, we do not poll.

CONSEQUENCE, deliberately accepted: AbCip and FOCAS browse a live backend but
implement no IRediscoverable, so they now have NO tag-change signal at all -- a
PLC re-download is invisible until someone re-browses by hand. Recorded in
CLAUDE.md so the next reader does not have to rediscover it, along with the
shape to reach for if that coverage is wanted back (event-driven, or triggered
by a reconnect/deploy, not a wall-clock timer).

Runtime.Tests 476 pass / 13 skipped (skip set unchanged); FOCAS 275, TwinCAT
192, MTConnect 491, AbCip 342, Core.Abstractions 266 all green.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-28 13:59:06 -04:00
Joseph Doherty f0bd73bc2d docs: record the merge, correct §9's commit SHAs, and flag drift for removal (#523)
v2-ci / build (push) Successful in 5m43s
v2-ci / unit-tests (push) Failing after 19m42s
Three corrections to deferment.md, plus a CLAUDE.md warning.

The commit column in §9 was wrong for every stage. Each SHA was captured with
git rev-parse HEAD and then written into the file by amending that same commit
— which changes the SHA — so every recorded value was the pre-amend one: a
dangling object on no branch, reachable only through the reflog and due to
vanish at the next gc. The register was citing commits that effectively did not
exist. Now verified against git log e08855fb..914d70bf.

§9 also still described this as work on a feature branch; it merged to master
as 914d70bf.

The 2026-07-28 drift-detector entry now carries a SUPERSEDED banner. The
feature is in master and the entry stays as the record of what was built and
why, but it is not the intended end state: the owner's decision is to remove
the periodic re-browse (#523). The objection is the one the original entry
never weighed — the check is recurring, unconfigurable and never live-gated
outbound traffic to real devices, and how often you browse someone's PLC is an
operational decision rather than an implementation detail.

Both the banner and #523 record the consequence rather than arguing it: AbCip
and FOCAS implement no IRediscoverable, so after removal they have no
tag-change signal at all. TwinCAT and MTConnect keep their native event-driven
signals and lose only redundant polling. The IRediscoverable consumption itself
is explicitly out of scope for the removal.
2026-07-28 13:27:29 -04:00
Joseph Doherty e77c8a3569 feat(drivers): tag-set drift detection, gated on SupportsOnlineDiscovery
The second thing #516/§8.2 deliberately did not build. The original objection
stands — Modbus, S7, MQTT, AbLegacy and Sql echo authored config from
DiscoverAsync, so diffing discovered against authored is a tautology for them
— but it is now ENCODED rather than used as a reason not to build.

The discriminator already existed: ITagDiscovery.SupportsOnlineDiscovery is
documented as "enumerates the tag set from the live backend rather than
replaying pre-declared/authored tags", and it separates the fleet cleanly —
FOCAS, TwinCAT, MTConnect and AbCip true; the five config-echo drivers
explicitly false. The check runs only for a driver that is
SupportsOnlineDiscovery AND reports the new AuthoredDiscoveryRefs. That is
nullable rather than empty-by-default on purpose: an empty collection
legitimately means "nothing authored, everything discovered is new", so
conflating the two would report total drift for a driver that never opted in.

Where the value is: AbCip and FOCAS browse a live backend but implement no
IRediscoverable, so before this they had no change signal at all. A periodic
compare is the only way to notice a PLC re-download.

A finding that changed the design: FOCAS, TwinCAT and AbCip all re-emit their
authored tags into the same DiscoverAsync stream as device-derived ones, so
the browse picker can show an operator their existing tags. Discovered is
therefore always a superset of authored, and a VANISHED tag can never appear
missing — one whole direction is structurally undetectable for three of the
four. Shipping a Vanished list that is permanently empty for them would have
been the half-inert seam this whole exercise removes. So the driver declares
DiscoveryStreamIncludesAuthoredTags, the detector does not compute the
undetectable half, and the operator message says "missing-tag detection
unavailable for this driver" rather than letting the absence of a missing-tag
clause read as reassurance. MTConnect is the only driver where both directions
work — its stream is purely probe-model-derived.

Behaviour: a failed browse is not drift (an unreachable device would otherwise
report every authored tag as vanished); a truncated capture is skipped for the
same reason; an unchanged drift is reported once rather than re-stamping its
timestamp; drift that resolves clears the prompt. Interval defaults to 5
minutes — it browses a real device, and a tag set changing is an engineering
event, not a runtime one. It reuses the Stage-2 surfacing, raising the same
/hosts re-browse prompt, and never rebuilds the served address space.

Comparison logic is a pure, separately-tested type rather than actor-inline.
14 new tests. The gate was verified load-bearing by deleting the
SupportsOnlineDiscovery half: the tautology-driver test goes red, and it
asserts discovery was never invoked rather than merely "no prompt raised",
which would have passed for the wrong reason if the timer never fired.

Full suite green except the same three pre-existing fixture-gated integration
suites, identical counts.
2026-07-28 00:48:39 -04:00
Joseph Doherty adce27e7fa refactor(drivers): delete the dead discovered-node injection path (§8.2, #507)
#507 was filed as "injection inert in v3, re-migrate onto the raw subtree".
Reading it, the retained code is not revivable: it resolves equipment from
EquipmentNode.DriverInstanceId UNION EquipmentTags, and BOTH are structurally
empty in v3 (AddressSpaceComposer always constructs EquipmentNode with a null
DriverInstanceId; DeploymentArtifact hard-codes an empty EquipmentTags set).
Removing the guard would have changed a log line and injected nothing. So it
is deleted rather than fixed, and #507 closes as superseded by /raw
browse-commit.

Removed: HandleDiscoveredNodes, PartitionDiscoveredByDeviceHost,
ShouldWarnPartition, PlansRoutingEqual, ApplyDiscoveredPlansForDriver,
_discoveredByDriver, the redeploy re-inject tail, DiscoveredNodeMapper,
DiscoveredInjection, AddressSpaceApplier.MaterialiseDiscoveredNodes,
OpcUaPublishActor.MaterialiseDiscoveredNodes, and the Runtime-local copies of
CapturingAddressSpaceBuilder/DiscoveredNode.

The connect-time discovery loop goes too (StartDiscovery, RediscoverTick,
HandleRediscoverAsync, DiscoveredNodesReady, TriggerRediscovery). With
injection gone it had no consumer, and leaving it would either dead-letter or
keep browsing real devices up to ~15x per connect to drop the result.
ITagDiscovery itself stays — the /raw browse picker drives it through
Commons/Browsing/DiscoveryDriverBrowser, which never went through the actor,
so the picker is unaffected.

deferment.md §4.4 called the 18 DiscoveryInjectionDormantV3 tests "Real —
blocked on #507". They are not: they assert an equipment-rooted graft
(EquipmentRootNodeId == "EQ-1") that v3 cannot produce, so they could never
have unskipped as written. Deleted along with DiscoveredNodeMapperTests, the
CapturingAddressSpaceBuilder tests, and the MaterialiseDiscoveredNodes tests
in AddressSpaceApplierTests/OpcUaPublishActorTests. Runtime.Tests skipped:
31 -> 13.

IHostConnectivityProbe is deliberately NOT resolved here. GetHostStatuses()
has no production call site and nothing writes a DriverHostStatus row, but the
table was re-created in the v3 initial migration, so deleting on inference
would be wrong. Split to Gitea #521 with both options costed.

CLAUDE.md, docs/drivers/{Galaxy,TwinCAT,MTConnect}.md updated — they described
the seam as dead.

Build clean; Runtime.Tests 476 passed / 13 skipped; OpcUaServer.Tests 362
passed / 4 skipped.
2026-07-27 18:57:09 -04:00
Joseph Doherty e08855fb9d docs: source-verified deferment register + correct 17 drifted docs
v2-ci / build (push) Successful in 4m52s
v2-ci / unit-tests (push) Failing after 15m58s
Adds deferment.md — a source-verified inventory of new-driver status, all 17
open issues, in-code deferrals, open live gates and plan bookkeeping. Every
claim was checked against src/ and git, not against documentation.

Headline finding: three subsystems are authored, persisted and shipped but
never executed —
  * Node ACLs: IPermissionEvaluator/TriePermissionEvaluator have zero
    production consumers and OtOpcUaNodeManager never references them, yet
    ClusterAcls.razor authors NodeAcl rows and ConfigComposer.cs:51 ships them
    in every artifact. An authored deny rule has no effect.
  * IRediscoverable / IHostConnectivityProbe raise into the void (#518/#507);
    nothing ever writes a DriverHostStatus row.
  * DriverTypeRegistry is vestigial, so no factory passes a tier and every
    driver runs Tier A with the Tier-C protections dormant.

Also records the Calculation driver as picker-visible but unauthorable
(DriverConfigModal has no case) — the "registered but unauthorable" class
recurring after the Sql picker defect — and notes that no parity test guards
DriverConfigModal/DeviceModal, which is why it survived review.

Documentation corrections (source-verified):
  * ReadWriteOperations.md claimed "a denied read never hits the driver" via
    four types that do not exist in src/. Bannered + struck through; a security
    review reading that page would have concluded a per-node ACL gate exists.
  * CLAUDE.md: the Change Detection sentence was false (DriverHost consumes
    nothing); the mesh Phase 4 and Phase 5 live gates and the auto-down 1-vs-1
    gate had all PASSED; the ScriptedAlarmState table was already dropped.
  * driver-expansion tracking: Modbus RTU and SQL poll are merged, not
    pending — its command table would have made someone rebuild two shipped
    drivers in a fresh worktree.
  * drivers/README.md: dead DriverTypeRegistry paragraph, retired
    SystemPlatform namespace kind, missing Sql + Calculation rows, missing
    Modbus RTU-over-TCP transport.
  * TwinCAT.md/Galaxy.md promised an address-space rebuild that never happens.
  * Historian.md gained the #491 unproven-value-capture pointer.
  * IncrementalSync.md and AddressSpace.md bannered as wholesale v2-era (the
    rewrite is recorded as still owed, not done here); four v2 status docs
    bannered as historical; three wrong-name-for-a-live-type fixes.

Left deliberately untouched: the secrets NoOpSecretReplicator line, which
makes a security claim and needs the real behaviour identified rather than a
rename.
2026-07-27 17:26:20 -04:00
Joseph Doherty 394558e6c2 docs: the umbrella-index edit belongs on scadaproj's main, and must be pushed
v2-ci / build (push) Successful in 5m39s
v2-ci / unit-tests (push) Failing after 3h12m23s
The cross-repo propagation rule said WHAT to update but not WHERE. scadaproj is
a separate repo with its own checkout, so committing there inherits whatever
branch it happens to be on — usually an unrelated feature branch, sometimes one
with no upstream — and the index then disagrees with reality for as long as that
branch stays unmerged.

Observed 2026-07-27: the Sql poll driver's index entry had sat unpushed on
feat/overview-dashboard since 2026-07-24 despite this rule, and the Mqtt driver's
entry was about to join it. Both recovered by cherry-picking onto main.

Records the explicit procedure and says to cherry-pick stranded index commits
rather than merge the branch they are stuck on. The canonical statement now also
lives in scadaproj's own CLAUDE.md (main @ 75f267c).

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-27 13:36:31 -04:00
Joseph Doherty d5bd4226ee docs(mqtt): Sparkplug P2 live-verified; MQTT driver complete
Task 26 — the P2 milestone gate. Ran the live /run verification on an isolated
docker-dev rig (project otopcua-mqtt, ports 9210/4850-4851/14350, image built
from this branch) against the real Mosquitto TLS+auth broker and the C#
Sparkplug edge-node simulator on 10.100.0.35, and recorded the result.

The gate found three defects. Two are the same defect class the P1 gate found
twice — a hand-maintained AdminUI surface left behind by a driver-side feature —
and the third is a pre-existing cross-driver bug that only a Sparkplug flow
could surface.

  1. CRITICAL, FIXED — MqttDriverForm still shipped its P1 Sparkplug PLACEHOLDER.
     Switching Mode to SparkplugB rendered a "not available yet" notice and NO
     Group ID field, so with Sparkplug ingest fully shipped there was still no
     way to author a Sparkplug driver from the AdminUI at all. Sparkplug.GroupId
     is the driver's entire subscription filter (spBv1.0/{GroupId}/#): blank ⇒
     connected, Healthy, ingesting nothing. Now authors all five Sparkplug keys,
     MERGES over the existing sub-object rather than replacing it (so a key a
     newer driver adds inside it survives an older AdminUI), leaves it untouched
     in Plain mode, and validates the group id as the topic segment it is.
     Pinned by 8 MqttDriverFormModelTests cases, verified falsifiable — 8 RED
     with the fix stubbed out.

  2. PRE-EXISTING + CROSS-DRIVER, FIXED — every node label in the shared
     DriverBrowseTree was an <a href="#">. In a Blazor Web App, blazor.web.js's
     enhanced-navigation click interceptor resolves a bare "#" against
     <base href="/">, so clicking ANY browse-tree node label navigated the whole
     AdminUI to "/", tore down the circuit, and destroyed the hosting modal —
     losing the browse session AND the tag selection. @onclick:preventDefault
     does not help: it suppresses the browser's default action, not Blazor's own
     interceptor. Labels are now <button type="button">. Dates to the component's
     introduction (2026-05-28) and affects EVERY driver's picker; it surfaced
     only now because choosing a rebirth scope is the first flow that requires
     clicking a label rather than the ▶ toggle (always a button, always fine).

  3. FIXED (mitigation) — the browse tree rendered exactly once, at open, and
     never again, while OpenAsync returns as soon as the SUBSCRIBE is granted.
     On Sparkplug that means it is almost always empty forever, because births
     are never retained. Added a Refresh button that re-reads the root against
     the SAME session (nothing reconnects, nothing observed is lost, the tag
     selection is kept, only the armed rebirth scope is cleared).

Live gate result (all against the real broker + simulator, group OtOpcUaSim):

  - driver authored end-to-end through the fixed MqttDriverForm; blob is
    Mode:SparkplugB + Sparkplug.GroupId:OtOpcUaSim with enums as names
  - browse: BROWSER OPEN chip, Sparkplug-only rebirth panel, tree renders
    Group → EdgeNode → [Device] → Metric with the device folder Filler1 and
    node-level metric leaves SIDE BY SIDE, and "Node Control/Rebirth" as ONE
    leaf (the metric-name-is-not-a-topic-segment rule holding)
  - rebirth: group scope names the group + the 32-node cap, Cancel published
    NOTHING (simulator log unchanged); edge-node scope → "1 command published"
    and the simulator logged "rebirth NCMD #1 received; republishing metadata";
    group scope → "2 commands published", both edge nodes re-birthed; a device
    and a metric both resolve UP to "edge node EdgeA"; the panel does NOT render
    for a Plain MQTT device
  - browse-commit wrote bindable tuples with no topic/address key —
    {"groupId","edgeNodeId","metricName","deviceId"} for the device metric and
    deviceId ABSENT (not blank) for the node-level one, datatypes inherited from
    the birth (Int64 / Float)
  - deployed (Sealed), and both nodes served live changing values through OPC UA
    at the 2 s cadence — Good 0x00000000, no BadNodeIdUnknown
  - death→STALE: stopping the simulator drove both nodes to 0x80000000 with an
    empty value ~2 s later, and restarting it recovered them to Good on the new
    birth (FillCount back to its birth-declared 1000, then counting)
  - rediscovery fired exactly TWICE (once per authored scope: OtOpcUaSim/EdgeA
    7 metrics, OtOpcUaSim/EdgeA/Filler1 3 metrics) and was then gated across
    many subsequent births/rebirths — the anti-storm change gate working, and
    EdgeB correctly filtered out as an unauthored scope
  - tag editor: opens in SparkplugB (mode inference on reopen) with all four
    descriptor fields populated; blank group → "A Sparkplug group ID is
    required."; "Plant/1" rejected; "Node Control/Rebirth" ACCEPTED and saved
    with the slash whole; dataType override persists and its key is ABSENT again
    after selecting "(from birth certificate)"

Two things are NOT verified, and are recorded rather than claimed:

  - Rediscovery is INERT in v3 and this is a platform gap, not an MQTT one:
    nothing subscribes to IRediscoverable.OnRediscoveryNeeded and
    DriverHostActor.HandleDiscoveredNodes hard-returns. A DBIRTH introducing a
    metric does NOT change the OPC UA tree; redeploy. Verified from the driver's
    log line only, exactly as far as it can be verified.
  - A metric name containing "/" cannot be BROWSE-COMMITTED: the derived raw tag
    Name is a RawPath segment and may not contain "/", so the spec-mandatory
    "Node Control/Rebirth" is refused ("Row 3: Name must not contain '/'"). The
    refusal is loud and all-or-nothing, never a silently mis-bound tag, and
    Manual entry is the working path. Fixing it needs a name-sanitisation policy
    with a collision answer, so it was recorded rather than guessed at.

Also left open + documented: an empty browse tree cannot arm a rebirth (the
scope comes from a tree click, and the session side already accepts a bare
{group}/{edgeNode} — only a UI path is missing); _canRebirth is captured at
browse-open; the tag editor injects the Plain-only payloadFormat key into a
Sparkplug blob (cosmetic — the factory routes on the tuple).

Suites: offline 1528 passed / 0 failed (581 Driver.Mqtt + 809 AdminUI + 138
Core.Abstractions); live 15/15 against the broker + simulator.

Docs: docs/drivers/Mqtt.md brought fully up to date for P2 (Sparkplug config
sub-object, both tag shapes, the ingest state machine, rediscovery, browse +
rebirth + Refresh, a Known-gaps table); Mqtt-Test-Fixture.md gains the simulator
and drops its "Sparkplug has no fixture" claims; infra/README.md §3 gains the
simulator row; CLAUDE.md gains the simulator endpoint facts; the tracking doc
marks MQTT/Sparkplug COMPLETE. docker-dev/docker-compose.mqtt.yml is the
isolated-rig overlay the gate ran on.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-25 01:11:56 -04:00
Joseph Doherty 07a4a5ff5e docs(mqtt): P1 plain-MQTT milestone live-verified; endpoint recorded
Task 14 — the P1 milestone gate. Ran the live /run verification on a docker-dev
rig against the real Mosquitto TLS+auth fixture, and recorded the result.

The gate did its job: it found MQTT was unauthorable in production.

  RawDriverTypeDialog's driver-type list is a hand-maintained array that nobody
  added MQTT to, so the /raw "New driver" picker never offered it — with every
  other layer (contracts, driver, browser, factory, host registration, typed tag
  editor) complete and 266 green unit tests. Nothing tied that array to
  DriverTypeNames and AdminUI has no bUnit, so no offline test could see it.
  Fixed, plus RawDriverTypeDialogParityTests: a reflection parity guard that
  fails for ANY DriverTypeNames entry missing from the picker. Verified
  falsifiable — RED with "…cannot be authored from the /raw New-driver picker,
  so they are unreachable in production: Mqtt" before the one-line fix.

A second gap is left OPEN and documented, not fixed (no task ever covered it):
MqttDriverForm / MqttDeviceForm do not exist, and DriverConfigModal/DeviceModal
have no raw-JSON fallback — so an operator can create an MQTT driver but cannot
author its broker endpoint or credentials. P1 is not operator-complete until
those two razor forms land. The gate authored config via SQL to get past it.

Live gate result (isolated 2-node MAIN stack, image built from this branch,
fixture at 10.100.0.35:8883, AllowUntrustedServerCertificate rather than
pinning the CA into the rig):

  - typed MQTT tag editor renders and round-trips; Json shows the JSON-path
    field, Raw/Scalar hide it; wildcard topic a/+/c blocked at save; data-type
    list offers Float64 and no Double; qos/retainSeed absent on "(driver
    default)" and qos:2 written as a number; isHistorized/historianTagname
    survive a topic-only edit; SparkplugB renders the P2 placeholder and
    switching back keeps Plain values; jsonPath is guidance not a gate in all
    three shapes ($ seeded only for a brand-new blank tag, blank stays absent,
    clearing saves fine)
  - deployed, and both editor-written blobs resolved and served changing live
    values through OPC UA (22.8 / 1629, StatusCode Good) — no BadNodeIdUnknown
  - the bespoke #-observation browser drove a real broker session and rendered
    the fixture's actual topic tree (line1/{counter,state,temperature},
    noise/chatter, retained/seed); a Modbus device's picker still correctly
    reports "Browsing unavailable"

Also found live and documented (not a code change): both nodes of a redundant
pair run the driver, so a FIXED clientId makes them evict each other forever —
the broker logs "already connected, closing old connection" and both nodes
reconnect every ~2s while still reporting Healthy. Unset (the default) is
correct; confirmed 0 reconnects/60s on both nodes after removing it.

Suites: offline 1134 passed / 0 failed (266 Driver.Mqtt + 730 AdminUI incl. the
3 new guards + 138 Core.Abstractions); live 7/7 against the fixture.

Docs: new docs/drivers/Mqtt.md + Mqtt-Test-Fixture.md (the per-driver + fixture
convention every other driver follows), MQTT rows in docs/drivers/README.md,
TestConnectProbes.md, root README.md, and infra/README.md §3.

CLAUDE.md drift corrected while adding the MQTT endpoints:
  - the "every fixture carries project: lmxopcua" claim was false — only the
    MQTT fixture does; the filter returns one stack, not the fleet
  - lmxopcua-fix.ps1 is Windows-VM-only; on macOS drive the host over ssh/rsync
    (and the docker-dev rig itself runs locally under OrbStack)

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-24 18:44:43 -04:00
Joseph Doherty d1dac87f6f docs: record the bootstrap guard (CLAUDE.md + Phase 7 note)
v2-ci / build (push) Successful in 4m38s
v2-ci / unit-tests (push) Failing after 16m7s
The simultaneous-cold-start split-brain, carried as a Phase 7 "candidate follow-up,
not shipped", is now closed by the opt-in Cluster:BootstrapGuard (279d1d0f). Documents
the guard in CLAUDE.md's bootstrap section and marks the Phase 7 finding closed.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-24 08:35:16 -04:00
Joseph Doherty ecf5410433 docs(claude): record the Phase 6 boot-crash wiring landmine
v2-ci / build (push) Successful in 5m6s
v2-ci / unit-tests (push) Failing after 16m29s
The cluster-redundancy singleton's role scope is computed at AddAkka-configurator
time and must come from AkkaClusterOptions, never by resolving IClusterRoleInfo
there (it depends on the ActorSystem being built → stack overflow at boot). Caught
by the Phase 6 live gate; fixed in 3a4ed8dd.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-24 07:09:38 -04:00
Joseph Doherty 8a173bbaf6 docs(mesh): Phase 6 — mark done, remove single-mesh caveats, document pair-local meshes
Redundancy.md: KNOWN-LIMITATION (per-Akka-cluster election) marked RESOLVED;
new 'Per-cluster meshes (Phase 6)' section + pair-local secrets note; manual-
failover mesh-scope caveat removed; auto-down 1-vs-1 note updated (every mesh is
now two nodes, drill deferred to Phase 7). IManualFailoverService + ClusterRedundancy
razor caveats updated to pair-local reality. Configuration.md documents the
cluster-{ClusterId} role, per-pair self-first seeds, SplitTopologyTransportValidator,
and the intentional Dps default. Program + design status tables + CLAUDE.md marked
Phase 6 DONE.

(Task 8 subagent completed the edits but died on an API error before committing;
reviewed and committed by the controller.)

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-24 03:02:35 -04:00
Joseph Doherty e99ea40e2f docs(mesh-phase5): telemetry stream — Telemetry.md, config keys, supersede §6.3, program status
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-23 16:33:29 -04:00
Joseph Doherty 55a6e1f2b2 docs(mesh-phase4): ServiceLevel semantics on DB-less nodes + driver ConfigDb cut
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-23 13:18:57 -04:00
Joseph Doherty cf6110d02c docs(mesh): ConfigSource/ConfigServe config, rig flip switch, Phase 3 as shipped
Phase 3 Task 9. docker-dev rig: central-1/2 carry ConfigServe (:4055 h2c + committed
dev key) and stay Direct; the four site nodes carry ConfigSource (both central
endpoints + matching key) defaulting to Direct. Flip only the site nodes with
OTOPCUA_CONFIG_MODE=FetchAndCache at 'docker compose up' — central keeps SQL and models
the target topology. Binding :4055 activates the dedicated-h2c Kestrel takeover on
central (re-binds :9000 too). Docs: new docs/Configuration.md ConfigSource/ConfigServe
section; docs/Redundancy.md 'config bytes travel out-of-band' note; the program plan
Phase 3 section + tracking row flipped to CODE-COMPLETE; a CLAUDE.md 'Config source'
section.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 20:06:23 -04:00
Joseph Doherty 32055c2238 docs(mesh): Phase 2 live-gate record + correct the buffer-size rationale
Flipped the docker-dev rig end-to-end to MeshTransport:Mode=ClusterClient
(OTOPCUA_MESH_MODE, no rebuild) and ran the gate. PASSED for the transport:
deploy seals on ClusterClient, a stopped node fails the deploy naming it, the
MaintenanceMode hatch shrinks the DB-sourced contact set live (6->5->6 on both
centrals independently), and the frame-size canary stayed silent.

Four findings, recorded in 2026-07-22-mesh-phase2-live-gate.md:

  A. A 10-30s post-restart delivery window exists, but the DPS control fails
     identically -- it is a cluster-convergence property, not a Phase 2
     regression. Running the control is the only reason this is a note and not
     a false attribution.

  B. buffer-size=0's stated rationale was OVERSTATED. A node applies a TimedOut
     deployment anyway -- live-observed at +44s -- via DriverHostActor.Bootstrap
     replaying the orphan Applying row on restart. Zero buffering removes the
     silent, timer-driven replay, not the boot-time DB one. Corrected in the
     plan decisions table, Configuration.md, and CLAUDE.md; the decision stands,
     the justification narrowed.

  C. Stopping BOTH centrals strands the non-seed nodes at the Akka membership
     layer (frozen mid-Exiting, not Unreachable, so auto-down does not help).
     Transport-independent; a Phase 7 downing/rejoin drill item.

  D. The AdminUI driver Restart/Reconnect buttons (DriverStatusPanel) are
     rendered by no page -- orphaned since v3 Batch 2 retired /clusters/{id}/
     drivers. So gate steps 6/7 could not run; the driver-control node-side leg
     is proven live only because DispatchDeployment shares its EventStream path.

Step 8's planned log-signal does not exist (HandleApplyAck is silent on
success); proven instead by both centrals independently logging per-instance
ClusterClient creation, which a singleton cannot.

Also adds a repo-root .dockerignore: docker-dev/Dockerfile does COPY . ., which
was pulling .claude/worktrees (19GB of agent checkouts with their own bin/obj)
into the build context and exhausting the Docker VM disk mid-COPY.

Rig restored to Mode=Dps, six members, no leftover maintenance flags.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 18:40:42 -04:00
Joseph Doherty e4cb9a08e0 docs(mesh): MeshTransport config, Phase 2 as shipped, design-doc correction
Rig: all six docker-dev nodes carry MeshTransport__Mode plus both central
contact points. Mode is "${OTOPCUA_MESH_MODE:-Dps}", so the rig still comes up
on the transport it has always used and the live gate can flip the whole fleet
at `docker compose up` without a compose edit or a rebuild.

Docs: a MeshTransport section in Configuration.md (including why contact points
are addresses only, why the contact set does not scope delivery, and why
buffer-size=0 is a behaviour decision rather than a tuning knob), a "Command
transport" section in Redundancy.md contrasting the two modes, and a summary
beside the Redundancy notes in CLAUDE.md.

Corrects the design doc's claim that ScadaBridge replies with a typed failure to
every unhandled message. Neither of their comm actors has a ReceiveAny or an
Unhandled override; the idiom is per message type and fires on a missing
registration, and an unknown type dead-letters there as it does anywhere else.

Records the exit-gate deviation in the program plan: "an Ask timing out cleanly
against a stopped node" cannot be run, because no Ask crosses this boundary. The
honest equivalents are a stopped node failing the deploy at the apply deadline
and an unreachable contact dropping the command with a Warning.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 17:46:46 -04:00
Joseph Doherty ee69caa270 feat(config): add ClusterNode.MaintenanceMode — the hatch the live gate proved missing
Phase 1 gate step 4 failed: setting Enabled = 0 to take a node out of service
returns 422 ClusterEnabledNodeCountMismatch, because
DraftValidator.ValidateClusterTopology requires the enabled-node count to equal
ServerCluster.NodeCount. On a Warm/Hot pair — every cluster on the rig, and
every cluster in the target topology — Enabled can therefore never be the
maintenance hatch. The plan called step 4 "the check that makes step 3
acceptable to ship", so Task 3's behaviour change was not shippable as it stood:
a node down for maintenance would block every deployment to its cluster.

The two rules were each reasonable and contradictory together — the validator
reads Enabled as "part of the declared topology", Phase 1 additionally read it
as "expect an ack". Split the meanings rather than weaken either rule:

  Enabled          part of the declared topology  (validator, untouched)
  MaintenanceMode  expected to participate now    (coordinator + reconciler)

Rejected alternatives: counting configured rather than enabled nodes (drops the
guard against booting a pair into InvalidTopology); downgrading the rule to a
warning (weakens a deploy gate for everyone); shipping with no hatch.

Sabotage: dropping !n.MaintenanceMode from the coordinator query turns the new
test red. It also asserts both nodes remain Enabled, so the fix cannot quietly
regress to disabling the row after all.

Configuration.Tests 95/95, ControlPlane.Tests 101/101, solution builds clean.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 09:07:29 -04:00
Joseph Doherty 57e1d01766 docs(mesh): rig seed + Phase 1 documentation
- docker-dev seed: AkkaPort = 4053 on all six ClusterNode rows, so a freshly
  seeded rig matches a migrated one. GrpcPort left null.
- config-db-schema.md: both columns, why AkkaPort is NOT NULL/4053 and GrpcPort
  is nullable, the unenforced duplication + its reconciler, and Enabled's new
  second meaning as the deploy path's expected-ack set.
- Configuration.md: Cluster:Port / PublicHostname now flag that they are stored
  twice, with the "update the row too" instruction and why the drift is silent.
- design doc §7: Phase 1 marked done, plus a "Phase 1 as shipped" note recording
  both deviations rather than leaving the sketch reading as what happened.
- program plan: Phase 1 marked done; AdminUI node edit explicitly deferred.
- CLAUDE.md: the deploy-path behaviour change and its three consequences.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 08:38:45 -04:00
Joseph Doherty 9af935f237 docs(cluster): self-first seed ordering replaces the self-form fallback
v2-ci / build (push) Successful in 3m44s
v2-ci / unit-tests (push) Failing after 10m7s
docs/Redundancy.md's bootstrap section is rewritten around the mechanism that is
actually in force: which of Akka's two bootstrap processes runs is decided by
whether seed-nodes[0] is this node's own address, so the ORDER of Cluster:SeedNodes
is the fix, not a timer. Adds the per-process table, the shipped self-first
example, why the validator is conditional (site nodes are not seeds) and why it
matches PublicHostname rather than the 0.0.0.0 bind address.

The retirement is documented rather than erased: a new subsection explains that a
watchdog outside the join handshake cannot tell "no seed answered" from "a seed
answered and the join is in flight", and the ea45ace1 live-gate finding is kept
verbatim as the evidence that produced that conclusion (InitJoinAck at 10:49:05,
old incarnation retired 10:49:06, watchdog fired 10:49:15, second cluster). The
watchdog's own earlier PASS is kept too — it did pass what it was gated on.

Also: a live gate for the NEW mechanism is registered as outstanding (not
re-drilled on the docker-dev rig since the swap), and the plan doc that shipped
the watchdog gets a superseded banner rather than an edit, so the execution record
and its Task 8 finding stay readable.

Ripple: CLAUDE.md cluster section, docs/Configuration.md + docs/v2/Cluster.md +
docs/ServiceHosting.md SeedNodes entries, the per-cluster-mesh design doc's two
"CLOSED by SelfFormAfter" notes, and mesh-program Phase 6 (which had this
convergence queued — now marked done early) + Phase 7's live-gate name.
2026-07-22 07:46:02 -04:00
Joseph Doherty b853185cfd docs(redundancy): SelfFormAfter closes the seed-node bootstrap gap
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-22 06:10:11 -04:00
Joseph Doherty 8e195263f6 docs(claude): record the two redundancy corrections in the project instructions
v2-ci / build (push) Successful in 3m44s
v2-ci / unit-tests (push) Failing after 9m55s
Both change how someone should reason about this area, and the second changes a
public message contract, so they belong in CLAUDE.md rather than only in
docs/Redundancy.md. Includes the rule the work established: assert the effective
cluster config off a running ActorSystem, never the typed options or the
akka.conf text, because several HOCON fragments compete and the losing one still
reads correctly in the file.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-21 19:05:32 -04:00
Joseph Doherty 2e4ccf7fe9 chore(localdb): phase-2 DoD sweep
Build: 0 errors solution-wide, and 0 warnings from every project this branch
touches. The ~816 solution-wide warnings are pre-existing xUnit1051 /
OTOPCUA0001 / CS86xx in untouched driver + client test projects.

Tests: full solution run compared against a full run on a detached worktree at
the pre-branch baseline 2e46d054. The two failure SETS are identical -- all 13
tests, same names, zero new failures. Net +26 tests: +3 Core.AlarmHistorian
(drain gate), +6 Runtime (role view), +17 Host.IntegrationTests (migrator +
convergence). Set comparison rather than counts, because the suite carries
standing environment- and load-dependent failures a count would hide.

The greps found real drift Task 6 missed -- eight live sites still naming the
deleted SqliteStoreAndForwardSink, including the AdminUI /alarms/historian
panel text, which is user-visible, and a <see cref> in HistorianAdapterActor
that resolved to nothing without warning. All repointed at
LocalDbStoreAndForwardSink; CLAUDE.md's alarm-history paragraph now also
records the LocalDb buffer and the primary-gated drain, and drops DatabasePath
from the knob list. docs/AlarmTracking.md still promised an
AlarmHistorianOptions.Validate() startup warning for a relative DatabasePath
and an empty SharedSecret; both branches are gone, so it now says so.

Code references to AlarmHistorian:DatabasePath reduce to exactly two intentional
ones: AlarmSfLegacyMigrator.LegacyPathKey and its test. No `new SqliteConnection`
remains anywhere in Core.AlarmHistorian.

Recon doc gains the durable verification record: guard-deletion evidence for
both vacuous passes, the two exact-set replicated-table pins (both assert set
equality, so an added or a dropped registration fails), and the baseline test
comparison with a per-failure account of why each of the 13 is not this
branch's.

Stops here per the plan. Task 8's live gate needs explicit go-ahead; nothing on
this branch is to be merged.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-21 05:03:57 -04:00
Joseph Doherty f76d1f91e5 docs+chore(localdb): phase-2 rig config + docs
Enables the alarm store-and-forward sink on all four driver nodes of the
docker-dev rig -- the replicating site-a pair and the default-OFF site-b pin
-- so the live gate has a real buffer to watch converge, and can see that
site-b's sink works as a plain node-local queue with no peer traffic.

Two rig details worth stating rather than rediscovering. The endpoint is
deliberately unresolvable: there is no HistorianGateway here, so every drain
attempt fails, which is exactly the historian-outage state the buffer exists
for. But it still has to be a syntactically valid absolute http(s) URI or the
host refuses to start, because ServerHistorianOptionsValidator is
consumer-gated -- AlarmHistorian:Enabled=true makes the endpoint required
even while ServerHistorian:Enabled=false. And MaxAttempts is raised far above
the production default of 10, which against a permanently unreachable gateway
would dead-letter the entire queue about five minutes in, turning a buffering
test into a dead-letter test.

Docs record what an operator now has to know: that a rising queue depth on a
Secondary is correct and on BOTH nodes is not (that shape is a redundancy
snapshot naming neither node -- check node identity before suspecting the
sink), that delivery is at-least-once across a failover by design with no
dedup layer, and that AlarmHistorian:DatabasePath is removed but should be
left in place through the upgrade because the migrator still reads it to find
the file to copy.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-21 04:43:37 -04:00
Joseph Doherty 5ebf9876a5 docs: file Gitea issues for every deferred item; correct a stale limitation
Deferred work was living in plan docs and a CLAUDE.md banner rather than the
issue tracker, so it was invisible unless you happened to read the right file.
Each item was verified against the current tree before filing — nothing
speculative, nothing already fixed:

- #487 scripted-alarm redeploy recovery (CONFIRMED pre-existing bug). Verified
  still unfixed: ReassertValue exists only for VirtualTags, there is no
  OnAlarmsLoaded equivalent. Carries double-alarm-history risk, hence the
  careful proposed shape (node-only re-assert, never the alerts topic).
- #488 periodic desired-vs-actual subscription reconcile. Records what exists
  today (Connected-entry re-assert + a health-poll tick that never touches
  subscriptions) and why a true reconcile needs a new ISubscribable member
  across 7 drivers, plus the cheap no-interface-change variant to try first.
- #489 raw-tag rename hot-rebind (today: deploy-THEN-recreate).
- #490 absolute-RawPath Monaco tag-path completion.
- #491 continuous-historization live end-to-end + restart-convergence gate.

CLAUDE.md's "KNOWN LIMITATION 1" was STALE and told the reader not to trust a
cutover that has in fact been live-validated (read/write-persist/alarm-send all
green; alarm-readback skipped as a confirmed protocol limitation, not a bug) —
evidence in docs/plans/2026-06-27-otopcua-historian-followups.md. Rewritten as
a passed gate, keeping the recipe for re-running it. Limitation 2 is genuinely
open and now points at #491.

The cross-repo ScadaBridge cutover was already tracked in that repo (its #14
open, #20 closed), so no duplicate was filed; the Batch 4 doc now says so, and
records that the umbrella-index half is done.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-21 03:37:25 -04:00
Joseph Doherty c834d89ae7 docs(localdb): phase-1 runbook + redundancy/config docs
New operations runbook (enable/disable, fail-closed ApiKey rule, stop/start-
together, tombstone-retention window, MaxBatchSize row-count-vs-4MB, never
sqlite3-a-live-WAL-DB cp-triplet recipe). Configuration.md gains a LocalDb
section; Redundancy.md gains a pair-local config cache section (what boot-from-
cache does and does not cover); CLAUDE.md gains the Phase 1 scope paragraph.
2026-07-20 22:06:58 -04:00
Joseph Doherty b8208b3312 test+docs(v3-batch4-wp5): 2-node dual-namespace harness tests + address-space docs
Tests:
- OpcUaServer.IntegrationTests/DualNamespaceAddressSpaceTests.cs (NEW, over-the-wire,
  offline-safe): both namespace URIs registered + distinct; Raw + UNS subtrees browse
  and read; UNS variable Organizes-references its raw node; single-source fan-out parity
  (identical value/quality/timestamp on both NodeIds); HistoryRead via either NodeId ->
  GoodNoData under the shared tagname; WriteOperate gate symmetric across both NodeIds.
- Host.IntegrationTests/EquipmentNamespaceMaterializationTests.cs (extended): full deploy
  -> persisted-artifact -> ParseComposition round-trip carrying both realms, sealing across
  the redundant 2-node cluster (redundancy non-interference). In-memory harness, offline.

Docs (dual-namespace reality):
- CLAUDE.md: new "v3 OPC UA Address Space (Batch 4)" section + Batch-4 testing paragraph.
- docs/Uns.md: address-space projection (two namespaces, Organizes edge, effective-name leaf).
- docs/Historian.md: dual-registration (both NodeIds -> one tagname); updated CLI examples.
- docs/ScriptedAlarms.md + docs/AlarmTracking.md: multi-notifier fan-out, ConditionId=RawPath.
- docs/ScriptEditor.md: dual-namespace clarification (script tag-path semantics unchanged).

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 13:11:34 -04:00
Joseph Doherty bdcb84bd7d docs(v3-batch3): UNS reference-only Equipment + {{equip}}/RefName resolution
Uns.md Tags section rewritten to the reference-only model (UnsTagReference list,
cluster-scoped picker, effective-name uniqueness, {{equip}}/RefName); CLAUDE.md
gains a v3 Batch 3 paragraph. ScriptEditor.md was updated by WP4.

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 07:37:36 -04:00
Joseph Doherty 33915c9e4d docs(v3): Batch 2 — /raw authoring tree guide + CLAUDE.md + PR description
v2-ci / build (pull_request) Successful in 3m25s
v2-ci / unit-tests (pull_request) Failing after 8m37s
docs/Raw.md (the /raw project tree: lazy tree, endpoint→DeviceConfig split, manual/CSV/browse
authoring, Calculation driver, routed-flow retirement); CLAUDE.md AdminUI section notes /raw +
DriverTypeNames rewire + Calculation driver; PR body with the 7-item live-gate evidence.

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 05:27:31 -04:00
Joseph Doherty b461b2bcf8 fix(deps): clear CVE-2025-6965 — pin SQLitePCLRaw bundle to 2.1.12, drop audit suppression
v2-ci / build (pull_request) Successful in 5m41s
v2-ci / unit-tests (pull_request) Failing after 10m49s
The patched native SQLite bundle shipped: SQLitePCLRaw.bundle_e_sqlite3 2.1.12
is the first version outside the CVE-2025-6965 / GHSA-2m69-gcr7-jv3q vulnerable
range (<= 2.1.11), embedding the SQLite 3.50.2+ fix.

- Bump Microsoft.Data.Sqlite 9.0.0 -> 10.0.7 (aligns with the EF Core 10.0.7 family).
  This alone still pulls the native bundle 2.1.11, so:
- Add a surgical direct pin of SQLitePCLRaw.bundle_e_sqlite3 2.1.12 in Core.AlarmHistorian
  (the sole consumer), overriding the transitive 2.1.11 without enabling
  CentralPackageTransitivePinningEnabled (which breaks the Roslyn 5.0/4.12 split).
- Remove the temporary NuGetAuditSuppress from Directory.Build.props.

Verified: dotnet restore/build clean with audit active (no GHSA-2m69), every
in-solution project resolves lib.e_sqlite3 2.1.12, AlarmHistorian tests 29/29 green
against the real native bundle.

Closes #458
2026-07-15 08:17:19 -04:00
Joseph Doherty 1d3b6d1264 docs(infra): add infra/ folder — canonical testing + deployment infra index
v2-ci / build (push) Successful in 4m4s
v2-ci / unit-tests (push) Failing after 12m22s
Consolidates the scattered infra facts (docker host layout, driver fixtures, the
three env-gated live-gate recipes, the Host.IntegrationTests harness, the HistGW
local deploy, docker-dev rig, shared GLAuth, deploy) into infra/README.md as the
single entry point, cross-linked to dev-environment.md + the deep docs. CLAUDE.md
points at it from the Docker Workflow section.
2026-07-15 08:00:37 -04:00
Joseph Doherty 3a48eb0c70 fix(harness): swap retired bitnami/openldap for GLAuth in the Host real-LDAP mode
v2-ci / build (pull_request) Successful in 4m13s
v2-ci / unit-tests (pull_request) Failing after 10m18s
The Host.IntegrationTests opt-in real-LDAP mode (OTOPCUA_HARNESS_USE_LDAP=1)
used bitnami/openldap:2.6, gone since Bitnami deprecated their free image
catalog in 2025. Replaced it with GLAuth — the same LDAP server the rest of
the project already uses (docker-dev + the live OPC UA data-plane auth, both
against the shared GLAuth on :3893) — removing the lone OpenLDAP outlier and
the gone-image breakage.

- Added tests/.../Host.IntegrationTests/glauth/config.toml: baseDN dc=zb,dc=local,
  a search-capable serviceaccount, dev users alice (full access) / bob (read-only),
  and role groups with the same gidnumbers as scadaproj/infra/glauth so GroupToRole
  maps identically.
- Compose ldap service -> glauth/glauth:latest, mounting the config, host :3894 -> :3893.
- Repointed the harness real-LDAP override from the OpenLDAP cn=admin/ldapadmin to
  GLAuth's cn=serviceaccount/serviceaccount123.

Live-verified against a deployed container on :3894: the serviceaccount bind
searches and returns alice + her 5 memberOf groups; alice/bob bind with the
correct password; a wrong password yields Invalid credentials (49) — exactly the
OtOpcUaLdapAuthService flow (proven identical to the data-plane's shared-GLAuth path).

NB: real-LDAP mode is opt-in; the default Host run uses StubLdapAuthService, so
this never blocked the default suite. Integration-sweep follow-up #6 (LDAP leg);
the amd64-emulated-SQL deadline leg remains open.
2026-07-15 06:03:50 -04:00
Joseph Doherty 9f62310b49 fix(historian): map Boolean historization to Int2, not degenerate Int1 (#441)
v2-ci / build (pull_request) Successful in 3m24s
v2-ci / unit-tests (pull_request) Failing after 8m19s
The historian's Int1 analog-tag creation path is server-degenerate:
EnsureTags(Int1) stores an unusable stub (live-probed on the deployed
2023 R2 historian; root-caused + documented gateway-side in
HistorianGateway #11 / PR #16). HistorianTypeMapper mapped
DriverDataType.Boolean -> HistorianDataType.Int1, so Boolean historized
tags failed auto-provisioning.

Map Boolean -> Int2 instead (store 0/1). Int2 is a proven-creatable
analog type, and the value-write path already sends every value as a
double on WriteLiveValues regardless of tag type, so Boolean 0.0/1.0
round-trips cleanly. Both Float and Int2 are supported, so a
pre-existing Float Boolean tag retypes cleanly on its next deploy.

- HistorianTypeMapper: Boolean => Int2 (+ remarks)
- Unit pins: HistorianTypeMapperTests, GatewayTagProvisionerTests
  (Boolean_definition_carries_explicit_Int2_presence)
- Live gate: EnsureTags_boolean_sandbox_provisions_as_Int2 (was _Int1);
  retype probe reframed Float->Int2 (drops the dead Int1 leg)
- Docs: docs/Historian.md 'Boolean historization maps to Int2', CLAUDE.md,
  archreview STATUS.md (R2-06 live gate -> 6/6)

Resolves #441.
2026-07-14 11:57:27 -04:00
Joseph Doherty 354d74d370 docs: ServerHistorian HistoryRead knobs + Security:Ldap resilience keys; R2-08 status (03/S2, 03/S3)
v2-ci / build (pull_request) Successful in 3m33s
v2-ci / unit-tests (pull_request) Failing after 9m24s
2026-07-13 12:21:33 -04:00
Joseph Doherty 12b3206555 Merge R2-05 AdminUI authorization (arch-review round 2) [PR #428]
v2-ci / build (push) Successful in 3m27s
v2-ci / unit-tests (push) Failing after 8m29s
Finding 04/C-1 (High): explicit named-policy gating on all 38 AdminUI pages via
new AdminUiPolicies (ConfigEditor + AuthenticatedRead) + anti-drift reflection
guard. T15 live pass deferred (docker-dev auto-admin can't observe deny).
STATUS.md conflict resolved additively. Build clean.
2026-07-13 10:12:57 -04:00
Joseph Doherty e5cf986a10 docs(r2-06): 0.2.0 DataType provisioning delta (upsert/retype) + HISTGW_BOOL_SANDBOX_TAG recipe (06/U-7) 2026-07-13 10:02:37 -04:00
Joseph Doherty dad783514a docs(security): document AdminUI policy tiers (ConfigEditor/AuthenticatedRead) + R2-05 status 2026-07-13 10:01:52 -04:00
Joseph Doherty 9fadead6a6 docs(archreview): remediation plans + fix flagged doc drift
Add the 7 per-domain design+implementation plans (archreview/plans/) with
an index, produced from the 2026-07-08 architecture review.

Fix two confirmed doc drifts the review flagged (theme #5):
- CLAUDE.md KNOWN LIMITATION 2: the continuous-historization historized-ref
  feed IS wired (AddressSpaceApplier.FeedHistorizedRefs -> UpdateHistorizedRefs
  -> recorder); rewrite to reflect that value-capture is code-complete and only
  the live end-to-end + restart-convergence verification remains.
- CLAUDE.md ScriptAnalysis gating: endpoints use Roles=Administrator,Designer
  via RequireAuthorization, not the FleetAdmin policy.
2026-07-08 16:14:37 -04:00