HistoryRead is served server-side by IHistorianDataSource; nothing ever dispatched
to a driver's IHistoryProvider. Removes IHistoryProvider, the OpcUaClient history
passthrough, DriverCapability.HistoryRead (+ resilience tier rows and AdminUI row),
HistoryReadStatusException and its node-manager catches, and the tests that only
covered them. Shared result types and HistoryAggregateNotSupportedException stay
(moved to Historian/HistoryReadContracts.cs). A stored ResilienceConfig still
carrying a HistoryRead key parses (diagnostic only; new test).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
DriverHostActor.StopChild released the tracker (Invalidate) and evicted the hub synchronously, but
Context.Stop is asynchronous: the stopping child's in-flight SubscribeBulk completed through its
capability invoker AFTER the release, and its RecordCallStart/RecordSuccess/RecordCallComplete
re-created the tracker entry, which DriverResilienceStatusPublisherService then re-emitted every tick
forever. Confirmed with a temporary stack capture in the tracker (removed): every post-Release write
came from DriverInstanceActor.HandleSubscribeAsync -> CapabilityInvoker.ExecuteAsync; the test failed
whenever it landed before the second publisher tick.
- DriverResilienceStatusTracker.Retire (Release) drops the id's snapshots and ignores every later
Record* for it; one guarded write path re-checks after the upsert so a write racing Retire cannot
survive it. Reinstate (called by DriverCapabilityInvokerFactory.Create) lifts it, so the same id
re-added in a later deployment records again.
- TelemetryLocalHub.Evict now also holds the id (late Health/Resilience dropped, neither cached nor
fanned out); ITelemetryLocalHub.Readmit (default no-op), called at the top of SpawnChild, lifts it.
- The removal test is now deterministic: an invoker decorator holds the child's subscribe inside the
real invoker's bracket across the removal and signals when its late writes are done (positive
control) — red 5/5 without the fix, green 20/20 with it. New unit tests for the tracker guard
(incl. a Retire race), the factory Release/Create pair, the hub hold/readmit, and a re-add-same-id
host test.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
With Security:Acl:Enabled=true, a session holding any grant narrower than
Cluster could not browse the served address space at all — confirmed by the
live gate.
Mechanism. Both realm trees hang under ONE anchor folder minted in
CreateAddressSpace: NodeId("OtOpcUa", <raw ns>). ResolveParentFolder parents
every top-level raw folder AND every top-level UNS area under it, and the
Browse override throws BadUserAccessDenied for the WHOLE call when its source
node is denied. DataPlaneAuthorizer.BuildScope split that identifier like a
path, producing {Kind: Raw, Segments: ["OtOpcUa"]} — but trie grant paths are
["raw", <RawPath segments…>] and "OtOpcUa" is never a path segment, so
PermissionTrie.HasBrowseTransit failed at the realm-child lookup and
CollectMatches yielded only Cluster-scoped grants. Deny, at the front door.
The trie already handled a zero-segment scope as "the realm root, itself an
ancestor of everything granted in that realm" (PermissionTrie.cs:101-118) —
no production path had ever produced that scope. This wires it up.
- V3NodeIds.SharedRootFolderId names the anchor identity; CreateAddressSpace
now mints the NodeId/BrowseName/DisplayName from it (behaviour-identical,
makes the coupling explicit).
- BuildScope maps that identifier (realm-qualified to Raw — in the UNS
namespace s=OtOpcUa is an ordinary Area) onto the zero-segment Raw scope.
- Evaluate adds a second-realm OR-leg for Browse on the anchor only: it fronts
both trees, so a principal whose only Browse grant lives in the UNS tree
still enters through this raw node. Browse only; transit confers navigation,
never read/write/subscribe.
- No trie, evaluator or builder change. No reordering above the decision cache
— key revocation is still re-checked before the lookup — and the cache key
is unchanged (the combined verdict caches under the anchor's own
identifier/realm; a generation advance clears it).
Falsifications actually run (implement-then-revert-the-line, plus one
genuine red-first):
- DataPlaneAuthorizerTests: the 7 new facts were written BEFORE the fix and 4
went red (the 3 deny-only ones passed vacuously, as a deny-only test must).
- Deleting the UNS OR-leg reddens exactly one fact,
The_shared_root_folder_is_browsable_on_a_uns_only_browse_grant (26/27 pass).
- Dropping the realm qualification from IsSharedRootFolder reddens exactly
A_uns_area_named_like_the_anchor_gets_no_anchor_semantics. Its first draft
did NOT catch this — a raw-only grant is refused either way; the
discriminating grant had to be an unrelated UNS one, whose own transit marks
the zero-segment UNS scope the un-qualified mapping would land on.
- Disabling the BuildScope anchor branch reddens the new end-to-end fact
NodeManagerStage6EnforcementTests.The_shared_root_folder_is_browsable_on_a_narrow_grant_and_still_filters
(real booted server, real Browse dispatch: BadUserAccessDenied on the whole
call), which also pins that the reference filter still withholds the
non-granted sibling.
- Core PermissionTrieBrowseTransitTests gains 3 characterization facts pinning
the previously-dead realm-root branch (transit true for the granting realm's
root, false for the other realm's, false without the Browse bit). These were
green on arrival — the trie was always correct; nothing called it.
Tests: Core 351 -> 354 passed; OpcUaServer 482 -> 490 passed. Build clean.
The connect path was the one capability call the resilience pipeline never
saw, so a driver that could not connect contributed zero breaker input and
the /hosts resilience panel read 'All targets healthy' against a dead
endpoint. InitializeAsync now routes through the invoker as
DriverCapability.Initialize, keyed on the driver-instance id.
- No pipeline retry: the actor's RetryConnect timer is the retry loop
(parser forces retryCount overrides to 0, same funnel as write-shaped).
- Breaker threshold 2, not the Read-sized 5: the breaker samples 30 s and
the reconnect timer fires every 10 s, so at most three attempts fit one
window — threshold 5 could never open. Pinned by its own test.
- An open breaker fast-fails into the same InitializeFailed path as a real
connect failure; the 15 s break window costs at most one extra tick.
The best-effort teardown ShutdownAsync stays unwrapped.
- Companion fix: DriverCapabilityInvokerFactory.Create now invalidates the
instance's tracker snapshots with its pipelines — a respawn during an
outage discarded the only breaker whose OnClosed could clear the OPEN
flag, stranding a false 'Breaker OPEN' chip forever.
Live gate PASSED on docker-dev (image symbol-verified per container):
rtu-gate against dead 10.100.0.35:5021 opened its Initialize breaker on
both centrals with the red 'BREAKER OPEN' chip on /hosts; a socat
5021→5020 forwarder recovered it (both breakers closed within 1 s, panel
back to 'All targets healthy'); healthy MAIN-modbus recorded zero breaker
events. Full suite 50/50 green. Evidence:
docs/plans/2026-08-07-adminui-followups-and-live-gates.md §7.
Completes data-plane enforcement behind Security:Acl:Enabled. Read (every
attribute, not just Value), Browse, TranslateBrowsePath, CreateMonitoredItems,
SubscribeToEvents and SetMonitoringMode now consult scoped NodeAcl grants.
With the flag off nothing changes.
Four things went beyond the stage as written, each because leaving it out
would have made the feature look like it worked:
- Browse needed ancestor visibility. The trie inherits downward, and a client
reaches a deep node by browsing the nodes above it, so without this every
grant narrower than the cluster was authored, shipped and unreachable. It is
a separate non-inheriting marker, NOT a synthesized Browse grant on each
ancestor: the additive downward walk would carry such a grant from Plant onto
Plant/Siemens/dev9 and every other sibling subtree. Pinned by a test that
goes through the evaluator — probing the trie helper directly passes against
the broken design, because a sibling is absent from the trie either way.
- Revocation had to reach LIVE subscriptions. Every other operation re-asks per
request; a subscription is authorized once and then streams, and streaming is
how machine consumers actually read this server — so stopping only the
re-asking operations would have made revocation close to cosmetic.
IDataPlaneAuthoritySource gained a Changed event.
A denied item is queued BadUserAccessDenied and then DETACHED from its
MonitoredNode2 fan-out, not set MonitoringMode.Disabled. Disabling discards
the item's queued notifications, so the client is silenced without ever being
told why. The over-the-wire test caught this; the unit-level path reports the
item as stood down either way. Detaching is also reversible, so a restored
grant resumes the feed with no client action.
- The decision cache holds the grant WALK, not the decision. Grants and the
trie are fixed for a generation; a key's ExpiresUtc passes with the clock, so
caching the whole decision would let an expired key keep working until the
next deploy — the stage-5 open-session hole in a subtler form.
- TranslateBrowsePath is gated, without which browse filtering is decorative: a
client refused a browse asks TranslateBrowsePathsToNodeIds for the same path.
Found en route: UserAuthorizationState's staleness ceiling couples two clocks.
MembershipResolvedUtc is stamped from DataPlaneAuthorizer's TimeProvider while
IsStale is evaluated against TriePermissionEvaluator's; they are the same
instance in production and diverge the moment anything injects a test clock, at
which point the evaluator denies EVERYTHING with a verdict indistinguishable
from "no grant matched". Stage 5's claim that these predicates cannot fire was
accidentally true; they are now disabled explicitly.
Also fixed: ContinuationPoint.NodeToBrowse is the manager HANDLE, not a
NodeState. The declared type is object, so the wrong cast compiled cleanly and
silently yielded null — skipping the browse source gate entirely.
Falsified: disabling the transit marking reddens exactly the three tests that
assert transit works; implementing transit as an ancestor grant reddens the
sibling-leak test; skipping the fan-out detach reddens the revocation test.
Full solution suite green.
The stage 8 live gate has NOT run yet.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
`Security:Acl:Enabled` (default false) makes scoped `NodeAcl` grants decide
data-plane authorization at the three seams that have one, and gives HistoryRead
a check it never had. Read, Browse and Subscribe stay ungated — stage 6.
The ACL decision REPLACES the server-wide role check rather than chaining with
it. Chained, a grant would have to confer the role as well as pass the scope
check, and only Cluster-scoped grants confer a role — so every narrow grant
would have stayed inert and the flag would have looked inert too. The
load-bearing test is an ALLOW: a session carrying no roles at all writing
successfully on the strength of its grant. A deny-only test passes against a
build where the ACL is never consulted, because the role gate denies too.
Found en route: the principal never reached the session. The authenticator
passed it as OpcUaUserAuthResult.DisplayName, but RoleCarryingUserIdentity
derives DisplayName from the presented token — for a key-bearing client that is
the static username `apikey`, identical fleet-wide. So the trie's matching key
was dropped at the identity boundary, and the audit ClientUserId on a rejected
write recorded `apikey` for every consuming system. Principal and ApiKeyId are
now carried explicitly.
Beyond the stage as sketched:
- Revocation reaches an OPEN session. Every decision re-checks the
authenticating key against the current snapshot; without it a key revoked by a
deployment kept its authority until the client chose to reconnect, which for a
machine-to-machine consumer can be indefinitely.
- UserAuthorizationState is built per CALL, not per session. The per-session
shape exists so an LDAP session can re-resolve membership that changed out of
band; nothing can change out of band under API keys, so the generation stamp is
the whole invalidation signal and a per-call record picks it up for free.
IsStale/NeedsRefresh therefore cannot fire here and are documented as vestigial.
- The trie is built inside DataPlaneAuthorityState.Apply, under the same
generation as the snapshot, rather than by a second wiring at the publish
actor. An uninstalled trie denies everything — indistinguishable from working
enforcement. Deleting the install turns 13 of the 22 new tests red.
- The deploy log reports what the BUILD dropped instead of re-deriving it. The
old count agreed with the builder on one of its four drop reasons; a rule with
a blank ScopeId, a retired scope kind, or a path not starting at a realm
vanished with the log reading zero.
Both IApiKeyStore implementations widened to the full grant set. The narrowing
was redundant — ApiKeyInterimRoleMapper applies the same filter itself — and
became a trap once scope was enforced. Pinned by asserting the roles are
unchanged, with the two grants carrying different flags so it cannot pass by
accident.
Falsified by mutation: enforcement never engaging, the write seam ignoring the
ACL, the realm dropped from the scope, the revocation check removed, and the
trie never installed each turn the intended tests red and no others.
Full solution suite green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Two independent defects surfaced in review of #520 stage 3.
#532 — Cluster:Roles had no OTOPCUA_ROLES fallback, despite two doc sites
promising one. A node configured only through the env var — which drives ALL of
Program.cs's conditional wiring, so it reads as the authoritative knob — bound an
empty Roles array and joined the cluster carrying no Akka roles at all. It ran,
served, and logged nothing wrong, while central reported "no driver-role nodes
are Up" and every deployment to it timed out. Not hypothetical: the field fix on
wonder-app-vd03 was to hand-add Cluster__Roles__0/1 to a Windows service's
registry environment. Anyone reading the code to diagnose that box would have
concluded from the XML doc that the fallback was already applied.
AddOtOpcUaCluster now post-configures AkkaClusterOptions through the shared
RoleParser.ResolveEffectiveRoles, so BuildClusterOptions (Akka's own member
roles), ClusterRoleInfo, the ControlPlane singleton scope and
ConfigDbApiKeyStore's ClusterId all see one answer. Two sibling validators
(ConfigSource, Telemetry) read Cluster:Roles directly and were exempting the
exact node they exist for — an env-only driver-only node has no ConfigDb and
could sit on ConfigSource:Mode=Direct unchallenged. They resolve the same way now.
Also collapsed the four open-coded `Roles.FirstOrDefault(IsClusterRole)` +
ClusterIdFromRole copies into RoleParser.ClusterIdFromRoles. They had drifted:
three classified the cluster role ordinally, so a verbatim-bound `Cluster-SITE-A`
yielded a null ClusterId — and a null ClusterId applies NO cluster filter, which
made #528's cross-cluster over-grant guard fail OPEN. IsClusterRole is now
case-insensitive; the ClusterId suffix keeps its authored spelling, because
normalizing it would silently rename the Akka role of a running deployment.
#533 — three string-identity rules in one authorization path. AclScopeRef used
default (ordinal) equality while the trie builder filtered ClusterId and hashed
principals OrdinalIgnoreCase; the deploy gate's cross-cluster check was ordinal
too, which was the worse half — it could reject a whole deploy over a ClusterId
case difference the runtime would have accepted. It only worked because resolver
and builder were handed the same NodeAcl instances.
The rule is now stated once, on AclScopeRef, and applied throughout:
identifiers (ScopeId, ClusterId, Principal) compare OrdinalIgnoreCase; path
segments compare Ordinal. AclScopePathResolver re-keys its own maps rather than
documenting a comparer contract for callers, so the two independent producers —
EF at the gate, artifact JSON on the node — cannot diverge from the outside.
PermissionTrieNode.Children stays Ordinal: those are browse names, and Speed /
speed are two nodes that can coexist under one device.
Split PermissionTrieBuildDiagnosticReason.RetiredScopeKind out of
UnresolvedScope. A legacy FolderSegment row and a typo'd id had the same message
and different remedies.
Falsified: five mutations, each reddening only its own guard — remove the
PostConfigure (3 tests), revert IsClusterRole to ordinal (3), revert AclScopeRef
equality (1), revert the resolver's map comparer (6), revert the gate's
comparisons (2), remove the FolderSegment short-circuit (1).
Test isolation: OTOPCUA_ROLES is process-global and xUnit runs classes in
parallel, so the three validator suites that now resolve effective roles share a
non-parallel collection. Without it, cases passing no roles would intermittently
observe another class's env var.
Full solution build clean; full suite green apart from two pre-existing failures
confirmed identical on 3a6b2a1d (PrimaryGateFailoverTests, and
DriverProbeRegistrationTests expecting 10 probe types where 12 now register)
plus the AbLegacy integration suite, which needs a docker fixture.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Makes the /raw device tree addressable by an ACL rule, and builds the
scope-path producer PermissionTrieBuilder has accepted since it was written
but which had no caller anywhere.
The plan for this stage said "add NodeHierarchyKind.Raw, whose path is the
RawPath segments Folder/Driver/Device/TagGroup/Tag". That shape is wrong.
RawFolder.ParentRawFolderId and TagGroup.ParentTagGroupId are both
self-referencing, so folders and tag groups each nest without bound and a real
RawPath is folder…/Driver/Device/group…/Tag. NodeScope was fixed-arity and
PermissionTrie.CollectMatches was five hardcoded steps, so the scope model
could not address a real raw tree at all. NodeScope is now {ClusterId, Kind,
Segments} and the walk is a segment loop. The trie STRUCTURE was already
variable-depth (NodeAclPath.Segments is a list, Children is a recursive
dictionary), so only the walker and the scope record had to change.
Four decisions worth keeping:
- The namespace level became the realm. Level 1 gated on NodeScope.NamespaceId,
but v3 has no Namespace entity — the snapshot was retired when the address
space went dual-namespace, and the identifier appears nowhere in production
outside NodeScope. So the level could only ever fail to match, which silently
made every sub-cluster grant unreachable while cluster grants kept working.
It is now a realm token derived from the scope's Kind, not carried on it, so
the walk and the producer cannot disagree about its spelling.
- Provenance comes from the granting row's own ScopeKind, recorded on TrieGrant
at build time. Depth-derived provenance could not name a raw level at all,
since the depth of "Device" varies with folder nesting.
- The trie comparer moved from OrdinalIgnoreCase to Ordinal. Raw sibling-name
uniqueness is ordinal, so Speed and speed can both exist under one device and
a grant on either would have matched both.
- The null-scopePaths "deterministic test mode" is deleted. It planted
unresolvable rows at a direct child of the trie root, a position the walker
cannot reach, so the grant looked placed and could never match — and every
test using that mode proved nothing about production placement. scopePaths is
now required, and a row that does not resolve is dropped with a diagnostic.
A scope resolves to a LIST of paths: a raw tag projected into N equipments is
reachable at N+1 NodeIds and a write through any of them lands on the same
driver reference, so a Tag-scoped grant covering only the raw node would be
bypassable through a UNS one. The lookup is keyed on (ScopeKind, ScopeId), not
ScopeId alone — ScopeId has no FK and no type discriminator, so an id collision
across entity types would hand a grant the wrong node's paths.
Also closed here, all found by review rather than asked for:
- NodeAcl rows had no deploy-time validation of any kind and were not reachable
from DraftValidator at all (DraftSnapshot carried no NodeAcls). They are now
gated for dangling ids, cross-cluster scopes, blank/contradictory ScopeId, the
retired FolderSegment kind, and undefined ordinals.
- ConfigDbApiKeyStore conferred roles from Cluster-scoped grants authored in ANY
cluster, because it filtered on scope kind with no ClusterId predicate while
ArtifactApiKeyStore read an already-cluster-filtered snapshot. On a fused
admin+driver node that was a cross-cluster over-grant. Its doc comment claimed
the two stores matched.
- The two AdminUI ACL banners still said enforcement was LDAP-group -> role
mapping, which stages 1/2 made false in both directions.
- The /apikeys page was never classified in PageAuthorizationGuardTests, so that
test has been red since the key-minting commit merged — I did not run
AdminUI.Tests before merging stage 4.
Falsified rather than assumed: 8 mutations against the trie tests, 5 against the
deploy gate (including one proving a cross-cluster row is not silently
mis-diagnosed as unresolved), a staged red->green for the over-grant regression,
and the realm gate, the comparer and the malformed-path guard each broken
individually to confirm only their own tests go red.
Node ACLs are still not ENFORCED — nothing constructs a NodeScope or calls the
evaluator yet. That is stages 5 and 6; this stage makes the model able to
express what those will enforce.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
A driver-only node has had no ConfigDb since per-cluster mesh Phase 4, so it
registered NullOpcUaUserAuthenticator and denied every UserName token. Its keys
now ride the deployment artifact instead.
- ConfigComposer snapshots ApiKeys alongside the NodeAcls it already carried.
Only the peppered HMAC ships, never a secret, and the pepper is node config
that is deliberately not in the artifact.
- DeploymentArtifact.ParseDataPlaneAuthority decodes both node-side. Grants are
scoped to the node's own cluster; keys are not, because ApiKey carries no
ClusterId — a key is a fleet-wide credential and the ACL rows are what bind it.
- OpcUaPublishActor installs them on DataPlaneAuthorityState on every apply,
BEFORE its plan.IsEmpty early-return: an artifact that only revokes a key
produces an empty address-space plan, so feeding after that return would leave
the revoked key working until an unrelated structural change redeployed.
- ArtifactApiKeyStore serves them to session activation; a node with a ConfigDb
keeps ConfigDbApiKeyStore, whose per-request read is what makes revocation
immediate there.
- OtOpcUaNodeManager gains DataPlaneAuthority — the cluster identity blocker #4
named. NodeScope.ClusterId is required and the evaluator fails closed on a
mismatch, so until now no scope it built could have evaluated. Nothing reads it
until stages 5/6.
The generation is a node-local COUNTER driven by the RevisionHash, not derived
from its bits. PermissionTrieCache treats the highest GenerationId as current, so
a generation taken from hash bytes would install a lower number about half the
time and the cache would go on serving the previous deployment's ACLs, silently.
Sessions are not stamped with it yet — that belongs with the first seam that
reads a UserAuthorizationState, in stage 5, rather than being built now and
consulted by nothing.
Live-gated on docker-dev against site-a-1 (driver-only, no ConfigDb): key
authenticates, wrong secret denied, both halves of the revocation asymmetry
observed, and a SITE-A grant confirmed not to reach site-b-1. Key minted with an
independent Python HMAC so the verifier was checked against something other than
itself. Rig left clean.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Pulled forward from stage 7 because without it the feature is unusable and
un-live-gateable: there was no way to create a key short of hand-inserting a row
and computing an HMAC by hand.
The minting logic is an IApiKeyAdminService in Core.Authorization, NOT logic in
the Razor page — this repo has no bUnit, so anything only a page can reach is
untestable, and minting credentials is not where that belongs. The page is a thin
shell over it.
Moved to Core.Authorization so both ends can reach them: ApiKeyOptions (was in
Host) and OpcUaPrincipals (was in OpcUaServer.Security). The OPC UA server
resolves principals; the AdminUI must refuse to mint a key claiming a reserved
one. OpcUaServer already gained a Core reference in stage 1b, so no new edge —
AdminUI gains one.
ApiKeyOptions is now bound UNCONDITIONALLY rather than in the driver branch: a
driver node needs the pepper to VERIFY keys and an admin node needs it to MINT
them, and getting that wrong on either side is a silent authentication failure
rather than a startup error.
Validation rules, all covered: the reserved 'anonymous' principal is refused
case-insensitively (so "ANONYMOUS" cannot slip past into a distinct principal
that the evaluator would then match case-INsensitively against anonymous's
grants); blank name/principal; a principal longer than the column, rejected
rather than truncated, because silent truncation yields a key that authenticates
and is granted nothing with no visible cause; a past expiry. Generation runs
BEFORE the DbContext is opened so an unconfigured pepper surfaces as a clear
error instead of a stored row whose hash nothing can verify.
The page shows the plaintext ONCE and says so; it is held in component state and
written nowhere else, because the server keeps only a hash. Revocation is flagged
in the UI as taking effect only after the next deployment — driver-only nodes
have no ConfigDb, so keys ride the artifact.
FleetAdmin-only (Administrator), not ConfigEditor: this mints credentials.
Core.Tests 68 green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Stage 1b. Still no enforcement wired — this adds the credential model the
identity cutover (stage 2) will read.
ApiKey holds one principal per key, deliberately. The reason to separate keys
from principals is so many keys share a grant set and rotating a key never
touches an ACL row, and that holds with one principal per key since nothing stops
two keys naming the same principal. A key belonging to several principals at once
buys little and costs a join table. UserAuthorizationState.Principals stays a
list, so adding it later needs no ACL-side change.
ApiKeySecret is the security-critical half and is pure, so it is tested on its
own: otua_<keyId>_<secret>, peppered HMAC-SHA256, FixedTimeEquals. Notes why a
plain HMAC rather than an iterated KDF (the secret is 32 bytes of CSPRNG, and the
call runs inline on session activation, which the SDK serializes server-wide
behind its SessionManager event lock), and fails closed on an unset pepper rather
than hashing under an empty key.
Found and fixed a bug in my own first draft: the key id was url-safe base64,
whose alphabet maps '/' to '_' — the very character TryParse splits on. Roughly
one key in five would have had its id truncated at a random position, an
intermittent auth failure reproducing only for SOME keys. The id is hex now, and
the guard is falsified: reverting the alphabet reds both
The_generated_key_id_never_contains_the_separator and, as a bonus signal,
Two_generated_keys_never_collide, because truncated ids collide.
ApiKeyId is NOT NULL, unlike the nullable NodeAclId precedent whose filtered
unique index would admit two NULL ids. Fine for a config row, not for a
credentials lookup key.
OpcUaServer.csproj now references Core. Until now it did not, so
IPermissionEvaluator was literally uncallable from the node manager — the place
enforcement has to happen. No new transitive package surface.
Schema doc updated for both, and its NodeAcl block flagged for pre-existing drift
(it still documents the GenerationId column v3 dropped) without fixing it here.
Core 247, OpcUaServer 364, Configuration 128 — all green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Stage 1a of the API-key ACL design. Behaviour-neutral: the trie walker already
matched an opaque string, so only what produces that string changes. No
enforcement is wired by this commit and none of the two live role gates move.
Renamed through the ACL model: NodeAcl.Principal, TrieGrant.Principal,
MatchedGrant.Principal, AuthorizationDecision.Principal,
UserAuthorizationState.Principals, and PermissionTrie.CollectMatches's parameter.
LdapGroupRoleMapping is a DIFFERENT entity that also has an LdapGroup column and
is deliberately untouched — it backs AdminUI login, which keeps LDAP.
The migration is the interesting part. EF scaffolds RenameColumn/RenameIndex, so
rows survive — but it cannot know that dbo.sp_ComputeGenerationDiff checksums
NodeAcl.LdapGroup. That proc is created in V3Initial.StoredProcedures and never
redefined since, and SQL Server does not validate a procedure body when a column
is renamed out from under it: the object keeps reporting healthy and fails at
CALL time with Invalid column name, silently breaking config-drift detection.
Verified against the live dev DB that the deployed proc really does reference the
column. The migration therefore re-emits the proc, as one parameterised template
shared by Up and Down so the two cannot drift.
Prose that describes what is enforced TODAY (fleet-wide LDAP group -> role) is
left alone, because it is still accurate until stage 2 lands.
UserAuthorizationState's refresh machinery is retained unchanged but now carries
a note that it is LDAP-shaped and largely vestigial here: an API-key principal
set cannot change out of band, since keys and ACL rows both ride the deployment
artifact, so the generation stamp rather than the freshness timer is the real
invalidation signal.
41 authorization tests pass; solution builds clean.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Both `IDriver` members had no production call site. Their only consumer was
`MemoryTracking`, deleted with the driver-tier recycle machinery in #522, and
`GetMemoryFootprint`'s doc-comment claimed "polled every 30s by Core" — which
had not been true since v3, even before #522 removed the poller.
Owner decision on #525 was (a) delete rather than (b) build the process-wide
watchdog that would consume them. The driver-side implementations are in git
history and were mostly reusable; what was always missing is the trigger, and a
member nobody calls is how the misleading doc-comment survived this long.
Removed from `IDriver`, all 12 driver implementations, and every test stub —
plus the support code that existed only to feed them, which is the part a
member-only deletion would have left behind as dead weight:
- MTConnect's three per-entry byte-weight constants, its cached
`ProbeModelDataItemCount`, and `CountDataItems` — a full device-tree walk run
on every cache fill to populate a number with no remaining reader.
- OpcUaClient's `_discoveredNodeCount` field and the discovery-pass write to it.
- MQTT's `ApproxBytesPerAuthoredTag`.
Two behavioural notes, neither a change in what the software does:
- `AbCipTemplateCache` now has no production reset path. Its only caller was
AB CIP's flush, which itself had no caller — so nothing changes except how
visible the fact is. Recorded on the type, along with the correction that its
comment's claim about a device `ReinitializeAsync` also flushing it was never
true. A UDT redefined on a live controller keeps its stale shape until the
driver instance is rebuilt.
- MTConnect's `/probe` cache is still dropped, by `InvalidateProbeModel()` on an
Agent restart. That is now `internal` rather than `private` so the tests which
used the flush purely to empty the cache drive the real path instead — they
now cover a case that actually occurs in production. Several unrelated doc
comments cited `FlushOptionalCachesAsync` as the canonical CAS example and are
repointed to it.
Tests: the ones that existed only to assert on the removed members are deleted
(3 OpcUaClient, 3 Galaxy, 2 MQTT, 1 MTConnect, 1 AbCip). Worth noting one was
vacuous — `FlushOptionalCachesAsync_resets_footprint_counter` asserted a
footprint of 0 before the flush and 0 after with no discovery in between, so it
would have passed against a flush that did nothing at all.
Full suite green except 7 failures verified identical on master before and
after (4 AbLegacy integration with the compose profile down, 3
Host.IntegrationTests), checked by name rather than assumed.
Docs corrected where they described this as live: `DriverLifecycle.md` (which
carried the #525 placeholder), `drivers/AbCip.md`, `drivers/AbLegacy.md`, and
`v2/driver-stability.md` — whose "polls this every 30 s" + per-driver
`DriverConfig` memory budget were never built as written and are now banner-ed
as design intent rather than description.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
The tier system was documented, operator-authorable, and inert. Deleted rather than activated,
because its premise is gone rather than merely unused.
"Tier C" meant a driver running out-of-process behind an IDriverSupervisor that could restart its
Host without tearing down the OPC UA session. No such process exists anywhere: Galaxy reaches
MXAccess over gRPC to the external mxaccessgw sidecar (PR 7.2 retired the in-process
Galaxy.Host/Proxy/Shared projects) and FOCAS has run in-process since its managed wire client
landed 2026-04-24. Consistently, IDriverSupervisor had ZERO implementations and there was nothing
for one to implement against.
The issue understated the inertness. It says the Tier-C-only protections never engaged, implying
the Tier A/B parts did. They did not: nothing constructs MemoryTracking, MemoryRecycle or
ScheduledRecycleScheduler outside their own unit tests, so the whole Core/Stability recycle layer
was dead — meaning option (a), "pass real tiers", was never a flag flip. It would have meant
writing the wiring that never existed AND arming it.
Deleted: MemoryTracking, MemoryRecycle, ScheduledRecycleScheduler, IDriverSupervisor, the
vestigial DriverTypeRegistry (referenced only by its own tests), and the RecycleIntervalSeconds
knob — which the AdminUI let an operator author and the parser validated while it configured
nothing.
DriverTier itself SURVIVES and is load-bearing: DriverResilienceOptions.GetTierDefaults supplies
the real per-capability timeout/retry/breaker policies via DriverFactoryRegistry.GetTier and
DriverCapabilityInvokerFactory. Only the isolation-and-recycle layer above it is gone.
Deliberately NOT deleted:
- WedgeDetector came along in the same directory and is equally dead in production, but it is
tier-agnostic and is not recycle machinery — it only shares the folder. Restored rather than
swept up in a decision that was not about it.
- IDriver.GetMemoryFootprint() and FlushOptionalCachesAsync() lose their only consumer here.
Removing them touches all 12 drivers and every test stub, so they are documented as
consumerless and filed as #525 instead of buried in this diff.
Compatibility: a deployed ResilienceConfig blob still carrying "recycleIntervalSeconds" parses
cleanly (unknown keys are ignored — guarded by a new test, because a blob that suddenly failed to
parse would fall back to tier defaults and silently discard the operator's real overrides), and
the AdminUI's preserve-unknown-keys bag keeps the key rather than rewriting stored config on an
unrelated edit.
The 01/U-6 knob-inertness guard carried an explicit carve-out admitting RecycleIntervalSeconds was
dormant and out of scope; that carve-out is now gone, so the expected set is literally what the
test's name claims.
Note: Host.IntegrationTests has 2 failures (DriverProbeRegistrationTests.is_idempotent,
PrimaryGateFailoverTests) — verified pre-existing by reproducing both on clean master dc9d947b.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Fix TESTS only (production already green) so the four in-scope test projects
compile + pass under the v3 dark address space:
- Core.Abstractions.Tests: drop AllowedNamespaceKinds/NamespaceKindCompatibility
from DriverTypeMetadata; rewrite EquipmentTagRefResolver tests to the single-func
RawPath-lookup contract (no parseRef/transient cache).
- Core.Tests: rewrite EquipmentNodeWalkerTests to EquipmentNamespaceContent(Areas,
Lines,Equipment,VirtualTags?,ScriptedAlarms?) — folders + VirtualTags +
ScriptedAlarms only; raw-tag variables dark (skipped Batch-4 placeholder). Drop
retired Equipment.DriverInstanceId.
- Runtime.Tests: rewrite golden corpus + TagConfigCorpusParityTests to the v3
RawPath/RawTagEntry round-trip (byte + TagConfigIntent parity; EquipmentTags dark).
Salvage VirtualTag + ScriptedAlarm artifact parity to the new Compose signature.
Retire the equipment-tag-materialization parity files (Array/Historize/Alias) and
the equipment-device-binding DeviceHost parity to documented skipped placeholders.
Skip 34 dark equipment-tag routing/value/alarm/write/discovery actor tests with a
shared DarkAddressSpaceReasons constant. Re-key cluster-scoped tests to v3 UNS
line->area attribution.
- ControlPlane.Tests: ConfigComposer round-trips re-keyed to RawFolder/Device (no
Namespaces); tag-config gate resolves driver via Device; deploy-gate collision test
re-keyed to UnsEffectiveNameCollision; drop retired BadCrossClusterNamespaceBinding
case.
The DriverInstance.ResilienceConfig column was authored in AdminUI, persisted to the
entity, and serialized into the deployment artifact by ConfigComposer — but the runtime
read path dropped it: DriverInstanceSpec didn't carry it and the invoker factory always
passed null, so every driver got tier defaults regardless of its configured overrides
(a silent dead-config gap — #10's residual sub-finding).
Read-path plumbing (write side was already complete):
- DriverInstanceSpec gains ResilienceConfig; DeploymentArtifact.TryReadSpec reads the column.
- IDriverCapabilityInvokerFactory.Create takes resilienceConfigJson; DriverHostActor.SpawnChild
threads spec.ResilienceConfig; the concrete factory parses it (ParseOrDefaults, layering on the
tier), logs any parse diagnostic (never throws), and builds the invoker with the merged options.
- Invalidate-on-change: the pipeline cache keys on (instance, host, capability) and ignores options
on a hit, so Create() now Invalidate()s the instance's cached pipelines first (no-op on first
spawn) — a respawn with changed options rebuilds them.
- DriverSpawnPlanner treats a ResilienceConfig change as a stop+respawn (the invoker/options are
bound to the child at spawn); a pure DriverConfig change stays an in-place delta (no reconnect).
- Host DI passes a logger to the factory for the parse diagnostic.
Verification (deterministic): factory Create applies a retryCount:0 override to actual execution
(control test proves tier default retries), invalidates the instance's cache on re-create (scoped —
sibling survives), malformed config logs+falls-back; planner respawns on ResilienceConfig change
(incl null→json) and stays delta on a pure config change; artifact parse carries/omits the column.
Core.Tests 243 (+5), Runtime.Tests 363 (+5), Host builds clean.
Remediates RESILIENCE-DISPATCH-GAP (surfaced by the 07/C-1 analyzer): the Phase 6.1
CapabilityInvoker resilience pipeline (retry / breaker / bulkhead / telemetry) was
constructed only in tests — the production dispatch layer called driver-capability
methods directly, bypassing it entirely.
The filed plan (thread CapabilityInvoker directly into DriverInstanceActor) is
INFEASIBLE: Runtime is deliberately Polly-free (references Core.Abstractions, not
Core — the same boundary IDriverFactory documents). So this introduces a seam,
mirroring IDriverFactory exactly:
- Core.Abstractions: IDriverCapabilityInvoker (+ NullDriverCapabilityInvoker
pass-through) and IDriverCapabilityInvokerFactory (+ null factory).
- Core: CapabilityInvoker now implements the interface; new
DriverCapabilityInvokerFactory builds a per-instance invoker over the
process-singleton pipeline builder + status tracker + tier resolver.
- Runtime: DriverInstanceActor takes an IDriverCapabilityInvoker (default =
pass-through) and routes all 6 dispatch sites (write, alarm-ack, subscribe,
unsubscribe, alarm-subscribe, discover) through it; per-host key resolved via
IPerCallHostResolver for single-ref calls, driver-instance key for bulk calls.
DriverHostActor builds + injects the real invoker per spawned driver.
- Host: DriverFactoryBootstrap registers the tracker, pipeline builder, and
concrete factory (needs DriverFactoryRegistry.GetTier). Runtime SCE resolves the
factory from DI like IDriverFactory; pass-through on nodes without it bound.
- Analyzer: IDriverCapabilityInvoker.ExecuteAsync/ExecuteWriteAsync are now
recognized wrapper homes (interface-typed invoker calls must suppress OTOPCUA0001
too). All 6 RESILIENCE-DISPATCH-GAP pragmas removed — the analyzer (an error in
Runtime) is now the standing regression guard.
Also adds retry / breaker-open / breaker-close LOGGING to the pipeline builder —
the operator-facing observability surface (the Admin /hosts reader, Phase 6.1
Stream E.2/E.3, was never built, so the pipeline otherwise runs silently).
Scope notes:
- Tier-DEFAULT policy only: DriverInstanceSpec (the deploy artifact) does not carry
the per-instance ResilienceConfig JSON, so overrides aren't applied yet (tracked
follow-up: plumb ResilienceConfig through the composer/artifact).
- GenericDriverNodeManager's call is NOT wired: that class is test-only scaffolding
(zero production references — the Server has its own address-space path); its
pragma is re-annotated accordingly, not left as a "tracked gap".
Verification (unit + analyzer; live behavioral gate still pending — see FOLLOWUP-10):
- Negative control: unwrapping any site fails the Runtime build with OTOPCUA0001.
- New Runtime guard (recording invoker): write routes via ExecuteWriteAsync with the
IPerCallHostResolver host; subscribe via ExecuteAsync — proves runtime routing.
- New analyzer test: the interface is a valid wrapper home.
- New pipeline-builder test: retry events are logged.
- Full solution builds clean (0 errors); Runtime.Tests 357, Core.Tests 238,
Analyzers.Tests 32 all green; pass-through default keeps existing dispatch tests
byte-for-byte unchanged.
Adds <summary>, <param>, <typeparam>, and <inheritdoc/> tags to public
members surfaced by commentchecker — resolves 5,847 of 5,869 issues
(99.6%) across three /fixdocs passes.
Adds Directory.Packages.props (ManagePackageVersionsCentrally) and
Directory.Build.props (net10.0/nullable/implicit usings/LangVersion latest).
Strips Version attributes from every csproj PackageReference and consolidates
versions into the central file.
Side fixes (necessary to keep the build green on .NET SDK 10.0.105 on macOS):
- Microsoft.CodeAnalysis.CSharp{,.Workspaces}: 5.3.0 -> 5.0.0. The 5.3.0
analyzer DLL references compiler 5.3.0.0 and the local SDK ships compiler
5.0.0.0, producing CS9057 on every project that loaded the Analyzers
output. Master itself was broken on this machine pre-change.
- Server + Server.Tests pin OPCFoundation.NetStandard.Opc.Ua.{Configuration,
Client} to 1.5.374.126 via VersionOverride, matching Opc.Ua.Server's
pin. Mixing 1.5.378.106 Opc.Ua.Core transitively with 1.5.374.126
Opc.Ua.Server breaks CustomNodeManager2 override signatures
(CS0115 on LoadPredefinedNodes/Browse/HistoryRead*) and CS7069 in
the tests. The pin disappears when the legacy Server project is
deleted in Task 56.
- Client.UI + Client.UI.Tests: NuGetAuditSuppress for
GHSA-xrw6-gwf8-vvr9 (Tmds.DBus.Protocol 0.20.0 reaches both projects
transitively from Avalonia.Desktop on Linux/macOS only).
Deviation from the plan: TreatWarningsAsErrors=true is NOT set in
Directory.Build.props because the pre-v2 Admin/Server test projects carry
~240 xUnit1051 analyzer warnings that would fail the build. New v2 projects
opt in via their own csproj; the global flag can return once the legacy
projects are deleted in Task 56.
- Core-004: add ConfigureAwait(false) to DriverHost.RegisterAsync /
UnregisterAsync / DisposeAsync.
- Core-008: rewrite the BuildAddressSpaceAsync XML doc to correctly name
the caller (OpcUaApplicationHost.PopulateAddressSpaces) that owns the
per-driver isolation.
- Core-009: snapshot DriverResilienceOptions once per non-idempotent write
in CapabilityInvoker.ExecuteWriteAsync.
- Core-010: switch DriverResilienceOptions.Resolve to TryGetValue with a
diagnostic error message when a tier table is missing a capability.
- Core-011: add an optional diagnostic callback to PermissionTrieBuilder
so production callers can surface scope-path mismatches.
- Core-012: correct the stale WedgeDetector ctor summary and add the
Reconnecting row to DriverHealthReport's state matrix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SubscribeAsync now wraps each driver handle in a private HostBoundHandle
that carries the resolved host name. UnsubscribeAsync unwraps it and
routes through the recorded host's resilience pipeline, correctly
charging the subscription's originating host's circuit breaker/bulkhead
instead of always using the default host. Falls back to the default
host for handles not created by this invoker. Two regression tests
added; update findings.md Open count from 10 to 6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BuildAddressSpaceAsync now checks _disposed (throws ObjectDisposedException)
and tears down the previous alarm forwarder + clears the sink registry
before re-walking, so a Galaxy-redeploy rebuild does not leak the old
forwarder and double-deliver alarm transitions. Three regression tests
added: double-build does not double-fire, sink count is correct after
rebuild, and post-dispose call throws.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Change ClusterEntry from sealed record to sealed class so TryUpdate
uses reference equality for the CAS comparison. Prune now uses a
read-compute-TryUpdate retry loop that restarts when a concurrent
Install updates the entry between the read and the write, preventing
a race that could silently drop the just-installed newest generation.
Two regression tests added to PermissionTrieCacheTests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add FolderSegment member to NodeAclScopeKind; update WalkSystemPlatform
to report NodeAclScopeKind.FolderSegment (not Equipment) for each
visited Galaxy folder level, so MatchedGrant.Scope in
AuthorizationDecision.Provenance correctly distinguishes Galaxy folder
grants from UNS Equipment grants in the audit trail and Admin UI
diagnostics. Three regression tests added to PermissionTrieTests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Core-001: swap the authorization-cache defaults so
MembershipFreshnessInterval (5 min, inner re-resolve trigger) is
strictly less than AuthCacheMaxStaleness (15 min, fail-closed
ceiling), so NeedsRefresh's warm-refresh path is reachable.
Core-002: TriePermissionEvaluator.Authorize now compares the trie's
GenerationId against the session's AuthGenerationId and re-fetches the
session's bound generation on mismatch, failing closed when that
generation has been pruned.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group all 69 projects into category subfolders under src/ and tests/ so the
Rider Solution Explorer mirrors the module structure. Folders: Core, Server,
Drivers (with a nested Driver CLIs subfolder), Client, Tooling.
- Move every project folder on disk with git mv (history preserved as renames).
- Recompute relative paths in 57 .csproj files: cross-category ProjectReferences,
the lib/ HintPath+None refs in Driver.Historian.Wonderware, and the external
mxaccessgw refs in Driver.Galaxy and its test project.
- Rebuild ZB.MOM.WW.OtOpcUa.slnx with nested solution folders.
- Re-prefix project paths in functional scripts (e2e, compliance, smoke SQL,
integration, install).
Build green (0 errors); unit tests pass. Docs left for a separate pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Joseph Doherty