Review of the ClusterClient-deletion docs sweep (406b1ae5) flagged two
pre-existing staleness items in the swept files, unrelated to ClusterClient
but caught while touching the same passages:
- docs/Telemetry.md: the Phase 5 telemetry section's Status line still said
"code-complete ... live gate has not yet run" — the gate PASSED 2026-07-23
(merge 35552a96), already recorded in CLAUDE.md. Updated to match.
- docs/v2/config-db-schema.md: the AkkaPort/GrpcPort note asserted in the
present tense that "central shares a gossip ring with every node," which
Phase 6 (2026-07-24) ended — central now runs one 2-node Akka mesh per
application Cluster and reaches a site pair only over gRPC. Reworded the
premise to past tense so it no longer contradicts the ClusterClient-deleted
parenthetical sitting right after it.
docs-only; dotnet build ZB.MOM.WW.OtOpcUa.slnx stays green (0 errors).
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Task 3.2 of the ClusterClient->gRPC migration. Sweeps the docs that still
described the Akka ClusterClient transport as live, following its Phase 3
deletion (this branch): MeshTransport:Mode is now Dps (pair-local DPS) or
Grpc (cross-mesh unary calls) only, ClusterClient fails validation loudly,
CentralContactPoints is deleted (not superseded), and the split-topology
validator requires exactly Grpc rather than ClusterClient.
- docs/Configuration.md: rewrite the MeshTransport section — table,
dark-switch/PSK-seeding prose, listener sharing (:4056 driver-role,
:4055 admin-role), remove the "shipped dark, cutover pending" framing.
- docs/Redundancy.md: replace the Dps/ClusterClient transport table with
Dps/Grpc, fix the Phase 6 CentralCommunicationActor + validator bullets.
- docs/Telemetry.md: fix stale ClusterClient cross-refs, add a note that
the driver-role telemetry listener also hosts NodeCommandService.
- docs/v2/config-db-schema.md: one-line "(retired: ...)" annotation on the
historical AkkaPort/GrpcPort note — left as a historical record otherwise.
- CLAUDE.md: rewrite "## Mesh command transport (MeshTransport)" for the
post-deletion world (no ClusterClient/receptionist, per-node gRPC dial,
fire-and-forget semantics unchanged) and fix two Phase 6 bullets.
deferment.md has no mesh-transport mentions; left untouched.
docs-only change; `dotnet build ZB.MOM.WW.OtOpcUa.slnx` stays green (0
errors) after the edit.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Stage 1b. Still no enforcement wired — this adds the credential model the
identity cutover (stage 2) will read.
ApiKey holds one principal per key, deliberately. The reason to separate keys
from principals is so many keys share a grant set and rotating a key never
touches an ACL row, and that holds with one principal per key since nothing stops
two keys naming the same principal. A key belonging to several principals at once
buys little and costs a join table. UserAuthorizationState.Principals stays a
list, so adding it later needs no ACL-side change.
ApiKeySecret is the security-critical half and is pure, so it is tested on its
own: otua_<keyId>_<secret>, peppered HMAC-SHA256, FixedTimeEquals. Notes why a
plain HMAC rather than an iterated KDF (the secret is 32 bytes of CSPRNG, and the
call runs inline on session activation, which the SDK serializes server-wide
behind its SessionManager event lock), and fails closed on an unset pepper rather
than hashing under an empty key.
Found and fixed a bug in my own first draft: the key id was url-safe base64,
whose alphabet maps '/' to '_' — the very character TryParse splits on. Roughly
one key in five would have had its id truncated at a random position, an
intermittent auth failure reproducing only for SOME keys. The id is hex now, and
the guard is falsified: reverting the alphabet reds both
The_generated_key_id_never_contains_the_separator and, as a bonus signal,
Two_generated_keys_never_collide, because truncated ids collide.
ApiKeyId is NOT NULL, unlike the nullable NodeAclId precedent whose filtered
unique index would admit two NULL ids. Fine for a config row, not for a
credentials lookup key.
OpcUaServer.csproj now references Core. Until now it did not, so
IPermissionEvaluator was literally uncallable from the node manager — the place
enforcement has to happen. No new transitive package surface.
Schema doc updated for both, and its NodeAcl block flagged for pre-existing drift
(it still documents the GenerationId column v3 dropped) without fixing it here.
Core 247, OpcUaServer 364, Configuration 128 — all green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Phase 1 gate step 4 failed: setting Enabled = 0 to take a node out of service
returns 422 ClusterEnabledNodeCountMismatch, because
DraftValidator.ValidateClusterTopology requires the enabled-node count to equal
ServerCluster.NodeCount. On a Warm/Hot pair — every cluster on the rig, and
every cluster in the target topology — Enabled can therefore never be the
maintenance hatch. The plan called step 4 "the check that makes step 3
acceptable to ship", so Task 3's behaviour change was not shippable as it stood:
a node down for maintenance would block every deployment to its cluster.
The two rules were each reasonable and contradictory together — the validator
reads Enabled as "part of the declared topology", Phase 1 additionally read it
as "expect an ack". Split the meanings rather than weaken either rule:
Enabled part of the declared topology (validator, untouched)
MaintenanceMode expected to participate now (coordinator + reconciler)
Rejected alternatives: counting configured rather than enabled nodes (drops the
guard against booting a pair into InvalidTopology); downgrading the rule to a
warning (weakens a deploy gate for everyone); shipping with no hatch.
Sabotage: dropping !n.MaintenanceMode from the coordinator query turns the new
test red. It also asserts both nodes remain Enabled, so the fix cannot quietly
regress to disabling the row after all.
Configuration.Tests 95/95, ControlPlane.Tests 101/101, solution builds clean.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
- docker-dev seed: AkkaPort = 4053 on all six ClusterNode rows, so a freshly
seeded rig matches a migrated one. GrpcPort left null.
- config-db-schema.md: both columns, why AkkaPort is NOT NULL/4053 and GrpcPort
is nullable, the unenforced duplication + its reconciler, and Enabled's new
second meaning as the deploy path's expected-ack set.
- Configuration.md: Cluster:Port / PublicHostname now flag that they are stored
twice, with the "update the row too" instruction and why the drift is silent.
- design doc §7: Phase 1 marked done, plus a "Phase 1 as shipped" note recording
both deviations rather than leaving the sketch reading as what happened.
- program plan: Phase 1 marked done; AdminUI node edit explicitly deferred.
- CLAUDE.md: the deploy-path behaviour change and its three consequences.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
ACL design defines NodePermissions bitmask flags covering Browse / Read / Subscribe / HistoryRead / WriteOperate / WriteTune / WriteConfigure / AlarmRead / AlarmAcknowledge / AlarmConfirm / AlarmShelve / MethodCall plus common bundles (ReadOnly / Operator / Engineer / Admin); 6-level scope hierarchy (Cluster / Namespace / UnsArea / UnsLine / Equipment / Tag) with default-deny + additive grants and Browse-implication on ancestors; per-LDAP-group grants in a new generation-versioned NodeAcl table edited via the same draft → diff → publish → rollback boundary as every other content table; per-session permission-trie evaluator with O(depth × group-count) cost cached for the lifetime of the session and rebuilt on generation-apply or LDAP group cache expiry; cluster-create workflow seeds a default ACL set matching the v1 LmxOpcUa LDAP-role-to-permission map for v1 → v2 consumer migration parity; Admin UI ACL tab with two views (by LDAP group, by scope), bulk-grant flow, and permission simulator that lets operators preview "as user X" effective permissions across the cluster's UNS tree before publishing; explicit Deny deferred to v2.1 since verbose grants suffice at v2.0 fleet sizes; only denied OPC UA operations are audit-logged (not allowed ones — would dwarf the audit log). Schema doc gains the NodeAcl table with cross-cluster invariant enforcement and same-generation FK validation; admin-ui.md gains the ACLs tab; phase-1 doc gains Task E.9 wiring this through Stream E plus a NodeAcl entry in Task B.1's DbContext list.
Dev-environment doc inventories every external resource the v2 build needs across two tiers per decision #99 — inner-loop (in-process simulators on developer machines: SQL Server local or container, GLAuth at C:\publish\glauth\, local dev Galaxy) and integration (one dedicated Windows host with Docker Desktop on WSL2 backend so TwinCAT XAR VM can run in Hyper-V alongside containerized oitc/modbus-server, plus WSL2-hosted Snap7 and ab_server, plus OPC Foundation reference server, plus FOCAS TestStub and FaultShim) — with concrete container images, ports, default dev credentials (clearly marked dev-only since production uses Integrated Security / gMSA per decision #46), bootstrap order for both tiers, network topology diagram, test data seed locations, and operational risks (TwinCAT trial expiry automation, Docker pricing, integration host SPOF mitigation, per-developer GLAuth config sync, Aveva license scoping that keeps Galaxy tests on developer machines and off the shared host).
Removes consumer cutover (ScadaBridge / Ignition / System Platform IO) from OtOpcUa v2 scope per decision #136 — owned by a separate integration / operations team, tracked in 3-year-plan handoff §"Rollout Posture" and corrections §C5; OtOpcUa team's scope ends at Phase 5. Updates implementation/overview.md phase index to drop the "6+" row and add an explicit "OUT of v2 scope" callout; updates phase-1 and phase-2 docs to reframe cutover as integration-team-owned rather than future-phase numbered.
Decisions #129–137 added: ACL model (#129), NodeAcl generation-versioned (#130), v1-compatibility seed (#131), denied-only audit logging (#132), two-tier dev environment (#133), Docker WSL2 backend for TwinCAT VM coexistence (#134), TwinCAT VM centrally managed / Galaxy on dev machines only (#135), cutover out of v2 scope (#136), dev credentials documented openly (#137).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Joseph Doherty