feat(acl): keys + ACLs reach driver-only nodes via the artifact (#520 stage 4)
A driver-only node has had no ConfigDb since per-cluster mesh Phase 4, so it registered NullOpcUaUserAuthenticator and denied every UserName token. Its keys now ride the deployment artifact instead. - ConfigComposer snapshots ApiKeys alongside the NodeAcls it already carried. Only the peppered HMAC ships, never a secret, and the pepper is node config that is deliberately not in the artifact. - DeploymentArtifact.ParseDataPlaneAuthority decodes both node-side. Grants are scoped to the node's own cluster; keys are not, because ApiKey carries no ClusterId — a key is a fleet-wide credential and the ACL rows are what bind it. - OpcUaPublishActor installs them on DataPlaneAuthorityState on every apply, BEFORE its plan.IsEmpty early-return: an artifact that only revokes a key produces an empty address-space plan, so feeding after that return would leave the revoked key working until an unrelated structural change redeployed. - ArtifactApiKeyStore serves them to session activation; a node with a ConfigDb keeps ConfigDbApiKeyStore, whose per-request read is what makes revocation immediate there. - OtOpcUaNodeManager gains DataPlaneAuthority — the cluster identity blocker #4 named. NodeScope.ClusterId is required and the evaluator fails closed on a mismatch, so until now no scope it built could have evaluated. Nothing reads it until stages 5/6. The generation is a node-local COUNTER driven by the RevisionHash, not derived from its bits. PermissionTrieCache treats the highest GenerationId as current, so a generation taken from hash bytes would install a lower number about half the time and the cache would go on serving the previous deployment's ACLs, silently. Sessions are not stamped with it yet — that belongs with the first seam that reads a UserAuthorizationState, in stage 5, rather than being built now and consulted by nothing. Live-gated on docker-dev against site-a-1 (driver-only, no ConfigDb): key authenticates, wrong secret denied, both halves of the revocation asymmetry observed, and a SITE-A grant confirmed not to reach site-b-1. Key minted with an independent Python HMAC so the verifier was checked against something other than itself. Rig left clean. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
@@ -110,23 +110,68 @@ mismatch).
|
||||
|
||||
Each stage is one commit, builds clean, and leaves the tree shippable.
|
||||
|
||||
1. **Foundation — principal model.** `NodeAcl.LdapGroup` → `Principal` (+ migration); `ApiKey` entity
|
||||
1. ✅ **Foundation — principal model.** `NodeAcl.LdapGroup` → `Principal` (+ migration); `ApiKey` entity
|
||||
+ store + hashing; `OpcUaServer.csproj` references `Core`. No behaviour change.
|
||||
2. **Identity cutover.** `ApiKeyOpcUaUserAuthenticator` replaces `LdapOpcUaUserAuthenticator` on the
|
||||
2. ✅ **Identity cutover.** `ApiKeyOpcUaUserAuthenticator` replaces `LdapOpcUaUserAuthenticator` on the
|
||||
OPC UA data plane; Anonymous resolves to the `anonymous` principal or is rejected per setting.
|
||||
AdminUI LDAP untouched.
|
||||
3. **Raw scope.** `NodeHierarchyKind.Raw` + RawPath scope paths; `scopePaths` producer built from the
|
||||
3. **Raw scope.** *(not started — sequenced after stage 4 at the owner's request)* `NodeHierarchyKind.Raw` + RawPath scope paths; `scopePaths` producer built from the
|
||||
artifact; coverage for the production hierarchy path that today has none.
|
||||
4. **Artifact + lifecycle.** Node-side ACL/key parsing out of the artifact; cluster identity on the
|
||||
node manager; `AuthGenerationId` from `RevisionHash`.
|
||||
4. **Artifact + lifecycle.** ✅ **Done.** `ConfigComposer` snapshots `ApiKeys` alongside the `NodeAcls`
|
||||
it already carried; `DeploymentArtifact.ParseDataPlaneAuthority` decodes both node-side (grants
|
||||
scoped to the node's own cluster, keys not — a key is a fleet-wide credential and the ACL rows are
|
||||
what bind it); `OpcUaPublishActor` installs them on `DataPlaneAuthorityState` on every apply;
|
||||
`ArtifactApiKeyStore` serves them to session activation on a driver-only node, which until now
|
||||
denied every UserName token; and `OtOpcUaNodeManager` gains a `DataPlaneAuthority` property, which
|
||||
is the cluster identity blocker #4 named.
|
||||
|
||||
Two things went differently from the sketch, both deliberately:
|
||||
|
||||
- **The generation is a counter, not the hash.** "Stamp the generation from the `RevisionHash`"
|
||||
reads as "derive a long from the hash bytes", and that is wrong: `PermissionTrieCache` treats the
|
||||
highest `GenerationId` as current (`ClusterEntry.WithAdditional`) and `Prune` keeps the highest,
|
||||
while hash fragments are unordered — so roughly half of all deploys would install a *lower*
|
||||
generation and the cache would keep serving the previous deployment's ACLs, silently. The hash
|
||||
still *drives* the stamp (a new hash advances it, a re-applied one does not); it just does not
|
||||
*become* it.
|
||||
- **Sessions are not stamped yet.** Building a `UserAuthorizationState` per session belongs with the
|
||||
first seam that reads one, in stage 5. Constructing one now would be a per-session object nothing
|
||||
consults — the dormant-wiring failure mode this repo has been bitten by before.
|
||||
|
||||
Also settled here: the feed runs **before** `OpcUaPublishActor`'s `plan.IsEmpty` early-return. An
|
||||
artifact that only revokes a key produces an empty address-space plan, so feeding after that return
|
||||
would leave the revoked key working until some unrelated structural change happened to redeploy.
|
||||
5. **Enforce write + alarms + HistoryRead** behind `Security:Acl:Enabled`, using the seams that
|
||||
already exist.
|
||||
already exist. Includes building the per-session `UserAuthorizationState` and stamping it with the
|
||||
generation stage 4 now supplies, plus widening both `IApiKeyStore` implementations from
|
||||
Cluster-scoped grants to the full grant set (they narrow today because
|
||||
`ApiKeyInterimRoleMapper` cannot honour scope).
|
||||
6. **Enforce read/browse/subscribe** — the new seam, with per-session decision caching.
|
||||
7. **AdminUI.** Key management (create/revoke, show-once), ACL authoring against principals, and
|
||||
7. **AdminUI.** Key management (create/revoke, show-once) ✅ **done early, in stage 4's predecessor**; ACL authoring against principals, and
|
||||
**remove the two "not enforced" warning banners** — which is the honest signal that this is done.
|
||||
8. **Rollout.** Docs, the ScadaBridge key hand-off, and flipping `Security:Acl:Enabled` on docker-dev
|
||||
as the live gate.
|
||||
|
||||
## Stage 4 live gate — PASSED (docker-dev, 2026-07-31)
|
||||
|
||||
Run against the six-node docker-dev rig, with **site-a-1** as the subject: `Cluster:Roles =
|
||||
driver,cluster-SITE-A`, no `ConnectionStrings:ConfigDb`, `ConfigSource:Mode=FetchAndCache`. Before this
|
||||
stage that node registered `NullOpcUaUserAuthenticator` and denied every UserName token.
|
||||
|
||||
The key was minted with an **independent Python HMAC** rather than by the C# minter, so the verifier was
|
||||
checked against something other than itself.
|
||||
|
||||
| # | Leg | Result |
|
||||
|---|---|---|
|
||||
| 1 | Boot from LocalDb cache installs an authority generation | `generation 1 installed (cluster=SITE-A, revision=3f4a908c, keys=0, grants=0)` — cluster identity resolved with no ConfigDb |
|
||||
| 2 | Deploy a key + a SITE-A cluster grant | `generation 2 installed (revision=cb165478, keys=1, grants=1)`; revision matches the deploy response |
|
||||
| 3 | The key authenticates **on the driver-only node** | `ApiKeyAuth: granted key Stage4 Live Gate (a1b2c3d4e5f60011) as principal stage4-gate with roles [WriteOperate]` |
|
||||
| 4 | A wrong secret is denied there | `denied — secret mismatch for key id a1b2c3d4e5f60011` |
|
||||
| 5 | **Revocation asymmetry, both halves** | `Enabled=0` in SQL, no deploy: central-1 denies on the next request (`no enabled, unexpired key…`) while site-a-1 still accepts. After deploying: `generation 3`, and site-a-1 denies. This is the documented sharp edge, observed rather than asserted. |
|
||||
| 6 | Grants do not leak across clusters | site-b-1 installed `keys=1, grants=0` from the same artifact — it received the key but not SITE-A's grant |
|
||||
|
||||
Rig left clean (0 keys, 0 ACL rows, redeployed, all six nodes healthy).
|
||||
|
||||
## Live gate (stage 8)
|
||||
|
||||
On docker-dev, with `Security:Acl:Enabled=true`:
|
||||
|
||||
+32
-10
@@ -113,10 +113,22 @@ The Server accepts three OPC UA identity-token types:
|
||||
| Token | Handler | Notes |
|
||||
|---|---|---|
|
||||
| Anonymous | No `IOpcUaUserAuthenticator` call — the SDK admits anonymous sessions at the channel. | ⚠️ An anonymous session carries **no roles**, so it is refused every write and every alarm ack — but it can **Browse, Read, Subscribe and HistoryRead the entire address space**. There is no per-node ACL check on those operations (see [Data-Plane Authorization](#data-plane-authorization)). Do not expose an Anonymous-admitting endpoint to an untrusted network. |
|
||||
| UserName/Password | `LdapOpcUaUserAuthenticator.AuthenticateUserNameAsync` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/LdapOpcUaUserAuthenticator.cs`, implements `IOpcUaUserAuthenticator`), backed by the app `ILdapAuthService` — `OtOpcUaLdapAuthService` (`src/Server/ZB.MOM.WW.OtOpcUa.Security/Ldap/OtOpcUaLdapAuthService.cs`). | LDAP bind + group lookup. The returned LDAP groups are mapped to roles via `IGroupRoleMapper<string>` (`OtOpcUaGroupRoleMapper`) and the **roles** are attached to the OPC UA session identity as a `RoleCarryingUserIdentity`. The raw group list is discarded at that point and never reaches the data plane. |
|
||||
| UserName/Password | `ApiKeyOpcUaUserAuthenticator.AuthenticateUserNameAsync` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyOpcUaUserAuthenticator.cs`, implements `IOpcUaUserAuthenticator`), backed by an `IApiKeyStore`. | The username is the literal `apikey` and the password is an `otua_<keyId>_<secret>` credential (#520). One indexed lookup plus one peppered HMAC-SHA256, no network I/O. The key resolves to a **principal**, whose `NodeAcl` grants become roles on a `RoleCarryingUserIdentity`. **LDAP is not on this path** — it backs AdminUI login only. |
|
||||
| X.509 Certificate | Stack-level acceptance during the secure-channel handshake. | The certificate must be trusted (see PKI trust flow). It carries **no** roles, so a cert-only session is read-everything / write-nothing. |
|
||||
|
||||
When no authenticator is supplied, `OpcUaApplicationHost` falls back to `NullOpcUaUserAuthenticator`; the Host wires the real `LdapOpcUaUserAuthenticator` as a singleton in `Program.cs`.
|
||||
When no authenticator is supplied, `OpcUaApplicationHost` falls back to `NullOpcUaUserAuthenticator`;
|
||||
the Host wires the real `ApiKeyOpcUaUserAuthenticator` as a singleton in `Program.cs`. Which
|
||||
**`IApiKeyStore`** backs it depends on the node's role:
|
||||
|
||||
| Node | Store | Revocation takes effect |
|
||||
|---|---|---|
|
||||
| `admin` (including fused `admin,driver`) | `ConfigDbApiKeyStore` — reads the config database per request | On the **next request** |
|
||||
| `driver`-only | `ArtifactApiKeyStore` — reads the keys carried by the deployment artifact, held in memory | On the **next deployment** |
|
||||
| neither role | `NullOpcUaUserAuthenticator` — denies every UserName token | n/a |
|
||||
|
||||
The deny-all is registered **explicitly**, not left absent: `OtOpcUaServerHostedService` takes
|
||||
`IOpcUaUserAuthenticator` as a required constructor parameter, so an unregistered seam fails DI and the
|
||||
node does not boot at all.
|
||||
|
||||
### LDAP bind flow (`OtOpcUaLdapAuthService`)
|
||||
|
||||
@@ -311,11 +323,17 @@ What is real about it:
|
||||
validated, versioned and persisted.
|
||||
- `ConfigComposer` **does** snapshot every `NodeAcl` row into every deployment artifact, so an ACL
|
||||
edit shifts the artifact's `RevisionHash` and ships to every node.
|
||||
- The node side **never deserializes them** — `DeploymentArtifact` has no ACL reader. The bytes arrive
|
||||
and are dropped.
|
||||
- The node side **does** deserialize them as of #520 stage 4 —
|
||||
`DeploymentArtifact.ParseDataPlaneAuthority` reads the artifact's `NodeAcls` (scoped to the node's own
|
||||
cluster) and `ApiKeys`, and `OpcUaPublishActor` installs both on the node's
|
||||
`IDataPlaneAuthoritySource` on every apply. The node manager holds that source, so it finally has a
|
||||
cluster identity.
|
||||
- **Nothing evaluates them yet.** The grants are decoded and available; the enforcement seams that would
|
||||
consult them land in stages 5 and 6.
|
||||
|
||||
So an operator can author a grant, deploy it green, and have it change nothing. That is the behaviour
|
||||
to expect until the tracking issue below is closed.
|
||||
So an operator can still author a grant, deploy it green, and have it change nothing at the node — but
|
||||
the grant now reaches the node rather than being dropped at its door. That is the behaviour to expect
|
||||
until the tracking issue below is closed.
|
||||
|
||||
**The four things blocking a wire-up** (all verified, none of them merely "call the evaluator"):
|
||||
|
||||
@@ -323,14 +341,18 @@ to expect until the tracking issue below is closed.
|
||||
`Equipment`. Raw NodeIds are `Folder→Driver→Device→TagGroup→Tag` RawPaths, which `NodeScope` cannot
|
||||
express — yet the write gate is attached to raw tags. Roughly half the writable address space has
|
||||
no scope to evaluate. This is a design decision, not wiring.
|
||||
2. **The identity carries roles, not groups.** The trie matches `NodeAcl.LdapGroup` against the
|
||||
session's LDAP groups, but `OpcUaUserAuthResult` carries only mapped role strings; the group list
|
||||
is discarded inside `LdapOpcUaUserAuthenticator`.
|
||||
2. ~~**The identity carries roles, not groups.**~~ **Dissolved by #520 stage 2.** The column is now
|
||||
`NodeAcl.Principal` and the principal comes from the presented API key (or the reserved `anonymous`),
|
||||
not from a directory — so there is no group list to lose.
|
||||
3. **`PermissionTrieBuilder`'s `scopePaths` argument has no producer.** Without it every sub-cluster
|
||||
grant lands in the builder's fallback bucket — the hazard its own comments warn about. It would have
|
||||
to be derived from the artifact's UNS relations.
|
||||
4. **No session lifecycle exists.** `UserAuthorizationState` has the freshness/staleness predicates but
|
||||
nothing in production constructs one, stamps a generation, or drives a refresh.
|
||||
nothing in production constructs one per session. **Half-closed by stage 4:** the *generation* now
|
||||
exists and advances — `DataPlaneAuthorityState` stamps a monotonic `GenerationId` each time an
|
||||
artifact with a new `RevisionHash` is applied — so there is something to stamp a session with. Doing
|
||||
the stamping is stage 5's job, because until an enforcement seam reads it, a per-session state object
|
||||
would be constructed and never consulted.
|
||||
|
||||
Two smaller inconsistencies inside the subsystem itself: `NodeAclScopeKind.FolderSegment` is offered in
|
||||
the authoring dropdown but the trie walker has no level to match it at, and `NodePermissions.AlarmRead`
|
||||
|
||||
Reference in New Issue
Block a user