Files

T

Joseph Doherty c8cf5c8032 test(host): the R3-01 falsifiability control asserts the cookie-challenge signature (a bare Exception was satisfiable by an anonymised endpoint too); docs wording

2026-08-20 10:43:49 -04:00

80 KiB

Raw Blame History

Security

v2 status (2026-05-26). The four security concerns below are unchanged in v2. Paths + project names moved: OtOpcUa.Server/Security/ → OtOpcUa.Security/ (Ldap/, Jwt/, Endpoints/AuthEndpoints.cs), OtOpcUa.Admin is gone (its auth + role-grant pages live in OtOpcUa.AdminUI), and Admin auth policies register from OtOpcUa.Host/Program.cs via AddOtOpcUaAuth (src/Server/ZB.MOM.WW.OtOpcUa.Security/ServiceCollectionExtensions.cs) rather than in a separate Admin process. The Admin UI uses a single Cookie authentication scheme — there is no AddJwtBearer pipeline. The Security:Jwt section configures JwtTokenService, which mints a JWT at the /auth/token endpoint for external consumers (OPC UA clients / automation scripts); the cookie itself stores the ClaimsPrincipal directly. DataProtection keys persist to the shared Config DB (PersistKeysToDbContext<OtOpcUaConfigDbContext>) so cookies survive failover between admin-role nodes.

See docs/plans/2026-05-26-akka-hosting-alignment-design.md §5 for the v2 auth + DataProtection rationale.

OtOpcUa has four independent security concerns. This document covers all four:

Transport security — OPC UA secure channel (signing, encryption, X.509 trust).
OPC UA authentication — Anonymous / UserName / X.509 session identities. A UserName token is an API key (username literally apikey, password otua_<keyId>_<secret>) verified by peppered HMAC — LDAP backs Admin UI login only since #520 (2026-08-01).
Data-plane authorization — who can do what on an OtOpcUa endpoint. ⚠️ Governed by one flag (#520 stages 5–6, 2026-08-01). With Security:Acl:Enabled false — the default — the only enforcement is two coarse, server-wide role gates (WriteOperate on Value writes, AlarmAck on the alarm methods), and Read, Browse, Subscribe and HistoryRead are ungated for any admitted session. With it true, scoped NodeAcl grants (the PermissionTrie subsystem) decide every data-plane operation — read, browse, subscribe, write, alarm commands and HistoryRead — replacing the server-wide roles. See Data-Plane Authorization.
Control-plane authorization — who can view or edit fleet configuration in the Admin UI. Gated by the AdminRole (Viewer / Designer / Administrator) claim resolved from LdapGroupRoleMapping.

Transport security and OPC UA authentication are per-node concerns configured in the Server's bootstrap appsettings.json. Data-plane ACLs and Admin role grants live in the Config DB.

Transport Security

Overview

The OtOpcUa Server supports configurable OPC UA transport security profiles that control how data is protected on the wire between OPC UA clients and the server.

There are two distinct layers of security in OPC UA:

Transport security -- secures the communication channel itself using TLS-style certificate exchange, message signing, and encryption. This is what the OpcUa:EnabledSecurityProfiles setting controls.
UserName token encryption -- protects user credentials (username/password) sent during session activation. The OPC UA stack encrypts UserName tokens using the server's application certificate regardless of the transport security mode. UserName authentication therefore works on None endpoints too — the credentials themselves are always encrypted. A secure transport profile adds protection against message-level tampering and eavesdropping of data payloads.

Supported security profiles

The profiles are the members of the OpcUaSecurityProfile enum (src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OpcUaApplicationHost.cs). The server ships three baseline profiles; the config value is the bare enum-member name (no hyphens, no underscores):

Enum member	Security Policy	Message Security Mode	Description
`None`	None	None	No signing or encryption. Suitable for development and isolated networks only.
`Basic256Sha256Sign`	Basic256Sha256	Sign	Messages are signed but not encrypted. Protects against tampering but data is visible on the wire.
`Basic256Sha256SignAndEncrypt`	Basic256Sha256	SignAndEncrypt	Messages are both signed and encrypted. Full protection against tampering and eavesdropping.

BuildSecurityPolicies (OpcUaApplicationHost.cs) maps each configured profile to an SDK ServerSecurityPolicy. The server exposes a separate endpoint per configured profile and clients select the one they prefer at session open. The enum's XML doc notes that Aes128/Aes256 variants can be added later by extending the enum + BuildSecurityPolicies — the wiring is profile-agnostic — but they are not implemented today. There is no SecurityProfileResolver class.

Config value form. The enum binds by member name, so a profile string with hyphens (e.g. Basic256Sha256-Sign) does not bind — use the exact enum-member spelling above. If EnabledSecurityProfiles is empty, the server falls back to a single None endpoint (logged, very visible) so it still has a listening endpoint.

Configuration

Transport security is configured in the OpcUa section of the Host process's bootstrap appsettings.json (bound to OpcUaApplicationHostOptions):

{
  "OpcUa": {
    "ApplicationName": "OtOpcUa",
    "ApplicationUri": "urn:node-a:OtOpcUa",
    "PublicHostname": "0.0.0.0",
    "OpcUaPort": 4840,
    "PkiStoreRoot": "C:/ProgramData/OtOpcUa/pki",
    "AutoAcceptUntrustedClientCertificates": false,
    "EnabledSecurityProfiles": [ "Basic256Sha256Sign", "Basic256Sha256SignAndEncrypt" ]
  }
}

EnabledSecurityProfiles is a list — the server publishes one endpoint per entry. The default (when the key is omitted) is all three baseline profiles (None, Basic256Sha256Sign, Basic256Sha256SignAndEncrypt); production deployments typically drop None. The list must contain at least one entry (OpcUaApplicationHostOptionsValidator enforces MinCount(…, 1)).

The server certificate is auto-generated on first start if none exists in PkiStoreRoot/own/. Always generated even for None-only deployments because UserName token encryption depends on it.

PKI directory layout

{PkiStoreRoot}/
  own/        Server's own application certificate and private key
  issuer/     CA certificates that issued trusted client certificates
  trusted/    Explicitly trusted client (peer) certificates
  rejected/   Certificates that were presented but not trusted

Certificate trust flow

When a client connects using a secure profile (Sign or SignAndEncrypt), the following trust evaluation occurs:

The client presents its application certificate during the secure channel handshake.
The server checks whether the certificate exists in the trusted/ store.
If found, the connection proceeds.
If not found and AutoAcceptUntrustedClientCertificates is true, the certificate is automatically copied to trusted/ and the connection proceeds.
If not found and AutoAcceptUntrustedClientCertificates is false, the certificate is copied to rejected/ and the connection is refused.

The Admin UI Certificates.razor page (src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Certificates.razor) lists the contents of each PKI sub-store (own / trusted / issuer / rejected) by reading the OpcUa:PkiStoreRoot path from configuration. FleetAdmin users get per-row actions on the trusted and rejected stores: Trust (move a rejected cert into trusted/), Untrust (move a trusted cert back into rejected/), and Delete (remove a cert from either store). Each action requires an inline confirmation, is re-checked against the FleetAdmin policy server-side, and is honored live — the SDK trust list re-enumerates the directory store on the next handshake, no restart required. (own / issuer remain read-only.) The actions are filesystem moves/deletes performed by CertificateStoreManager, which finds the target by matching the certificate thumbprint — never by building a path from caller input.

Production hardening

Set AutoAcceptUntrustedClientCertificates = false.
Drop None from EnabledSecurityProfiles.
Promote trusted client certs with the Admin UI Certificates page's Trust action (FleetAdmin) — or by moving the .der from rejected/ to trusted/certs/ — rather than relying on the auto-accept fallback.
Periodically audit the rejected/ directory; an unexpected entry is often a misconfigured client or a probe attempt.

Internal gRPC planes (h2c) — accepted cleartext posture

Everything above concerns the OPC UA endpoint — the surface external clients bind to. OtOpcUa also runs five internal gRPC planes between its own processes (central ↔ driver node, and node ↔ pair partner). Those are a separate transport story, recorded here so it is written down once rather than rediscovered per plane.

#	Plane	gRPC service	Listener (config key)	Credential (config key)	Interceptor
1	Live telemetry stream (central dials the node)	`telemetry.v1.TelemetryStreamService/Subscribe`	driver node, `Telemetry:GrpcListenPort`	`Telemetry:ApiKey` (dialer presents `TelemetryDial:ApiKey`)	`TelemetryStreamAuthInterceptor`
2	Config-serve / artifact fetch (node fetches from central)	`deployment_artifact.v1.DeploymentArtifactService/Fetch`	central (admin role), `ConfigServe:GrpcListenPort`	`ConfigServe:ApiKey` (fetcher presents `ConfigSource:ApiKey`)	`ConfigServeAuthInterceptor`
3	LocalDb pair sync (one node dials its partner)	`localdb_sync.v1.LocalDbSync/Sync`	driver node, `LocalDb:SyncListenPort`	`LocalDb:Replication:ApiKey` (byte-identical on both nodes of the pair)	`LocalDbSyncAuthInterceptor`
4	Mesh commands (central drives the node)	`mesh_transport.v1.NodeCommandService/Execute`	shares the node's `Telemetry:GrpcListenPort` listener	`MeshTransport:GrpcPsk` — per application `Cluster`, not fleet-wide	`MeshCommandAuthInterceptor`
5	Mesh deploy acks (node reports to central)	`mesh_transport.v1.CentralCommandService/ReportApply`	shares central's `ConfigServe:GrpcListenPort` listener	per-cluster key resolved by `IMeshPskProvider` — `MeshTransport:ClusterPsks:<clusterId>` override, else the `OT-MESH-PSK-<clusterId>` secret	`CentralCommandAuthInterceptor`

Planes 4 and 5 claim no port of their own; they ride the listener their sibling already binds. On the docker-dev rig those two config keys resolve to :4056 (driver nodes) and :4055 (central) — rig values of the keys above, not hardcoded ports.

The posture. All five listeners are bound HttpProtocols.Http2 on a cleartext port, and every client dials a prior-knowledge http://host:port address — h2c, no TLS, no ALPN. (Cleartext is structural, not incidental: a cleartext Http1AndHttp2 endpoint cannot negotiate HTTP/2 without ALPN, which is why these are dedicated HTTP/2-only ports rather than a multiplex onto the main one.) Authentication on all five is a shared bearer token in an Authorization header, compared in constant time and fail-closed when unconfigured. Because the transport is cleartext, the bearer itself crosses the wire in the clear — anyone who can observe or interpose on the link can read a key and then replay it.

What that buys an attacker is not uniform:

Planes 1–2 are read surfaces: the live telemetry feed, and the full deployed configuration.
Plane 3 writes rows into a node's deployment-artifact cache — which is what that node boots from when central SQL is unreachable.
Planes 4–5 are the command surfaces, and are the sharpest: plane 4 carries RestartDriver, AlarmCommand and deployment dispatch; plane 5 carries the apply acks the deploy coordinator counts, so forged acks can seal a deployment nodes never applied.

Decision: risk accepted, 2026-08-06 (Gitea lmxopcua#545, option 1). Accepted once, for all five planes together, on the trusted-link assumption every one of them already makes: the central↔site and pair-local links are private (VPN or dedicated WAN), not routed over anything an untrusted party can observe. Two things bound the consequences if that assumption is wrong:

Per-cluster PSKs bound blast radius on the two command planes. A captured SITE-A key drives SITE-A only; it is refused by SITE-B's node and by central when it claims to be SITE-B. A single fleet-wide key would have made one observed link equal to fleet-wide command authority.
Production keys are delivered as ${secret:...} references, never committed literals, and MUST NOT be reused across clusters — reusing one key collapses the isolation above back into a fleet-wide key with extra steps. The three literal dev-mesh-psk-* keys exist only on the docker-dev rig.

Future work. The recorded alternative is a UseTls/scheme option on these planes (dial https://, bind the listener with a certificate). If it is ever built it must cover all five planes together — a mixed posture, where some internal links are encrypted and others are not, is harder to reason about than either uniform answer, and invites exactly the "which one was it again?" mistake this note exists to prevent. See issue #545.

OPC UA Authentication

The Server accepts three OPC UA identity-token types:

Token	Handler	Notes
Anonymous	No `IOpcUaUserAuthenticator` call — the SDK admits anonymous sessions at the channel.	⚠️ Since #520 an anonymous session is a normal grantable principal (`anonymous`), not a bypass — see Anonymous is a normal principal. With `Security:Acl:Enabled` off (the default) it typically holds no roles, so every write and alarm ack is refused — but it can Browse, Read, Subscribe and HistoryRead the entire address space (those operations are ungated). With the flag on, those operations too are decided by the `anonymous` principal's `NodeAcl` grants (see Data-Plane Authorization). Do not expose an Anonymous-admitting endpoint to an untrusted network.
UserName/Password	`ApiKeyOpcUaUserAuthenticator.AuthenticateUserNameAsync` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyOpcUaUserAuthenticator.cs`, implements `IOpcUaUserAuthenticator`), backed by an `IApiKeyStore`.	The username is the literal `apikey` and the password is an `otua_<keyId>_<secret>` credential (#520). One indexed lookup plus one peppered HMAC-SHA256, no network I/O. The key resolves to a principal, whose `NodeAcl` grants become roles on a `RoleCarryingUserIdentity`. LDAP is not on this path — it backs AdminUI login only.
X.509 Certificate	Stack-level acceptance during the secure-channel handshake.	The certificate must be trusted (see PKI trust flow). It carries no roles, so a cert-only session is read-everything / write-nothing.

When no authenticator is supplied, OpcUaApplicationHost falls back to NullOpcUaUserAuthenticator; the Host wires the real ApiKeyOpcUaUserAuthenticator as a singleton in Program.cs. Which IApiKeyStore backs it depends on the node's role:

Node	Store	Revocation takes effect
`admin` (including fused `admin,driver`)	`ConfigDbApiKeyStore` — reads the config database per request	On the next request
`driver`-only	`ArtifactApiKeyStore` — reads the keys carried by the deployment artifact, held in memory	On the next deployment
neither role	`NullOpcUaUserAuthenticator` — denies every UserName token	n/a

The deny-all is registered explicitly, not left absent: OtOpcUaServerHostedService takes IOpcUaUserAuthenticator as a required constructor parameter, so an unregistered seam fails DI and the node does not boot at all.

LDAP bind flow (`OtOpcUaLdapAuthService`)

LDAP is configured under the Security:Ldap section (bound to LdapOptions, src/Server/ZB.MOM.WW.OtOpcUa.Security/Ldap/LdapOptions.cs, SectionName = "Security:Ldap"). The app authenticator is OtOpcUaLdapAuthService — a thin wrapper around the shared ZB.MOM.WW.Auth.Ldap directory client that adds two app-only concerns the shared library deliberately does not model: the Enabled master switch and DevStubMode. Since #520 it serves the Admin UI cookie login (/auth/login) only — LdapOpcUaUserAuthenticator is deleted and the OPC UA UserName path authenticates API keys instead (see API-key identity).

OtOpcUaLdapAuthService.AuthenticateAsync:

If Enabled = false, denies outright — no bind, no DevStub bypass (the master switch wins).
If DevStubMode = true, accepts any non-empty credentials and grants the Administrator role without any network bind (dev only — must be false in production).
Refuses to bind over a plaintext transport (Transport = None) unless AllowInsecure = true (dev/test only). This is enforced at login, not at startup.
Delegates the real path to the shared ZB.MOM.WW.Auth.Ldap client: it binds (search-then-bind via ServiceAccountDn, or direct-bind cn={user},{SearchBase} when no service account is set), verifies the password, and reads the user's group memberships.
Returns an LdapAuthResult carrying the validated username + the groups (never roles). Failure codes are folded into opaque user-facing error strings so a probe cannot distinguish "unknown user" from "wrong password".

Group → role mapping happens downstream, not in the auth service: the /auth/login handler (AuthEndpoints) resolves IGroupRoleMapper<string> (OtOpcUaGroupRoleMapper) per login — it applies the appsettings GroupToRole baseline, merges system-wide DB grants from the user's LDAP groups, and unions any pre-resolved roles (the DevStub grant) — and the resulting roles land in the Admin UI cookie principal. A mapper fault (e.g. a Config DB outage) falls back to the pre-resolved baseline rather than denying an otherwise-authenticated login. Since #520 nothing on the OPC UA data plane consumes LDAP groups — data-plane session roles come from the authenticated principal's NodeAcl grants (see Role grant source).

Transport replaces the former UseTls bool: Ldaps (implicit TLS), StartTls (upgrade), or None (plaintext, requires AllowInsecure). Configuration example (Active Directory production):

{
  "Security": {
    "Ldap": {
      "Enabled": true,
      "DevStubMode": false,
      "Server": "dc01.corp.example.com",
      "Port": 636,
      "Transport": "Ldaps",
      "AllowInsecure": false,
      "SearchBase": "DC=corp,DC=example,DC=com",
      "ServiceAccountDn": "CN=OtOpcUaSvc,OU=Service Accounts,DC=corp,DC=example,DC=com",
      "ServiceAccountPassword": "<from your secret store>",
      "GroupAttribute": "memberOf",
      "DisplayNameAttribute": "cn",
      "UserNameAttribute": "sAMAccountName",
      "GroupToRole": {
        "OPCUA-Designers": "Designer",
        "OPCUA-Admins": "Administrator",
        "OPCUA-Operators": "Operator"
      }
    }
  }
}

GroupToRole maps LDAP group names → roles (case-insensitive); a user gets every role whose source group is in their membership. The values are the canonical control-plane role strings (Viewer / Designer / Administrator, plus the appsettings-only Operator for the DriverOperator policy); the same map also supplies data-plane role strings (ReadOnly, WriteOperate, WriteTune, WriteConfigure, AlarmAck) — see Role grant source (data-plane) below. UserNameAttribute: "sAMAccountName" is the critical AD override — the GLAuth dev default is cn, which is not how AD users are looked up; use userPrincipalName instead if operators log in with user@corp.example.com form. LdapOptionsValidator (src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LdapOptionsValidator.cs) fails startup when Transport = None and AllowInsecure = false on a real-LDAP (non-DevStub) config.

API-key identity (OPC UA data plane)

The OPC UA data plane is machine-to-machine, so a consuming system authenticates with an API key rather than an LDAP user (Gitea #520). The client presents an OPC UA UserName token whose username is the fixed literal apikey and whose password is the key:

otua_<keyId>_<secret>

ApiKeyOpcUaUserAuthenticator verifies it — one indexed lookup on keyId plus one peppered HMAC-SHA256 over the secret, compared with CryptographicOperations.FixedTimeEquals. No network I/O. The key resolves to a principal, and NodeAcl rows naming that principal are what grant it permissions.

Key	Notes
`Security:ApiKeys:Username`	The static username clients present. Default `apikey`. Checked rather than ignored, so a typo is an explicit denial.
`Security:ApiKeys:Pepper`	Server-side pepper for the stored hashes. Supply via `Security__ApiKeys__Pepper` or `${secret:}` — never commit it. Rotating it invalidates every existing key. Has no default: an empty pepper still hashes "successfully", so `ApiKeySecret.ComputeHash` throws on one rather than producing a working-looking deployment with no protection.

Secrets are never stored — only the hash. The plaintext is shown once at creation and is unrecoverable afterwards.

Every rejection returns the same opaque message ("Invalid API key"), with the real reason logged server-side only, so the server cannot be used as an oracle for whether a key id exists or which half of a credential was wrong.

⚠️ Revocation requires a deploy. Per-cluster mesh Phase 4 leaves driver-only nodes with no ConfigDb, so keys reach them inside the deployment artifact exactly as NodeAcl rows do. Clearing Enabled takes effect on a node only once a new artifact reaches it.

Anonymous is a normal principal

Anonymous sessions are admitted by default (OpcUa:AllowAnonymous, default true — flipping the default would silently deny every client that has no key the moment this code deploys, including ScadaBridge). Set it to false to make keys mandatory: the Anonymous token policy is then not advertised, so a client is refused at discovery rather than only at activation, and anonymous activation is rejected independently in case a client attempts a token type the endpoint never offered.

When admitted, an anonymous session is not a special case — it authenticates as the reserved principal anonymous, and NodeAcl rows naming it grant permissions exactly as they would a key's principal. It can therefore be granted write, not merely tolerated as read-only. It arrives at the downstream gates as the same RoleCarryingUserIdentity a key produces, so no gate carries an anonymous-shaped branch and there is no bypass path to get wrong.

Two consequences worth knowing:

anonymous is reserved: an API key may not use it as its principal, or a credentialled client would silently inherit whatever anonymous was granted. Rejected at authentication as well as at authoring.
If the anonymous principal's grants cannot be read, the session is denied, not admitted with no roles. Admitting is the pre-#520 behaviour, and defaulting to it precisely when the server cannot determine what anonymous may do stops being harmless once reads are gated.

⚠️ Node ACLs are enforced only when Security:Acl:Enabled is set. With the flag off — the default — a key's grants are collapsed into the two server-wide role gates and only Cluster-scoped grants count, so a narrower grant confers nothing rather than silently widening to server-wide authority, and read, browse, subscribe and HistoryRead are ungated for any admitted session. With it on, every data-plane operation is decided by scoped NodeAcl grants — read, browse, subscribe, write, HistoryRead and the alarm commands (stage 6). See Node ACL enforcement and docs/plans/2026-07-31-apikey-acl-enforcement-design.md.

LDAP resilience — mostly retired with the data-plane authenticator (#520)

Changed 2026-07-31. The OPC UA data plane no longer uses LDAP at all — it authenticates with API keys (see API-key identity). LdapOpcUaUserAuthenticator is deleted, and with it four of the five Security:Ldap resilience keys: AuthTimeoutMs, MaxConcurrentAuthentications, OutageFailureThreshold and OutageCooldownSeconds. Setting them now does nothing (options binding ignores unknown keys); remove them from any config that carries them.

Those four existed for one specific reason. The OPC UA SDK raises its impersonation callback synchronously, inside a SessionManager-wide event lock (verified against OPCFoundation.NetStandard.Opc.Ua.Server 1.5.378), so a slow or hung LDAP bind in that callback did not stall one SDK thread — it serialized every session activation server-wide, Anonymous and X509 included. That constraint is gone: key verification is one indexed lookup plus one HMAC, with no network I/O on the path.

LDAP now runs only on AdminUI login, an ordinary async request path where a slow directory affects the one request making it. One key remains:

Key	Default	Bounds
`ConnectionTimeoutMs`	`10000`	Projected into the shared library's `ConnectionTimeoutMs` — bounds the socket connect and each per-operation search. Validated `> 0` at startup by `LdapOptionsValidator`.

Data-Plane Authorization

⚠️ Read this first (updated 2026-08-01, #520 stages 5–6). Whether the NodeAcl / PermissionTrie subsystem runs depends on one flag. With Security:Acl:Enabled false — the default — it does not: the only authorization in the server is the coarse role checks in What is enforced with the flag off, and read, browse, subscribe and HistoryRead are ungated for any admitted session including Anonymous. With it true, those role checks are replaced by scoped ACL decisions and every data-plane operation is gated — see Node ACL enforcement.

What is enforced with the flag off

There are exactly three authorization checks in the whole OPC UA server layer. All three are coarse, server-wide role string checks with no per-node component.

Operation	Gate	Requires
Write (Value attribute, either namespace)	`OtOpcUaNodeManager.EvaluateEquipmentWriteGate`	role `WriteOperate`
Alarm Acknowledge / Confirm / AddComment / Shelve / Unshelve (scripted alarms)	`OtOpcUaNodeManager.HandleAlarmCommand`	role `AlarmAck`
Alarm acknowledge (driver-fed native alarms)	`OtOpcUaNodeManager.HandleNativeAlarmAck`	role `AlarmAck`

Both gates fail closed — a session with no identity or no matching role gets BadUserAccessDenied — and read identity.Roles off the session's RoleCarryingUserIdentity (src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/RoleCarryingUserIdentity.cs). Role strings are the constants in OpcUaDataPlaneRoles, derived from the principal's cluster-scoped NodeAcl grants (see Role grant source).

What has no authorization check at all:

Read, Browse, TranslateBrowsePaths — no override, no handler. Any admitted session, including an Anonymous one, sees and reads the entire address space.
HistoryRead — all four overrides (HistoryReadRawModified, HistoryReadProcessed, HistoryReadAtTime, HistoryReadEvents) run without an identity check.
CreateMonitoredItems / subscriptions / TransferSubscriptions.
Non-Value attribute writes — the gate hangs off OnWriteValue.

And what the write gate does not distinguish:

WriteTune and WriteConfigure are never checked. They exist in the role vocabulary and in NodePermissions, but every write — whatever the tag's security classification — is gated on the single WriteOperate string.
AlarmAcknowledge / AlarmConfirm / AlarmShelve are not distinguished. One AlarmAck role covers all five alarm methods; the operation argument is passed to the router but never consulted by the gate.
There is no per-node granularity. A session that may write one tag may write every writable tag on the node.

The gate is realm-aware in one narrow sense: it is attached per-node during materialization for both the Raw and UNS realms, and the write dispatch it guards is realm-qualified. The role decision itself takes no realm and no node.

Node ACL enforcement (`Security:Acl:Enabled`)

Key	Default	Effect
`Security:Acl:Enabled`	`false`	`true` makes scoped `NodeAcl` grants decide every data-plane operation — read, browse, subscribe, write, alarm commands and HistoryRead

Live-gated on docker-dev 2026-08-01 (site-a-1 with the flag on, central-1 as the same-artifact control): scoped read/browse/write behave as described below, and revoking a grant stopped a running subscription mid-stream — the client received BadUserAccessDenied, then 25 seconds of silence, then the feed resumed by itself when the grant was restored. Evidence + what the gate did not cover (UNS grants, alarm scoping) in docs/plans/2026-07-31-apikey-acl-enforcement-design.md.

It is a config change, not a redeploy. Both paths are wired in both modes — the node always builds its permission trie on every deploy and always constructs the gate; the flag only chooses which decision the seams consult. Same dark-switch discipline as MeshTransport, ConfigSource and Telemetry.

What changes when it is on

Operation	Off	On
Write (Value)	role `WriteOperate`	`NodePermissions.WriteOperate` on that node's scope
Alarm Acknowledge / AddComment / Enable / Disable	role `AlarmAck`	`AlarmAcknowledge` on the condition's scope
Alarm Confirm	role `AlarmAck`	`AlarmConfirm`
Alarm OneShotShelve / TimedShelve / Unshelve	role `AlarmAck`	`AlarmShelve`
HistoryRead (all four variants)	nothing	`HistoryRead` on each node's scope, decided per node in the batch
Read (every attribute, not just Value)	nothing	`Read` on that node's scope, decided per node in the batch
Browse / TranslateBrowsePath	nothing	`Browse` on the source node, and on each returned reference
CreateMonitoredItems / SubscribeToEvents	nothing	`Subscribe` on that node's scope, decided per item

⚠️ The ACL decision REPLACES the role check; the two are not chained. Chaining them would require a grant to confer the server-wide role as well as pass the scope check, and only Cluster-scoped grants confer a role (ApiKeyInterimRoleMapper) — so every narrow grant would remain inert and the flag would look like it did nothing. Narrow grants working is the whole point of the stage.

Behaviours worth knowing before enabling it:

A deploy takes effect on sessions that are already open. The gate stamps each decision with the current authority generation rather than the one the session activated under, so an ACL edit lands without bouncing clients — in both directions, widening and narrowing.
Revocation also reaches an open session. Every decision re-checks the authenticating key against the current snapshot, so a key disabled or expired by a later deployment stops working immediately rather than at the client's next reconnect. On a driver-only node the artifact is the only revocation channel there is, so without this a revoked credential could keep its authority indefinitely.
Revocation reaches a live subscription too (#520 stage 6). Every other operation re-asks on each request; a subscription is authorized once and then streams, so it needed a push. A deployment re-authorizes every live monitored item: one whose grant has gone is queued a BadUserAccessDenied and then detached from its node's value fan-out, and a later deployment that restores the grant re-attaches it. This matters more than it sounds — subscriptions are how machine consumers actually read this server, so without it revoking a grant would close the operations nobody uses and leave the feed running.
An ancestor of a granted node is browsable, or no narrow grant would be reachable: a client arrives at a deep node by browsing its ancestors, and the trie otherwise inherits downward only. Note what this necessarily discloses — the names of the folders on the path to a granted node are visible to whoever holds that grant. It is conferred only by grants that actually carry Browse, and it confers nothing but Browse.
…including the shared root folder every browse starts at. Both realm trees hang under a single anchor folder (s=OtOpcUa, minted in the raw namespace by CreateAddressSpace; top-level raw folders and top-level UNS areas alike parent under it). Its identifier is a display name rather than a path, so splitting it produced a segment no grant path can contain, and the Browse gate refuses the whole call when its source node is denied — every narrow Browse grant was therefore unreachable by navigation, denied at the front door. The gate maps that one identifier onto the trie's zero-segment realm-root scope, and because the anchor fronts both trees its browse considers transit in either realm: a principal whose only grant lives in the UNS tree still enters through this raw-namespace node. The mapping is realm-qualified — in the UNS namespace s=OtOpcUa is an ordinary Area an operator happened to name that way and is evaluated as the path it is.
A denied node reports BadUserAccessDenied rather than BadNodeIdUnknown, because the gate runs before node resolution. That is the information-hiding direction — error codes can no longer be used to enumerate the address space — but it does mean "access denied" is not evidence the node exists.
The shared root folder remains listed for any admitted session. It is referenced from the ns=0 Objects folder, whose browse the SDK's own node manager serves, so a session with no grants sees the anchor in that listing — and is refused the moment it tries to browse into it (the transit above is what a granted session enters on). There is one anchor, not one per namespace.

Cost

Read, Browse and subscription creation ask the same question many times per request — one Browse of a device folder asks once per child. Those seams go through a per-(session, node, operation, realm) decision cache (SessionAclDecisionCache), cleared wholesale whenever the authority generation advances.

⚠️ The cache holds the grant walk, not the decision. Grants, principals and the trie are all fixed for the life of a generation, so memoizing the walk is sound. A key's ExpiresUtc passes with the clock, not with a deployment — so revocation and expiry are re-checked on every call, outside the cache. Caching them would have let an expired key keep working until the next deploy.

The cache is sharded per calling KEY, because a flat one let any client evict every other client's decisions. It used to be a single dictionary with the caller identity folded into the composite key and one process-wide bound of MaxEntries = 100_000, whose only eviction was a wholesale Clear() at the cap. So one machine consumer crawling a large address space past 100k entries wiped the memoized walks of every other caller on the server, and each store paid a ConcurrentDictionary.Count — an all-locks snapshot — to test the cap. It is now a dictionary of shards keyed by ApiKeyId ?? Principal, each holding its own (operation, realm, nodeIdentifier) entries, with two bounds:

Bound	Value	What it clears
`MaxEntriesPerSession`	20 000	that one key's shard
`MaxEntries`	100 000	every shard (the shard objects are dropped, not emptied)

⚠️ The shard key is the API key, not the session — the constant's name predates it. Every session authenticating with the same otua_<keyId>_… credential shares one shard and one 20 000-entry budget, so a crawl by one of them can clear the memoized decisions of its sibling sessions. What the sharding guarantees is isolation across distinct keys: one key's crawl can never evict another key's entries. (An Anonymous session shards under the reserved anonymous principal, so all of them likewise share one shard.)

Both bounds are compile-time constants, not appsettings keys. Five keys simultaneously at their per-key ceiling reach the global cap; the number of shards is not itself bounded — the bound is entries.

Things to know before changing it:

There is no session-close eviction hook, deliberately. The shard of a key with no live sessions left is shed the next time a wholesale clear happens — a generation bump, or the global cap tripping. Dropping the shard objects rather than emptying them is what makes that sweep work.
The counters are approximate under concurrency, by design. The cap test is a check-then-act over Volatile.Read of Interlocked counters rather than a lock, so concurrent stores can overshoot a cap by roughly one shard's worth. Every clear resets the counters, so drift cannot ratchet. Inserts use TryAdd (not an indexer assignment) so that only a genuine insert moves a counter — otherwise two threads racing the same key would inflate them and permanently pull the ceilings in.
Only the CAS winner clears on a generation bump. InvalidateIfStale compare-exchanges the generation, so a post-deploy burst of requests repopulates the cache once instead of each thread clearing it out from under the others.

The hot path looks the entry up before it builds anything. The gate used to construct the node scope — splitting the s= identifier and allocating a NodeScope — and then allocate a five-capture closure for the walk, on every call including cache hits. Now the identifier is extracted once, the cache is consulted immediately, and a hit returns having built no scope, no closure and no UserAuthorizationState; the scope and the walk happen only on a miss. GetOrAdd was split into an allocation-free TryGet plus a Set to make that possible, and TryGet deliberately does not create a shard, so a hit-only session never mints one and a just-cleared shard is not resurrected until something is actually stored.

⚠️ Revocation is checked ahead of the cache, not behind it — the key-usability lookup runs above the TryGet, so a cache hit cannot serve an allow for a key that has since been disabled or expired. Reversing those two lines is the subtle way to reintroduce the bug the paragraph above warns about: the generation never moves when a key merely expires, so nothing would ever drop the stale allow. Two tests pin the ordering (Key_revocation_is_rechecked_on_a_cache_hit and An_expired_key_is_denied_even_though_its_allow_is_cached).

Deny reasons are composed lazily. A denial on the cached path logs at Debug, and the interpolated reason strings (which generation, which key, which operation) used to be built at every call site before the log level was consulted — so a Browse denied across a thousand children formatted a thousand strings with Debug off. The interpolating denials now guard on _log.IsEnabled(level) first, and the BadUserAccessDenied ServiceResult is a shared static instance rather than one allocation per denied node.

Everything denies when something is missing

No identity, no principal on the identity, no cluster identity on the node, a NodeId with no string identifier, or a trie that was never installed — each returns BadUserAccessDenied. The client is told nothing beyond that; the reason is logged server-side (AclGate: denied …) so the server cannot be used as an oracle for which nodes exist or which grants a principal holds.

The consequence to plan for: a wiring or configuration mistake presents as everything denied, not everything allowed. Confirm the trie exists at all before debugging a specific rule — the OpcUaPublish: data-plane authority generation N installed line reports the grant count, and a separate warning names every rule the build dropped (a dangling ScopeId, a blank one, a retired scope kind, or a scope path that does not start at a realm). A dropped rule can never match anything.

The `NodeAcl` + `PermissionTrie` subsystem

What is real about it:

Operators can author NodeAcl rows in the Admin UI (/clusters/{id}/acls), and those rows are validated, versioned and persisted.
ConfigComposer does snapshot every NodeAcl row into every deployment artifact, so an ACL edit shifts the artifact's RevisionHash and ships to every node.
The node side does deserialize them (#520 stage 4) — DeploymentArtifact.ParseDataPlaneAuthority reads the artifact's NodeAcls (scoped to the node's own cluster) and ApiKeys, and OpcUaPublishActor installs both on the node's IDataPlaneAuthoritySource on every apply. The node manager holds that source, so it has a cluster identity.
The same Apply call builds and installs the permission trie (#520 stage 5), under the same generation as the snapshot. Deliberately one call rather than two wirings: an uninstalled trie denies everything, which is indistinguishable from working enforcement until someone checks that an ALLOW is possible at all.
DataPlaneAuthorizer evaluates them at every data-plane seam (#520 stages 5–6), when the flag is on. With the flag off an operator can still author a grant, deploy it green, and have it change nothing beyond the two coarse role gates — that is the documented default, not an unfinished state.

The four things blocking a wire-up (all verified, none of them merely "call the evaluator"):

~~Raw-realm nodes have no representable scope.~~ Closed by #520 stage 3. NodeScope is now (ClusterId, Kind, Segments) with a variable-length path, and NodeHierarchyKind carries both Uns and Raw. The previous fixed-arity shape could not address a real raw tree at all — folder chains and tag-group chains both nest without bound — so it silently capped at five levels and mislabelled every one of them. Four Raw* scope kinds were appended for authoring.
~~The identity carries roles, not groups.~~ Dissolved by #520 stage 2. The column is now NodeAcl.Principal and the principal comes from the presented API key (or the reserved anonymous), not from a directory — so there is no group list to lose.
~~PermissionTrieBuilder's scopePaths argument has no producer.~~ Closed by #520 stage 3. AclScopePathResolver is that producer, built on both sides of the split — EF entities at the deploy gate, artifact JSON on the node — and scopePaths is now a required argument with no fallback: an unresolvable scope is dropped with a diagnostic rather than planted at a trie position the walk can never reach. A scope resolves to several paths when one entity is reachable at several NodeIds (a raw tag projected into N equipments has one raw path plus N UNS paths), because a grant that covered only some of them would be bypassable through the one it missed.
~~No session lifecycle exists.~~ Closed by #520 stage 5, though not as the blocker imagined. The generation exists and advances (stage 4), and DataPlaneAuthorizer now builds a UserAuthorizationState stamped with it — but per authorization call, not per session. The per-session design was LDAP-shaped: it assumed a directory could change a session's group membership out of band, so a long-lived session had to re-resolve on a timer and serve cached memberships in between. Under API keys nothing can change out of band — the principal comes from the key, and keys and grants both ride the deployment artifact — so the generation stamp is the entire invalidation signal, and rebuilding a small record per call picks it up for free while a cached one would have to be explicitly invalidated. The consequence is that UserAuthorizationState.IsStale and NeedsRefresh can never fire on this path; they are vestigial here rather than load-bearing. Caching becomes necessary at stage 6's read/browse/subscribe seams, which are on the publish path — and it will want a per-(session, node) decision cache, not this type.

NodeAclScopeKind.FolderSegment is the retired v2 kind with no v3 entity behind it. The stage-7 picker does not offer it at all, so it can only arrive on a legacy row or a hand-edited one; it can never resolve, and every end now says so specifically rather than reporting it as a missing id: the AdminUI refuses to save it, the deploy gate rejects it with NodeAclScopeKindRetired, and the trie builder drops it with a RetiredScopeKind diagnostic (Gitea #533 — it used to arrive as UnresolvedScope, which sent an operator holding a legacy row hunting for a typo in an id that was never the problem). One inconsistency remains: NodePermissions.AlarmRead has no OpcUaOperation mapping, so it is grantable but unreachable.

String identity in this subsystem (Gitea #533): identifiers — ScopeId, ClusterId, Principal — compare OrdinalIgnoreCase, because they are opaque handles whose spelling genuinely varies by source (a ClusterId arrives lower-cased through OTOPCUA_ROLES and verbatim through Cluster:Roles; a ScopeId comes from the stage-7 picker but a legacy row's may be anything). Path segments — every level of a scope path, and therefore PermissionTrieNode.Children — compare Ordinal, because they are browse names and Speed / speed are two different nodes that can coexist under one device. The rule is stated in full on AclScopeRef. It matters most where the deploy gate and the node are fed from different snapshots, which is exactly the split AclScopePathResolver exists to bridge: case drift there would silently drop every grant on the affected scope.

Authoring grants (#520 stage 7)

Grants are authored at /clusters/{clusterId}/acls. Two things about the surface are worth knowing before relying on it, because both close a way a grant could previously be saved and deployed while authorizing nobody.

The scope kind and the scope target are chosen together, from a tree. "Choose scope…" opens a picker showing the cluster's raw device tree (lazily expanded, exactly the /raw hierarchy) beside its UNS tree, plus a whole-namespace row on each side. Selecting a node sets ScopeKind and ScopeId in one action. Until this landed the two were an unrelated dropdown and a free-text box, so RawDevice + a TagId was a well-formed row: it saved, it deployed, and it resolved to nothing.

Consequences of the trees being rooted at the grant's own cluster and of a node supplying its own kind:

A cross-cluster scope is structurally unreachable — the picker's counterpart to the deploy gate's NodeAclScopeCrossCluster rule, rather than a second copy of it.
A Cluster row deliberately produces no ScopeId. The deploy gate rejects a Cluster grant that carries one (NodeAclScopeIdUnexpected), so a picker that helpfully filled in the cluster id would author rows that cannot be deployed.
UNS tags are not offered. A UNS signal is a UnsTagReference to a raw tag, and a Tag grant already covers the raw node and every UNS projection of it — so a UNS tag row would produce a grant that reads narrower than the one it creates. Pick the tag in the raw tree.
FolderSegment is not offered at all (see above).

Rows are resolved against the live configuration, at authoring time. IAclScopeService.CheckScopeAsync mirrors DraftValidator.ValidateNodeAclScopes rule for rule, so the page rejects what the deployment would reject:

The editor refuses to save a grant whose scope does not resolve — re-checked on submit, not trusted from when the form loaded, because an existing row's entity may have been deleted since and the operator may never touch the picker.
The list page resolves every row and marks a broken one unresolved with the reason, so a grant that has rotted since it was authored is visible without waiting for a deploy to fail.
The principal field suggests every known principal (API keys, existing grants, anonymous) and reports how many live keys currently authenticate as the one typed. Reaching nobody is a warning, not a block — authoring a grant before minting its key is legitimate; a typo is not, and the two are indistinguishable to the server.

The checks are in AclScopeService rather than in the Razor pages because this repo has no bUnit: logic only a page can reach is untestable, and "the grant authorizes nobody" is not a failure mode to leave unpinned.

Hierarchy (design)

ACLs are evaluated against the node's scope path. NodeScope (src/Core/ZB.MOM.WW.OtOpcUa.Core/Authorization/NodeScope.cs) carries a Kind that selects between two hierarchy shapes:

Equipment (UNS) kind:        Cluster → Namespace → UnsArea → UnsLine → Equipment → Tag
SystemPlatform (Galaxy) kind: Cluster → Namespace → FolderSegment(s) → Tag

On the Galaxy/SystemPlatform path each folder segment takes one trie level, so a deeply-nested Galaxy folder reaches the same depth as a full UNS path. Unset mid-path levels leave the corresponding id null and the evaluator walks only as far as the scope goes.

Each level can carry NodeAcl rows (src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Entities/NodeAcl.cs) that grant a permission bundle to a set of LdapGroups.

Permission flags

NodePermissions (src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/NodePermissions.cs), stored as an int bitmask in NodeAcl.PermissionFlags:

[Flags]
public enum NodePermissions : int
{
    None = 0,

    Browse            = 1 << 0,
    Read              = 1 << 1,
    Subscribe         = 1 << 2,
    HistoryRead       = 1 << 3,
    WriteOperate      = 1 << 4,
    WriteTune         = 1 << 5,
    WriteConfigure    = 1 << 6,
    AlarmRead         = 1 << 7,
    AlarmAcknowledge  = 1 << 8,
    AlarmConfirm      = 1 << 9,
    AlarmShelve       = 1 << 10,
    MethodCall        = 1 << 11,
    HistoryUpdate     = 1 << 12,   // OPC UA annotation / insert / delete (separate from HistoryRead)

    ReadOnly  = Browse | Read | Subscribe | HistoryRead | AlarmRead,
    Operator  = ReadOnly | WriteOperate | AlarmAcknowledge | AlarmConfirm,
    Engineer  = Operator | WriteTune | AlarmShelve,
    Admin     = Engineer | WriteConfigure | MethodCall,
}

The three Write tiers map to Galaxy's v1 SecurityClassification — FreeAccess/Operate → WriteOperate, Tune → WriteTune, Configure → WriteConfigure. SecuredWrite / VerifiedWrite / ViewOnly classifications remain read-only from OPC UA regardless of grant.

HistoryUpdate (bit 12) is a separate permission from HistoryRead so a read-only grant cannot also authorize OPC UA HistoryUpdate (annotation / insert / delete) operations. TriePermissionEvaluator maps OpcUaOperation.HistoryUpdate to this bit, closing the read⇒update hole that existed when the only history permission was HistoryRead. Note: HistoryUpdate is not included in any composite bundle (ReadOnly / Operator / Engineer / Admin) because the HistoryUpdate service surface (the actual insert/replace/delete backend RPC) is not yet implemented — a client calling HistoryUpdate still receives the SDK's default reject regardless of the grant.

Evaluator — `PermissionTrie`

src/Core/ZB.MOM.WW.OtOpcUa.Core/Authorization/:

Class	Role
`PermissionTrie`	Cluster-scoped trie; each node carries `(GroupId → NodePermissions)` grants.
`PermissionTrieBuilder`	Builds a trie from the current `NodeAcl` rows in one pass and installs it into the cache.
`PermissionTrieCache`	Process-singleton cache keyed on `(ClusterId, GenerationId)`. Generation-sealed: `Install(trie)` adds a new generation + advances the "current" pointer; older generations are retained (in-flight requests still resolve) and GC'd by `Prune`. `Invalidate(clusterId)` drops every cached trie for a cluster. There is no `AclChangeNotifier` — a publish installs a new generation rather than signalling an invalidation.
`TriePermissionEvaluator`	Implements `IPermissionEvaluator.Authorize(session, operation, scope)`. Walks the cluster trie for the supplied `NodeScope`, unions grants along the path, and returns an `AuthorizationDecision`. Evaluates against the session's bound generation (`session.AuthGenerationId`), not just "current", so a grant added/removed in a newer generation cannot take effect mid-session.

NodeScope is described above (Equipment-kind vs SystemPlatform-kind). The evaluator unions the matched grants along the path — a tag-level ACL and an area-level ACL both contribute.

Dispatch gate — `DataPlaneAuthorizer` over `IPermissionEvaluator`

IPermissionEvaluator.Authorize(UserAuthorizationState session, OpcUaOperation operation, NodeScope scope) (default impl TriePermissionEvaluator at src/Core/ZB.MOM.WW.OtOpcUa.Core/Authorization/TriePermissionEvaluator.cs) returns an AuthorizationDecision.

The dispatch gate exists since #520 stages 5–6 (2026-08-01): DataPlaneAuthorizer (src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/DataPlaneAuthorizer.cs, registered in Host/Program.cs, consulted by OtOpcUaNodeManager) calls Authorize at every data-plane seam — read, browse, subscribe, write, alarm commands and HistoryRead — when Security:Acl:Enabled is true. With the flag off (the default) the authorizer is constructed but not consulted, and the effective posture for Read/Browse/Subscribe/HistoryRead is default-ALLOW (see What is enforced with the flag off).

Properties of the wired path:

Driver-agnostic. No driver-level code participates in authorization decisions. Drivers report SecurityClassification as metadata on tag discovery; everything else flows through the evaluator.
Strictly fail-closed (default-deny) when consulted. Every guard path returns NotGranted — a stale session (past the staleness ceiling, decision #152), a cluster mismatch between session and scope, a missing trie, a pruned bound generation, or simply no matching grant. A misconfigured node therefore denies everything rather than allowing everything, which presents as an outage of the gated operations rather than a subtle permission bug.
Evaluator stays pure. TriePermissionEvaluator has no OPC UA stack dependency and is tested directly from xUnit.

Full model

See docs/v2/acl-design.md for the complete design: trie invalidation, flag semantics, per-path override rules, and the reasoning behind additive-only (no Deny). Most of it shipped with #520 stages 5–7 (behind Security:Acl:Enabled); where it and this document disagree, this document plus docs/plans/2026-07-31-apikey-acl-enforcement-design.md describe the running behaviour.

Note also that the SystemPlatform hierarchy kind named above was retired with the v3 dual-namespace address space; NodeHierarchyKind now has only Equipment.

Role grant source (data-plane)

Since #520 (2026-08-01), data-plane roles come from the authenticated principal's NodeAcl grants, not from LDAP. ApiKeyInterimRoleMapper (src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/ApiKeyInterimRoleMapper.cs, #520 stage 2) derives the session's server-wide role set from the grants naming the principal — a key's principal, or the reserved anonymous — and attaches it to the RoleCarryingUserIdentity the gates read. Neither Security:Ldap:GroupToRole nor the Config-DB LdapGroupRoleMapping table touches this path any more; both feed Admin UI login only.

Two properties of that derivation matter to an operator:

⚠️ Only Cluster-scoped grants confer a role. This is deliberate, and it under-grants: honouring a narrower grant's flags while unable to honour its scope would turn "WriteOperate on Equipment X" into server-wide write authority. A scope-narrower-than-cluster grant is simply inert with the flag off (the AdminUI banner says so) — it takes effect only when Security:Acl:Enabled turns scoped evaluation on.
Flag vocabulary ≠ role vocabulary. The WriteOperate permission flag maps to the WriteOperate role; the AlarmAcknowledge flag (the spelling an operator authors) maps to the AlarmAck role, which covers acknowledge/confirm/shelve as one gate. ReadOnly, WriteTune and WriteConfigure flags confer no role — with the flag off no gate reads them, and with it on the scoped ACL decision replaces roles entirely.

If a principal has no cluster-scoped grant carrying WriteOperate / AlarmAcknowledge, writes and alarm acknowledgement default-deny: they return BadUserAccessDenied even for sessions that authenticate successfully. (The same requirement gates both the scripted-alarm and the native Galaxy-alarm Part-9 ack/confirm/shelve paths.) Reads, browses, subscribes and history reads are unaffected with the flag off — they succeed with or without any grant.

Control-Plane Authorization

Control-plane authorization governs the Admin UI — who can view fleet config, edit drafts, publish generations, manage cluster nodes + credentials.

Per decision #150 control-plane roles are deliberately independent of data-plane ACLs. An operator who can read every OPC UA tag in production may not be allowed to edit cluster config; conversely a Designer may not have any data-plane grants at all.

Roles

The AdminRole enum (src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Enums/AdminRole.cs) defines three roles. Task 1.7 standardized the member names on the canonical ZB.MOM.WW.Auth CanonicalRole vocabulary (ConfigViewer → Viewer, ConfigEditor → Designer, FleetAdmin → Administrator); a data migration (CanonicalizeAdminRoles) rewrote existing rows. This was a rename, not a permission change.

Role	Capabilities
`Viewer`	Read-only access to drafts, generations, audit log, fleet status. (Was `ConfigViewer`.)
`Designer`	Viewer plus draft authoring (UNS, equipment, tags, ACLs, driver instances, reservations, CSV imports). Cannot publish. (Was `ConfigEditor`.)
`Administrator`	Designer plus publish, cluster/node CRUD, credential management, role-grant management. Satisfies both the `FleetAdmin` and `DriverOperator` authorization policies. (Was `FleetAdmin`.)

DriverOperator is an authorization policy name (kept stable), not an AdminRole member. It gates Reconnect / Restart commands against live driver instances from the Admin UI DriverStatusPanel and requires the canonical role Operator or Administrator (policy.RequireRole("Operator", "Administrator") in AddAuthorization, src/Server/ZB.MOM.WW.OtOpcUa.Security/ServiceCollectionExtensions.cs). Operator is an appsettings-only string role (not an AdminRole member); map an LDAP group to it via GroupToRole, e.g. "ot-driver-operator": "Operator". The FleetAdmin policy requires the Administrator role.

AdminUI page authorization policies (R2-05)

Every routable AdminUI page carries an explicit named-policy @attribute [Authorize(Policy = …)] — there are no bare [Authorize] (authenticated-only) config pages and no per-page Roles="…" strings. The policy names are constants on AdminUiPolicies (src/Server/ZB.MOM.WW.OtOpcUa.Security/Auth/AdminUiPolicies.cs), the single source of truth referenced by pages, AuthorizeView blocks, imperative IAuthorizationService.AuthorizeAsync checks, and endpoint groups (never string literals). All four are registered by AddOtOpcUaAuth:

Policy	Satisfied by roles	Gates
`AdminUiPolicies.ConfigEditor`	`Administrator`, `Designer`	The config-authoring surface: `/uns`, equipment, cluster/node/namespace/ACL editors, all 8 driver pages (which author live retry/breaker `ResilienceConfig`), Deployments, Scripts, ScriptEdit, and the `/api/script-analysis/*` endpoint group.
`AdminUiPolicies.FleetAdmin`	`Administrator`	`RoleGrants` page; the `Certificates` Trust/Untrust/Delete actions (`AuthorizeView` + server-side re-check).
`AdminUiPolicies.DriverOperator`	`Operator`, `Administrator`	Imperative live-ops actions only (Alerts ack/shelve, DriverStatusPanel reconnect/restart, live address pickers).
`AdminUiPolicies.AuthenticatedRead`	any authenticated user	The read-only dashboards, lists, and live tails (Home, Fleet, Hosts, Alerts, Certificates, cluster read views, etc.). Semantically identical to the `FallbackPolicy`; named so read pages carry a non-bare attribute.

Login is the only [AllowAnonymous] page (Admin-001). A reflection guard — PageAuthorizationGuardTests (tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Authorization/PageAuthorizationGuardTests.cs) — enumerates every routable type in the AdminUI assembly and fails the build if any page carries a bare [Authorize], a Roles="…" idiom, an unknown policy name, or is missing from the classification map. This is the anti-drift anchor (the repo has no bUnit): a new page fails the test until its author consciously classifies it. AdminUiPoliciesTests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests/AdminUiPoliciesTests.cs) pins each policy's role semantics.

In v2 the authentication + authorization stack is wired centrally by AddOtOpcUaAuth (src/Server/ZB.MOM.WW.OtOpcUa.Security/ServiceCollectionExtensions.cs), which also installs a FallbackPolicy that requires an authenticated user. Razor pages gate inline with the named-policy attribute above, e.g. @attribute [Authorize(Policy = AdminUiPolicies.ConfigEditor)]. Nav-menu sections hide via <AuthorizeView>.

Address-picker browse is an outbound-connect capability (trust boundary)

Worth stating plainly, because "browse a configured database" reads weaker than what it is: the AdminUI's live address pickers dial a target named in the browse form, from the AdminUI host, before anything is persisted. The Sql picker is the sharpest case — SqlDriverBrowser.OpenAsync (src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Browser/SqlDriverBrowser.cs) accepts an ad-hoc connectionString literal an operator pastes for a database that has no connectionStringRef yet, and hands it to the ADO.NET provider verbatim. That is an arbitrary host and port plus whatever the provider's keyword set allows (Microsoft.Data.SqlClient alone offers AttachDbFilename, Failover Partner, Enlist). The OPC UA client and MQTT pickers have the same shape against their own endpoint fields.

The control is the page policy, and it is admin-grade. The only surfaces that reach a browse session are /raw and /uns, both @attribute [Authorize(Policy = AdminUiPolicies.ConfigEditor)] — satisfied by Administrator or Designer only. There is no HTTP endpoint for browse: IBrowserSessionService is a scoped service consumed from those pages' Blazor circuits, so it inherits the page's policy and the FallbackPolicy beneath it, and PageAuthorizationGuardTests fails the build if either page's attribute drifts. A Viewer cannot open a picker; AdminUiPolicies.DriverOperator gates only the live-ops actions inside one (Sparkplug rebirth), never the connect.

Supporting hygiene, none of which is the control:

The pasted literal is never persisted — SqlDriverConfigDto has no such property, so the driver factory would drop it; the picker composes a session-only document and the browser keeps no field.
A literal supplied alongside a connectionStringRef logs a warning naming the shadowed ref (never any connection text), so the override is visible rather than silent.
Every failure on the open path is redacted (SqlDriverBrowser.Sanitize), and the provider's own connection-string-parser message is withheld outright because it quotes connection-string text verbatim.
NodeEdit.razor warns against pasting a literal into a saved DriverConfig, which would persist it.

If a deployment needs the browse surface narrower than "anyone who may author config", the lever is the page policy (and the LDAP group→role mapping behind it), not the browser.

Role grant source

Admin reads LdapGroupRoleMapping rows from the Config DB (src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Entities/LdapGroupRoleMapping.cs) — the same pattern as the data-plane NodeAcl but scoped to Admin roles + (optionally) one cluster for multi-site fleets (a system-wide row, IsSystemWide = true, stacks additively with cluster-scoped rows). The RoleGrants.razor page lets Administrators edit these mappings without leaving the UI.

Headless deploy API (`POST /api/deployments`)

For CI / scripts that need to trigger a deployment without driving the Blazor "Deploy current configuration" button, admin-role nodes expose POST /api/deployments (DeployApiEndpoints, src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Api/DeployApiEndpoints.cs). It forwards to the same IAdminOperationsClient.StartDeploymentAsync the button calls.

Auth is a single configured secret checked from the X-Api-Key header in fixed time — deliberately orthogonal to the cookie-only web auth (OPC UA Authentication above) so automation needs no LDAP login round-trip. The endpoint is AllowAnonymous so the FallbackPolicy doesn't 401 it, and enforces the key itself. It self-disables (503) until Security:DeployApiKey is set, so it is never open by default.

curl -X POST https://<admin-host>/api/deployments \
     -H 'X-Api-Key: <Security:DeployApiKey>' \
     -H 'Content-Type: application/json' \
     -d '{"createdBy":"ci-bot"}'

Responses: 202 Accepted ({ outcome, deploymentId, revisionHash }) when a deployment was sealed, 200 for NoChanges, 409 when another deployment is in flight, 422 when rejected, 401 for a missing/wrong key, 503 when unconfigured. Set the secret via Security:DeployApiKey (env Security__DeployApiKey) on admin nodes only; treat it like any deploy credential (rotate, keep out of source).

Mesh gRPC ingress is `AllowAnonymous` by design

Archreview 03/S14 (Critical, closed R3-01). On an admin-role node AddOtOpcUaAuth installs a global FallbackPolicy = RequireAuthenticatedUser bound to the cookie scheme (the same fallback that protects every AdminUI page above). ASP.NET Core's authorization middleware applies that fallback to every endpoint that carries no authorization metadata of its own — gRPC endpoints included. The five mesh gRPC endpoints (config-serve artifact fetch, central command ack, telemetry stream, node command, LocalDb sync) authenticate with a per-cluster/per-node PSK bearer inside a gRPC interceptor, and that interceptor runs inside the endpoint invocation — i.e. strictly after the authorization middleware. Without an explicit .AllowAnonymous() opt-out, a correctly-keyed node call is challenged by the cookie handler before the interceptor's verdict ever matters: on a login-enabled admin node this is a fleet-wide deploy-plane outage (artifact fetches fail, deploy acks are dropped, deployments seal TimedOut), and it went unnoticed for seven phases of prior live gates because the docker-dev rig always ran Security__Auth__DisableLogin=true (an always-succeeding handler registered under the cookie scheme name, so the fallback trivially passes) and every prior mesh-gRPC test built its own auth-free pipeline by hand.

Why AllowAnonymous is the correct fix, not a hole. All five interceptors are fail-closed: an unset key rejects every call, and the presented token is compared with CryptographicOperations.FixedTimeEquals. Program.cs:574 states the design contract directly — "the interceptor is the ONLY inbound auth on any endpoint (the libraries verify nothing)" — which was already true of the gRPC pipeline; the missing opt-out made it false of the HTTP pipeline in front of it. The pattern itself predates this fix: static assets (MapStaticAssets().AllowAnonymous()), the metrics endpoint (MapZbMetrics().AllowAnonymous()), and the headless deploy API above are all AllowAnonymous for the identical reason — each is gated by something other than the cookie, and the fallback would otherwise 401/redirect it regardless.

Endpoint	Role gate	Affected without `AllowAnonymous`, on a login-enabled node
`DeploymentArtifactService.Fetch`	`hasAdmin && ConfigServe:GrpcListenPort > 0`	Always — every `FetchAndCache` node's artifact fetch.
`CentralCommandService.ReportApply`	same	Always — every node's deploy ack, from every cluster.
`TelemetryStreamService.Subscribe`	`hasDriver && Telemetry:GrpcListenPort > 0`	Only on a fused `admin,driver` node (e.g. docker-dev's central pair) — a driver-only node never registers the cookie stack at all, so it is structurally unaffected either way.
`NodeCommandService.Execute`	same	Only on a fused node, same reasoning.
`localdb_sync.v1.LocalDbSync/Sync`	`hasDriver && LocalDb:SyncListenPort > 0`	Only on a fused node that also sets a sync port — no node in the fleet currently has that shape, but the map is anonymised anyway.

A fused central pair is affected on its OWN dispatch, not only inbound from sites. CentralCommunicationActor dials every enabled ClusterNode row, its own pair included, and central's telemetry dialer dials its own pair too — so a login-enabled fused central breaks its own command/telemetry plane, not just the sites'.

Mapping mechanics. All five maps live in one place, MeshGrpcEndpoints.MapOtOpcUaMeshGrpcServices (src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/MeshGrpcEndpoints.cs) — Program.cs calls it once and carries no map sites of its own. Four maps chain .AllowAnonymous() directly onto MapGrpcService<T>(). The fifth, MapZbLocalDbSync(), returns IEndpointRouteBuilder rather than an IEndpointConventionBuilder, so it cannot take the same chain; it is wrapped in an empty-prefix app.MapGroup(string.Empty).AllowAnonymous().MapZbLocalDbSync(), which carries the convention without changing the wire route pattern.

Three standing guards, all in tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/, so a future map cannot silently reopen this:

MeshGrpcAnonymousMetadataTests — an endpoint-metadata census that calls the SAME MapOtOpcUaMeshGrpcServices production helper across all three role shapes (admin-only, driver-only, fused) and asserts every mesh endpoint carries IAllowAnonymous, with an exact expected count per shape and a scope check that the opt-out never leaks onto an unrelated endpoint.
MeshGrpcMapConventionTests — a Program.cs source scan asserting zero direct MapGrpcService</ MapZbLocalDbSync( occurrences and exactly one call to the mapper, so a map added straight into Program.cs (bypassing the helper, and therefore invisible to the census) fails MeshGrpcMapConventionTests.
MeshGrpcAuthStackTests / MeshGrpcDriverOnlyPipelineTests drive the mesh endpoints through the REAL AddOtOpcUaAuth + UseAuthentication/UseAuthorization pipeline (DisableLogin=false) and the driver-only (no-auth-at-all) shape respectively — the CI-shaped reproduction of the outage, with a correctly-keyed call succeeding as the load-bearing assertion (a blocked call and a refused call both "fail"; only a genuine success through the real cookie stack proves the opt-out works) and an un-anonymised negative control proving the fixture's auth stack is actually live.

Rig: docker-dev/docker-compose.yml's central-1/central-2 Security__Auth__DisableLogin is ${OTOPCUA_DISABLE_LOGIN:-true} — default unchanged, but OTOPCUA_DISABLE_LOGIN=false docker compose ... up -d central-1 central-2 drives the rig through the real cookie login (alice/bob, password password, against the shared GLAuth) for exactly this kind of verification, instead of every prior mesh live gate running with the one middleware that breaks production neutralized.

See archreview/plans/R3-01-mesh-grpc-allowanonymous-plan.md (R3-01) for the full verification and design record, and docs/Configuration.md §MeshTransport / §ConfigSource-ConfigServe and docs/Telemetry.md for the affected listener ports.

OTOPCUA0001 Analyzer — Compile-Time Guard

Per-capability resilience (retry, timeout, circuit-breaker) is applied by CapabilityInvoker in src/Core/ZB.MOM.WW.OtOpcUa.Core/Resilience/. A driver-capability call made outside the invoker bypasses resilience entirely — which in production looks like inconsistent timeouts, un-wrapped retries, and unbounded blocking.

OTOPCUA0001 (Roslyn analyzer at src/Tooling/ZB.MOM.WW.OtOpcUa.Analyzers/UnwrappedCapabilityCallAnalyzer.cs) fires with category OtOpcUa.Resilience and default severity Warning (per AnalyzerReleases.Shipped.md) when a method on one of the seven guarded capability interfaces (IReadable, IWritable, ITagDiscovery, ISubscribable, IHostConnectivityProbe, IAlarmSource, IHistoryProvider — all in ZB.MOM.WW.OtOpcUa.Core.Abstractions) is invoked outside a lambda passed to CapabilityInvoker.ExecuteAsync / ExecuteWriteAsync. AlarmSurfaceInvoker is not a wrapper home — its own implementation is covered transitively because it routes through the inner CapabilityInvoker.ExecuteAsync. The analyzer walks up the syntax tree from the call site, finds any enclosing invoker invocation, and verifies the call lives transitively inside that invocation's anonymous-function argument — a sibling pattern (do the call, then invoke ExecuteAsync on something unrelated nearby) does not satisfy the rule.

The xunit.v3 + Shouldly suite at tests/Tooling/ZB.MOM.WW.OtOpcUa.Analyzers.Tests/UnwrappedCapabilityCallAnalyzerTests.cs covers the common fail/pass shapes + the sibling-pattern regression guard.

The rule is intentionally scoped to async surfaces — pure in-memory accessors like IHostConnectivityProbe.GetHostStatuses() return synchronously and do not require the invoker wrap.

Write-Outcome Self-Correction

When an OPC UA client writes a value and the driver reports a failure, the server automatically reverts the variable node to its prior value and surfaces the Bad-quality status code to the client. This prevents the node from showing a phantom Good value after a device-level write error. The revert is local to the node that received the write — no cluster coordination is required.

Additionally, an AuditWriteUpdateEvent is emitted on every write attempt (success or failure), consistent with OPC UA Part 4 audit requirements. The event carries the source node id, the requested value, and the final status code so audit log consumers can trace write outcomes without polling the node.

(Implemented in master 1d797c1c.)

Audit Logging

Server: authentication, certificate-validation, write, and write-denial events are logged through the regular Serilog rolling file sink; write outcomes also emit an AuditWriteUpdateEvent (see Write-Outcome Self-Correction).
Admin: AuditWriterActor (src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane/Audit/AuditWriterActor.cs) writes ConfigAuditLog rows (src/Core/ZB.MOM.WW.OtOpcUa.Configuration/Entities/ConfigAuditLog.cs) to the Config DB for publish, rollback, cluster-node CRUD, and credential rotation. Visible on the cluster Audit page (ClusterAudit.razor) for operators with Viewer or above.

Config secrets (`${secret:}` delivery)

OtOpcUa never commits secret values to appsettings*.json. Every real secret is supplied at deploy time — either as a plain environment variable, or as a ${secret:<name>} token backed by the shared ZB.MOM.WW.Secrets encrypted store.

A pre-host expander (SecretReferenceExpander.ExpandConfigurationAsync, wired in OtOpcUa.Host/Program.cs) walks the assembled configuration and rewrites every ${secret:<name>} token into its resolved plaintext before the host is built — so every downstream binder/validator (AddZbSerilog, AddOtOpcUaConfigDb, the first ValidateOnStart) sees resolved values. The store's key-encryption key (KEK) comes from the ZB_SECRETS_MASTER_KEY environment variable (Secrets:MasterKey:Source=Environment); the encrypted SQLite store lives at Secrets:SqlitePath.

The expander is fail-closed and section-agnostic: a ${secret:<name>} token whose secret is absent throws SecretNotFoundException at startup, regardless of which feature owns the key or whether that feature is enabled. Keys prefixed with _ (comment keys) are skipped, so a ${secret:...} example inside a _…Comment value is never resolved.

The five config secrets and their canonical secret names:

Config key	Owner	Secret name
`Security:Jwt:SigningKey`	`JwtOptions`	`otopcua/jwt/signing-key`
`Security:Ldap:ServiceAccountPassword`	`LdapOptions`	`otopcua/ldap/service-account-password`
`Security:DeployApiKey`	`DeployApiEndpoints`	`otopcua/deploy/api-key`
`ConnectionStrings:ConfigDb`	`AddOtOpcUaConfigDb`	`otopcua/sql/configdb-connstr`
`ServerHistorian:ApiKey`	`ServerHistorianOptions`	`otopcua/historian/api-key`

Delivery. By default these are delivered as plain environment variables (e.g. ServerHistorian__ApiKey=histgw_…), never committed. To deliver one from the encrypted store instead, seed it once with the secret CLI (from ZB.MOM.WW.Secrets), then supply the token in place of the literal — via env var or a deployment appsettings overlay:

secret set otopcua/historian/api-key <value>       # seed the encrypted store first
ServerHistorian__ApiKey='${secret:otopcua/historian/api-key}'

Only switch a value to a ${secret:} token after the secret is seeded — an unseeded token fails the boot fail-closed. Do not commit the KEK (ZB_SECRETS_MASTER_KEY) or any secret value.

For the production posture needed so every clustered node (admin, driver, and any fused role) resolves the same secrets from the same store, see docs/operations/2026-07-16-secrets-clustered-master-key.md.

Troubleshooting

Certificate trust failure

Check {PkiStoreRoot}/rejected/ for the client's cert. As a FleetAdmin, use the Admin UI Certificates page's Trust action to move it into trusted/ (or copy the .der to trusted/certs/ by hand); the SDK trust list reloads on the next handshake.

LDAP users can connect but fail authorization

Verify (a) Security:Ldap:GroupAttribute (default memberOf) returns the user's groups, (b) Security:Ldap:GroupToRole maps those groups to the expected roles, and (c) a NodeAcl grant exists at some level of the node's scope path that unions to the required permission. The data-plane evaluator is strictly default-deny — there is no fail-open mode to fall back on.

LDAP bind rejected as "insecure"

Set Security:Ldap:Transport = "Ldaps" (or "StartTls") with the matching port (636 for AD Ldaps), or temporarily set Security:Ldap:AllowInsecure = true in dev. Production Active Directory increasingly refuses plain-LDAP bind under LDAP-signing enforcement.

Stale ACL trie after a publish

A publish installs a new generation into PermissionTrieCache via PermissionTrieBuilder rather than signalling an invalidation; the evaluator binds each session to a generation. If grants appear stale, confirm the new generation was installed (publish completed) and that sessions re-resolved their auth state — a session past its staleness ceiling fails closed and must re-authenticate. As a last resort PermissionTrieCache.Invalidate(clusterId) drops a cluster's cached tries.

80 KiB Raw Blame History Unescape Escape