Cluster:Roles has no OTOPCUA_ROLES fallback, despite docs claiming one — ClusterId resolves null #532

New Issue

2026-08-01T01:10:31-04:00

dohertj2 commented

2026-08-01 01:10:31 -04:00

Found while fixing the ConfigDbApiKeyStore cross-cluster over-grant (see the companion issue).

AkkaClusterOptions.Roles' own XML doc and SplitTopologyTransportValidator (lines ~63-68) both state that
"when empty the role list comes from OTOPCUA_ROLES via RoleParser". No such fallback exists in code.
AddOtOpcUaCluster only does AddValidatedOptions<AkkaClusterOptions>(configuration, "Cluster"); the
validator cross-reads the env var itself, and Program.cs parses it separately for role gating.

Consequence. On a node configured only via OTOPCUA_ROLES with no Cluster:Roles, anything deriving a
ClusterId from AkkaClusterOptions gets null. For the over-grant fix that means it fails open — the node
keeps the old cross-cluster behaviour. The same derivation backs IClusterRoleInfo.ClusterId and the Phase-6
per-cluster redundancy singleton scope, so this is not confined to ACLs.

docker-dev sets both, so the rig does not exhibit it; a real deployment setting only the env var would.

Decide one of: implement the documented fallback, or delete the claim from both doc sites and make the
env-only configuration a validation failure. Silently disagreeing docs are the worst of the three.

Related, same area: RoleParser.IsClusterRole uses an ordinal prefix test, so a mixed-case
Cluster-SITE-A in Cluster:Roles yields a null ClusterId too. SplitTopologyTransportValidator normalizes
for this; ClusterRoleInfo does not.

There is also no shared AkkaClusterOptions → ClusterId helper — four call sites now open-code
Roles.FirstOrDefault(RoleParser.IsClusterRole) + ClusterIdFromRole (ClusterRoleInfo,
ControlPlane/ServiceCollectionExtensions, SplitTopologyTransportValidator, ConfigDbApiKeyStore).
A RoleParser.ClusterIdFromRoles(string[]) would collapse all four and give the fallback one home.

Found while fixing the `ConfigDbApiKeyStore` cross-cluster over-grant (see the companion issue). `AkkaClusterOptions.Roles`' own XML doc and `SplitTopologyTransportValidator` (lines ~63-68) both state that "when empty the role list comes from `OTOPCUA_ROLES` via `RoleParser`". **No such fallback exists in code.** `AddOtOpcUaCluster` only does `AddValidatedOptions<AkkaClusterOptions>(configuration, "Cluster")`; the validator cross-reads the env var itself, and `Program.cs` parses it separately for role gating. **Consequence.** On a node configured *only* via `OTOPCUA_ROLES` with no `Cluster:Roles`, anything deriving a ClusterId from `AkkaClusterOptions` gets null. For the over-grant fix that means it fails **open** — the node keeps the old cross-cluster behaviour. The same derivation backs `IClusterRoleInfo.ClusterId` and the Phase-6 per-cluster redundancy singleton scope, so this is not confined to ACLs. docker-dev sets both, so the rig does not exhibit it; a real deployment setting only the env var would. Decide one of: implement the documented fallback, or delete the claim from both doc sites and make the env-only configuration a validation failure. Silently disagreeing docs are the worst of the three. Related, same area: `RoleParser.IsClusterRole` uses an ordinal prefix test, so a mixed-case `Cluster-SITE-A` in `Cluster:Roles` yields a null ClusterId too. `SplitTopologyTransportValidator` normalizes for this; `ClusterRoleInfo` does not. There is also **no shared `AkkaClusterOptions → ClusterId` helper** — four call sites now open-code `Roles.FirstOrDefault(RoleParser.IsClusterRole)` + `ClusterIdFromRole` (`ClusterRoleInfo`, `ControlPlane/ServiceCollectionExtensions`, `SplitTopologyTransportValidator`, `ConfigDbApiKeyStore`). A `RoleParser.ClusterIdFromRoles(string[])` would collapse all four and give the fallback one home.

dohertj2 referenced this issue

2026-08-01 01:10:46 -04:00

Node ACLs are authored and deployed but never enforced — wire IPermissionEvaluator or retire the surface #520

dohertj2 commented

2026-08-01 01:51:56 -04:00

Fixed in 06c56d77 (branch fix/gitea-532-533-roles-and-string-identity).

Decision: implement the documented fallback. The issue framed this as a three-way choice and warned that silently disagreeing docs were the worst option. What settled it was docs/deployments/wonder-app-vd03-makino-z-34184.md, which records this defect happening in the field — that node joined role-less, central reported "no driver-role nodes are Up", no deployment could complete, and the fix was hand-adding Cluster__Roles__0/1 to a Windows service's registry environment. That doc's own product follow-up asked for exactly this. Deleting the claim would have left a real deployment shape broken on purpose.

What shipped

AddOtOpcUaCluster post-configures AkkaClusterOptions through a new shared RoleParser.ResolveEffectiveRoles. One place, so BuildClusterOptions (Akka's own member roles), ClusterRoleInfo, the ControlPlane singleton scope and ConfigDbApiKeyStore's ClusterId all get the same answer. Configured roles still win outright.
The four open-coded derivations collapse into RoleParser.ClusterIdFromRoles, and IsClusterRole is case-insensitive. That closes the second half of the over-grant fail-open: three of the four classified ordinally, so a verbatim-bound Cluster-SITE-A produced a null ClusterId, and a null ClusterId applies no filter at all.

Two things found while fixing it, both included

ConfigSourceOptionsValidator and TelemetryOptionsValidator also read Cluster:Roles directly with no fallback — the issue named only SplitTopologyTransportValidator. The ConfigSource one is the sharper of the two: it exempted precisely the node it exists for, since an env-only driver-only node has no ConfigDb (Program.cs gates that on the same env var) yet saw an empty role list and could sit on Mode=Direct unchallenged, reading its configuration from nothing. Both resolve through the shared helper now. Note the consequence: such a node now fails host start instead of booting into that state.

OTOPCUA_ROLES is process-global and xUnit runs test classes in parallel, so once those two validators read it, their no-roles cases could observe another class's env var. The three suites share a non-parallel collection.

Deliberately not done: normalizing the ClusterId suffix. Cluster:Roles binds verbatim and RoleParser.Parse lower-cases the env path, so the two sources produce SITE-A and site-a. Folding either would rename the Akka role of a running deployment — the docker-dev rig configures cluster-MAIN / cluster-SITE-A verbatim today, and a mid-upgrade role rename splits a singleton. Consumers compare ClusterIds case-insensitively instead, which #533 makes the stated rule rather than an accident. The residual is documented in docs/Configuration.md: configure both nodes of a pair through the same source.

Falsified by reverting the PostConfigure (3 tests red) and by reverting IsClusterRole to ordinal (3 tests red), each time only its own guard.

Fixed in `06c56d77` (branch `fix/gitea-532-533-roles-and-string-identity`). **Decision: implement the documented fallback.** The issue framed this as a three-way choice and warned that silently disagreeing docs were the worst option. What settled it was `docs/deployments/wonder-app-vd03-makino-z-34184.md`, which records this defect happening in the field — that node joined role-less, central reported "no driver-role nodes are Up", no deployment could complete, and the fix was hand-adding `Cluster__Roles__0/1` to a Windows service's registry environment. That doc's own product follow-up asked for exactly this. Deleting the claim would have left a real deployment shape broken on purpose. **What shipped** - `AddOtOpcUaCluster` post-configures `AkkaClusterOptions` through a new shared `RoleParser.ResolveEffectiveRoles`. One place, so `BuildClusterOptions` (Akka's own member roles), `ClusterRoleInfo`, the ControlPlane singleton scope and `ConfigDbApiKeyStore`'s ClusterId all get the same answer. Configured roles still win outright. - The four open-coded derivations collapse into `RoleParser.ClusterIdFromRoles`, and `IsClusterRole` is case-insensitive. That closes the second half of the over-grant fail-open: three of the four classified ordinally, so a verbatim-bound `Cluster-SITE-A` produced a null ClusterId, and a null ClusterId applies no filter at all. **Two things found while fixing it, both included** `ConfigSourceOptionsValidator` and `TelemetryOptionsValidator` also read `Cluster:Roles` directly with no fallback — the issue named only `SplitTopologyTransportValidator`. The ConfigSource one is the sharper of the two: it exempted precisely the node it exists for, since an env-only driver-only node has no ConfigDb (Program.cs gates that on the same env var) yet saw an empty role list and could sit on `Mode=Direct` unchallenged, reading its configuration from nothing. Both resolve through the shared helper now. Note the consequence: such a node now **fails host start** instead of booting into that state. `OTOPCUA_ROLES` is process-global and xUnit runs test classes in parallel, so once those two validators read it, their no-roles cases could observe another class's env var. The three suites share a non-parallel collection. **Deliberately not done: normalizing the ClusterId suffix.** `Cluster:Roles` binds verbatim and `RoleParser.Parse` lower-cases the env path, so the two sources produce `SITE-A` and `site-a`. Folding either would rename the Akka role of a running deployment — the docker-dev rig configures `cluster-MAIN` / `cluster-SITE-A` verbatim today, and a mid-upgrade role rename splits a singleton. Consumers compare ClusterIds case-insensitively instead, which #533 makes the stated rule rather than an accident. The residual is documented in `docs/Configuration.md`: configure both nodes of a pair through the same source. Falsified by reverting the PostConfigure (3 tests red) and by reverting `IsClusterRole` to ordinal (3 tests red), each time only its own guard.

dohertj2 closed this issue

2026-08-01 01:51:56 -04:00

dohertj2 referenced this issue

2026-08-01 01:51:56 -04:00

ACL: three different string-identity rules in one authorization path #533

dohertj2 referenced this issue from a commit

2026-08-01 02:38:21 -04:00

fix(cluster,acl): OTOPCUA_ROLES fallback + one string-identity rule (#532, #533)

dohertj2 referenced this issue from a commit

2026-08-01 02:38:21 -04:00

merge: ACL scope model + role/identity fixes + four real test failures (#520 s3, #532, #533, #534)

dohertj2 referenced this issue