06c56d7719
Two independent defects surfaced in review of #520 stage 3.
#532 — Cluster:Roles had no OTOPCUA_ROLES fallback, despite two doc sites
promising one. A node configured only through the env var — which drives ALL of
Program.cs's conditional wiring, so it reads as the authoritative knob — bound an
empty Roles array and joined the cluster carrying no Akka roles at all. It ran,
served, and logged nothing wrong, while central reported "no driver-role nodes
are Up" and every deployment to it timed out. Not hypothetical: the field fix on
wonder-app-vd03 was to hand-add Cluster__Roles__0/1 to a Windows service's
registry environment. Anyone reading the code to diagnose that box would have
concluded from the XML doc that the fallback was already applied.
AddOtOpcUaCluster now post-configures AkkaClusterOptions through the shared
RoleParser.ResolveEffectiveRoles, so BuildClusterOptions (Akka's own member
roles), ClusterRoleInfo, the ControlPlane singleton scope and
ConfigDbApiKeyStore's ClusterId all see one answer. Two sibling validators
(ConfigSource, Telemetry) read Cluster:Roles directly and were exempting the
exact node they exist for — an env-only driver-only node has no ConfigDb and
could sit on ConfigSource:Mode=Direct unchallenged. They resolve the same way now.
Also collapsed the four open-coded `Roles.FirstOrDefault(IsClusterRole)` +
ClusterIdFromRole copies into RoleParser.ClusterIdFromRoles. They had drifted:
three classified the cluster role ordinally, so a verbatim-bound `Cluster-SITE-A`
yielded a null ClusterId — and a null ClusterId applies NO cluster filter, which
made #528's cross-cluster over-grant guard fail OPEN. IsClusterRole is now
case-insensitive; the ClusterId suffix keeps its authored spelling, because
normalizing it would silently rename the Akka role of a running deployment.
#533 — three string-identity rules in one authorization path. AclScopeRef used
default (ordinal) equality while the trie builder filtered ClusterId and hashed
principals OrdinalIgnoreCase; the deploy gate's cross-cluster check was ordinal
too, which was the worse half — it could reject a whole deploy over a ClusterId
case difference the runtime would have accepted. It only worked because resolver
and builder were handed the same NodeAcl instances.
The rule is now stated once, on AclScopeRef, and applied throughout:
identifiers (ScopeId, ClusterId, Principal) compare OrdinalIgnoreCase; path
segments compare Ordinal. AclScopePathResolver re-keys its own maps rather than
documenting a comparer contract for callers, so the two independent producers —
EF at the gate, artifact JSON on the node — cannot diverge from the outside.
PermissionTrieNode.Children stays Ordinal: those are browse names, and Speed /
speed are two nodes that can coexist under one device.
Split PermissionTrieBuildDiagnosticReason.RetiredScopeKind out of
UnresolvedScope. A legacy FolderSegment row and a typo'd id had the same message
and different remedies.
Falsified: five mutations, each reddening only its own guard — remove the
PostConfigure (3 tests), revert IsClusterRole to ordinal (3), revert AclScopeRef
equality (1), revert the resolver's map comparer (6), revert the gate's
comparisons (2), remove the FolderSegment short-circuit (1).
Test isolation: OTOPCUA_ROLES is process-global and xUnit runs classes in
parallel, so the three validator suites that now resolve effective roles share a
non-parallel collection. Without it, cases passing no roles would intermittently
observe another class's env var.
Full solution build clean; full suite green apart from two pre-existing failures
confirmed identical on 3a6b2a1d (PrimaryGateFailoverTests, and
DriverProbeRegistrationTests expecting 10 probe types where 12 now register)
plus the AbLegacy integration suite, which needs a docker fixture.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
154 lines
8.5 KiB
C#
154 lines
8.5 KiB
C#
namespace ZB.MOM.WW.OtOpcUa.Cluster;
|
|
|
|
public static class RoleParser
|
|
{
|
|
/// <summary>The admin-role name.</summary>
|
|
public const string Admin = "admin";
|
|
|
|
/// <summary>The driver-role name. Single source of truth — downstream <c>DriverRole</c>
|
|
/// constants (<c>RedundancyStateActor</c>, <c>ClusterNodeAddressReconcilerActor</c>,
|
|
/// <c>ServiceCollectionExtensions</c>) alias this value rather than duplicating the literal.</summary>
|
|
public const string Driver = "driver";
|
|
|
|
/// <summary>The dev-role name.</summary>
|
|
public const string Dev = "dev";
|
|
|
|
/// <summary>Prefix identifying a cluster-scoped role, e.g. <c>cluster-SITE-A</c> (per-cluster mesh, Phase 6).</summary>
|
|
public const string ClusterRolePrefix = "cluster-";
|
|
|
|
private static readonly HashSet<string> Allowed = new(StringComparer.Ordinal)
|
|
{
|
|
Admin, Driver, Dev,
|
|
};
|
|
|
|
/// <summary>Returns true when <paramref name="role"/> is a cluster-scoped role — starts with
|
|
/// <see cref="ClusterRolePrefix"/> and carries a non-empty ClusterId suffix.</summary>
|
|
/// <remarks>
|
|
/// The prefix test is <b>case-insensitive</b> (Gitea #532). It was ordinal, which meant a
|
|
/// <c>Cluster:Roles</c> entry spelled <c>Cluster-SITE-A</c> — that path binds verbatim, unlike
|
|
/// the <see cref="Parse"/> path, which lower-cases — classified as an ordinary role. The node
|
|
/// then resolved a <see langword="null"/> ClusterId while still gossiping a cluster-scoped role,
|
|
/// so it looked split-topology to its peers and un-scoped to itself.
|
|
/// <see cref="SplitTopologyTransportValidator"/> pre-normalized for exactly this reason;
|
|
/// <c>ClusterRoleInfo</c> did not, and got the null.
|
|
/// </remarks>
|
|
/// <param name="role">The role name to test.</param>
|
|
public static bool IsClusterRole(string role) =>
|
|
role is not null
|
|
&& role.StartsWith(ClusterRolePrefix, StringComparison.OrdinalIgnoreCase)
|
|
&& role.Length > ClusterRolePrefix.Length;
|
|
|
|
/// <summary>Extracts the ClusterId from a cluster-scoped role, e.g. <c>cluster-SITE-A</c> → <c>SITE-A</c>.
|
|
/// Assumes <see cref="IsClusterRole"/> is true for <paramref name="role"/>.</summary>
|
|
/// <remarks>
|
|
/// The suffix keeps the operator's own spelling. It is NOT case-normalized here because the two
|
|
/// role sources disagree by construction — <c>Cluster:Roles</c> binds verbatim
|
|
/// (<c>cluster-SITE-A</c> ⇒ <c>SITE-A</c>) while <see cref="Parse"/> lower-cases the
|
|
/// <c>OTOPCUA_ROLES</c> path (⇒ <c>site-a</c>) — and normalizing one side would silently rename
|
|
/// the Akka role of a running deployment (docker-dev's rig configures <c>cluster-MAIN</c> /
|
|
/// <c>cluster-SITE-A</c> verbatim today). Every consumer of a ClusterId therefore compares it
|
|
/// case-insensitively; see the identity rule on <c>AclScopeRef</c> (Gitea #533).
|
|
/// </remarks>
|
|
/// <param name="role">The cluster-scoped role name.</param>
|
|
public static string ClusterIdFromRole(string role) => role[ClusterRolePrefix.Length..];
|
|
|
|
/// <summary>
|
|
/// The cluster-scoped roles in <paramref name="roles"/>, in the order given. More than one is a
|
|
/// misconfiguration the caller is expected to report — a node belongs to exactly one mesh.
|
|
/// </summary>
|
|
/// <param name="roles">The node's roles.</param>
|
|
/// <returns>The cluster-scoped roles, original spelling preserved.</returns>
|
|
public static IReadOnlyList<string> ClusterRolesIn(IEnumerable<string>? roles) =>
|
|
roles is null ? Array.Empty<string>() : roles.Where(IsClusterRole).ToArray();
|
|
|
|
/// <summary>The first cluster-scoped role in <paramref name="roles"/>, or null when there is none.</summary>
|
|
/// <param name="roles">The node's roles.</param>
|
|
/// <returns>The cluster role, e.g. <c>cluster-SITE-A</c>.</returns>
|
|
public static string? ClusterRoleFromRoles(IEnumerable<string>? roles) =>
|
|
roles?.FirstOrDefault(IsClusterRole);
|
|
|
|
/// <summary>
|
|
/// The ClusterId this node belongs to, derived from its own roles; null when it carries no
|
|
/// cluster-scoped role (a legacy / single-mesh node).
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// The single home for a derivation that was open-coded at four call sites before Gitea #532
|
|
/// (<c>ClusterRoleInfo</c>, the ControlPlane singleton scope, <c>SplitTopologyTransportValidator</c>
|
|
/// and <c>ConfigDbApiKeyStore</c>) — each with its own view of whether the classification was
|
|
/// case-sensitive, which is how the mixed-case gap above survived in three of the four.
|
|
/// </remarks>
|
|
/// <param name="roles">The node's roles.</param>
|
|
/// <returns>The ClusterId, or <see langword="null"/>.</returns>
|
|
public static string? ClusterIdFromRoles(IEnumerable<string>? roles)
|
|
{
|
|
var role = ClusterRoleFromRoles(roles);
|
|
return role is null ? null : ClusterIdFromRole(role);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Resolves a node's <b>effective</b> Akka roles: <c>Cluster:Roles</c> when it is non-empty,
|
|
/// otherwise the <c>OTOPCUA_ROLES</c> environment variable through <see cref="Parse"/>.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// ⚠️ <b>This fallback was documented for a year before it existed</b> (Gitea #532).
|
|
/// <c>AkkaClusterOptions.Roles</c>' own XML doc and <see cref="SplitTopologyTransportValidator"/>
|
|
/// both described it; only the validator implemented it, privately. A node configured solely
|
|
/// through <c>OTOPCUA_ROLES</c> — which drives every bit of <c>Program.cs</c>'s conditional
|
|
/// wiring, so it reads as the authoritative knob — joined the cluster carrying <b>no Akka
|
|
/// roles at all</b>. It ran, served, and logged nothing wrong, while central reported "no
|
|
/// driver-role nodes are Up" and every deployment to it timed out. That is not hypothetical:
|
|
/// see <c>docs/deployments/wonder-app-vd03-makino-z-34184.md</c>, where the field fix was to
|
|
/// hand-add <c>Cluster__Roles__0/1</c> to a Windows service's registry environment.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>Configured wins outright</b> — a stray <c>OTOPCUA_ROLES</c> can never widen an explicit
|
|
/// <c>Cluster:Roles</c>. The two sources normalize differently on purpose (see
|
|
/// <see cref="ClusterIdFromRole"/>), so a redundant pair should be configured through the
|
|
/// same one.
|
|
/// </para>
|
|
/// </remarks>
|
|
/// <param name="configuredRoles">The bound <c>Cluster:Roles</c> value.</param>
|
|
/// <param name="envRoles">The raw <c>OTOPCUA_ROLES</c> value.</param>
|
|
/// <returns>The effective roles; empty when neither source names any.</returns>
|
|
/// <exception cref="ArgumentException">The environment value names an unknown role.</exception>
|
|
public static string[] ResolveEffectiveRoles(IReadOnlyList<string>? configuredRoles, string? envRoles)
|
|
{
|
|
if (configuredRoles is { Count: > 0 }) return [.. configuredRoles];
|
|
return Parse(envRoles);
|
|
}
|
|
|
|
/// <summary>Reads the effective roles for the current process, resolving the environment itself.</summary>
|
|
/// <param name="configuredRoles">The bound <c>Cluster:Roles</c> value.</param>
|
|
/// <returns>The effective roles.</returns>
|
|
public static string[] ResolveEffectiveRoles(IReadOnlyList<string>? configuredRoles) =>
|
|
ResolveEffectiveRoles(configuredRoles, Environment.GetEnvironmentVariable(RolesEnvVar));
|
|
|
|
/// <summary>The environment variable the role list falls back to when <c>Cluster:Roles</c> is empty.</summary>
|
|
public const string RolesEnvVar = "OTOPCUA_ROLES";
|
|
|
|
/// <summary>Parses a comma-separated string of role names into a validated array.</summary>
|
|
/// <param name="raw">The raw role string to parse.</param>
|
|
/// <returns>The distinct, lower-cased, validated role names.</returns>
|
|
public static string[] Parse(string? raw)
|
|
{
|
|
if (string.IsNullOrWhiteSpace(raw)) return Array.Empty<string>();
|
|
|
|
var roles = raw
|
|
.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)
|
|
.Select(r => r.ToLowerInvariant())
|
|
.Distinct()
|
|
.ToArray();
|
|
|
|
foreach (var r in roles)
|
|
{
|
|
if (!Allowed.Contains(r) && !IsClusterRole(r))
|
|
throw new ArgumentException(
|
|
$"Unknown role '{r}'. Allowed: {string.Join(", ", Allowed)}, or '{ClusterRolePrefix}<ClusterId>'.",
|
|
nameof(raw));
|
|
}
|
|
|
|
return roles;
|
|
}
|
|
}
|