06c56d7719
Two independent defects surfaced in review of #520 stage 3.
#532 — Cluster:Roles had no OTOPCUA_ROLES fallback, despite two doc sites
promising one. A node configured only through the env var — which drives ALL of
Program.cs's conditional wiring, so it reads as the authoritative knob — bound an
empty Roles array and joined the cluster carrying no Akka roles at all. It ran,
served, and logged nothing wrong, while central reported "no driver-role nodes
are Up" and every deployment to it timed out. Not hypothetical: the field fix on
wonder-app-vd03 was to hand-add Cluster__Roles__0/1 to a Windows service's
registry environment. Anyone reading the code to diagnose that box would have
concluded from the XML doc that the fallback was already applied.
AddOtOpcUaCluster now post-configures AkkaClusterOptions through the shared
RoleParser.ResolveEffectiveRoles, so BuildClusterOptions (Akka's own member
roles), ClusterRoleInfo, the ControlPlane singleton scope and
ConfigDbApiKeyStore's ClusterId all see one answer. Two sibling validators
(ConfigSource, Telemetry) read Cluster:Roles directly and were exempting the
exact node they exist for — an env-only driver-only node has no ConfigDb and
could sit on ConfigSource:Mode=Direct unchallenged. They resolve the same way now.
Also collapsed the four open-coded `Roles.FirstOrDefault(IsClusterRole)` +
ClusterIdFromRole copies into RoleParser.ClusterIdFromRoles. They had drifted:
three classified the cluster role ordinally, so a verbatim-bound `Cluster-SITE-A`
yielded a null ClusterId — and a null ClusterId applies NO cluster filter, which
made #528's cross-cluster over-grant guard fail OPEN. IsClusterRole is now
case-insensitive; the ClusterId suffix keeps its authored spelling, because
normalizing it would silently rename the Akka role of a running deployment.
#533 — three string-identity rules in one authorization path. AclScopeRef used
default (ordinal) equality while the trie builder filtered ClusterId and hashed
principals OrdinalIgnoreCase; the deploy gate's cross-cluster check was ordinal
too, which was the worse half — it could reject a whole deploy over a ClusterId
case difference the runtime would have accepted. It only worked because resolver
and builder were handed the same NodeAcl instances.
The rule is now stated once, on AclScopeRef, and applied throughout:
identifiers (ScopeId, ClusterId, Principal) compare OrdinalIgnoreCase; path
segments compare Ordinal. AclScopePathResolver re-keys its own maps rather than
documenting a comparer contract for callers, so the two independent producers —
EF at the gate, artifact JSON on the node — cannot diverge from the outside.
PermissionTrieNode.Children stays Ordinal: those are browse names, and Speed /
speed are two nodes that can coexist under one device.
Split PermissionTrieBuildDiagnosticReason.RetiredScopeKind out of
UnresolvedScope. A legacy FolderSegment row and a typo'd id had the same message
and different remedies.
Falsified: five mutations, each reddening only its own guard — remove the
PostConfigure (3 tests), revert IsClusterRole to ordinal (3), revert AclScopeRef
equality (1), revert the resolver's map comparer (6), revert the gate's
comparisons (2), remove the FolderSegment short-circuit (1).
Test isolation: OTOPCUA_ROLES is process-global and xUnit runs classes in
parallel, so the three validator suites that now resolve effective roles share a
non-parallel collection. Without it, cases passing no roles would intermittently
observe another class's env var.
Full solution build clean; full suite green apart from two pre-existing failures
confirmed identical on 3a6b2a1d (PrimaryGateFailoverTests, and
DriverProbeRegistrationTests expecting 10 probe types where 12 now register)
plus the AbLegacy integration suite, which needs a docker fixture.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
196 lines
8.5 KiB
C#
196 lines
8.5 KiB
C#
using Microsoft.Extensions.Configuration;
|
|
using Microsoft.Extensions.Options;
|
|
|
|
namespace ZB.MOM.WW.OtOpcUa.Cluster;
|
|
|
|
/// <summary>
|
|
/// Node-side selection of the transport a node's live-telemetry stream is carried over
|
|
/// (per-cluster mesh Phase 5). The dark switch that lets a node stop publishing telemetry over
|
|
/// the mesh-wide DistributedPubSub topic and instead serve it over a dedicated gRPC stream that
|
|
/// does not require sharing a gossip ring with whoever is watching.
|
|
/// </summary>
|
|
public sealed class TelemetryOptions
|
|
{
|
|
/// <summary>Configuration section name.</summary>
|
|
public const string SectionName = "Telemetry";
|
|
|
|
/// <summary>Publish telemetry over the mesh-wide DistributedPubSub topic — the default.</summary>
|
|
public const string ModeDps = "Dps";
|
|
|
|
/// <summary>Serve telemetry over a dedicated gRPC stream.</summary>
|
|
public const string ModeGrpc = "Grpc";
|
|
|
|
/// <summary>
|
|
/// <see cref="ModeDps"/> (default) or <see cref="ModeGrpc"/>. Any other value fails host
|
|
/// start.
|
|
/// </summary>
|
|
public string Mode { get; set; } = ModeDps;
|
|
|
|
/// <summary>
|
|
/// Dedicated gRPC listen port for the telemetry stream. <c>0</c> (the default) disables it —
|
|
/// nothing is bound. Required on a driver-role node under <see cref="ModeGrpc"/>.
|
|
/// </summary>
|
|
public int GrpcListenPort { get; set; }
|
|
|
|
/// <summary>
|
|
/// Shared bearer key the serve-side interceptor checks. Supply via the environment
|
|
/// (<c>Telemetry__ApiKey</c>) — never commit it. Required under <see cref="ModeGrpc"/> on any
|
|
/// roled node — fail-closed against hosting an un-keyed telemetry surface.
|
|
/// </summary>
|
|
public string ApiKey { get; set; } = string.Empty;
|
|
}
|
|
|
|
/// <summary>
|
|
/// Central-side selection of how central dials a node for live telemetry (per-cluster mesh
|
|
/// Phase 5). Mirrors <see cref="TelemetryOptions"/>'s mode but carries the dial-side knobs
|
|
/// (contact refresh cadence, per-call timeout) instead of a listen port.
|
|
/// </summary>
|
|
public sealed class TelemetryDialOptions
|
|
{
|
|
/// <summary>Configuration section name.</summary>
|
|
public const string SectionName = "TelemetryDial";
|
|
|
|
/// <summary>Read telemetry from the mesh-wide DistributedPubSub topic — the default.</summary>
|
|
public const string ModeDps = "Dps";
|
|
|
|
/// <summary>Dial nodes' dedicated gRPC telemetry streams.</summary>
|
|
public const string ModeGrpc = "Grpc";
|
|
|
|
/// <summary>
|
|
/// <see cref="ModeDps"/> (default) or <see cref="ModeGrpc"/>. Any other value fails host
|
|
/// start.
|
|
/// </summary>
|
|
public string Mode { get; set; } = ModeDps;
|
|
|
|
/// <summary>
|
|
/// Shared bearer key; must equal a dialled node's <see cref="TelemetryOptions.ApiKey"/>.
|
|
/// Supply via the environment (<c>TelemetryDial__ApiKey</c>) — never commit it. Required
|
|
/// under <see cref="ModeGrpc"/>.
|
|
/// </summary>
|
|
public string ApiKey { get; set; } = string.Empty;
|
|
|
|
/// <summary>How often central refreshes its set of dialable node contacts, in seconds.</summary>
|
|
public int ContactRefreshSeconds { get; set; } = 60;
|
|
|
|
/// <summary>Per-call deadline for a gRPC telemetry dial, in seconds.</summary>
|
|
public int CallTimeoutSeconds { get; set; } = 30;
|
|
}
|
|
|
|
/// <summary>
|
|
/// Fails the host at startup on a <see cref="TelemetryOptions"/> shape that would leave a node
|
|
/// unable to serve its live-telemetry stream.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// Like <see cref="ConfigSourceOptionsValidator"/>, every fault caught here otherwise surfaces as
|
|
/// an <i>absence</i> — a telemetry consumer that simply never sees data from this node, with no
|
|
/// stack trace and no failing node to point at; refusing to start is cheaper to diagnose.
|
|
/// </remarks>
|
|
public sealed class TelemetryOptionsValidator : IValidateOptions<TelemetryOptions>
|
|
{
|
|
private readonly IConfiguration _configuration;
|
|
|
|
/// <summary>
|
|
/// DI-constructed by <c>AddValidatedOptions</c> (a plain <c>AddSingleton</c>), so a
|
|
/// constructor dependency is safe here. <see cref="IConfiguration"/> — not
|
|
/// <c>IClusterRoleInfo</c> — is the source of this node's roles: <c>IClusterRoleInfo</c>'s
|
|
/// implementation needs the <c>ActorSystem</c>, which does not exist yet at
|
|
/// <c>ValidateOnStart</c> time.
|
|
/// </summary>
|
|
public TelemetryOptionsValidator(IConfiguration configuration)
|
|
{
|
|
_configuration = configuration;
|
|
}
|
|
|
|
/// <inheritdoc />
|
|
public ValidateOptionsResult Validate(string? name, TelemetryOptions options)
|
|
{
|
|
ArgumentNullException.ThrowIfNull(options);
|
|
|
|
var errors = new List<string>();
|
|
|
|
var isDps = string.Equals(options.Mode, TelemetryOptions.ModeDps, StringComparison.OrdinalIgnoreCase);
|
|
var isGrpc = string.Equals(options.Mode, TelemetryOptions.ModeGrpc, StringComparison.OrdinalIgnoreCase);
|
|
|
|
if (!isDps && !isGrpc)
|
|
{
|
|
errors.Add(
|
|
$"{TelemetryOptions.SectionName}:{nameof(TelemetryOptions.Mode)} is '{options.Mode}'. "
|
|
+ $"Expected '{TelemetryOptions.ModeDps}' or '{TelemetryOptions.ModeGrpc}'.");
|
|
}
|
|
|
|
// Through ResolveEffectiveRoles so the OTOPCUA_ROLES fallback applies (Gitea #532) — a node
|
|
// configured only through the env var is still a driver node and must satisfy the Grpc-mode
|
|
// requirements below.
|
|
var roles = RoleParser.ResolveEffectiveRoles(
|
|
_configuration.GetSection("Cluster:Roles").Get<string[]>());
|
|
var isDriver = Array.IndexOf(roles, RoleParser.Driver) >= 0;
|
|
|
|
if (isGrpc)
|
|
{
|
|
if (isDriver && options.GrpcListenPort <= 0)
|
|
{
|
|
errors.Add(
|
|
$"Cluster:Roles has 'driver' and {TelemetryOptions.SectionName}:"
|
|
+ $"{nameof(TelemetryOptions.Mode)} is '{TelemetryOptions.ModeGrpc}', but "
|
|
+ $"{TelemetryOptions.SectionName}:{nameof(TelemetryOptions.GrpcListenPort)} is "
|
|
+ $"{options.GrpcListenPort}. A driver node in Grpc telemetry mode must set "
|
|
+ $"{TelemetryOptions.SectionName}:{nameof(TelemetryOptions.GrpcListenPort)}.");
|
|
}
|
|
|
|
if (roles.Length > 0 && string.IsNullOrEmpty(options.ApiKey))
|
|
{
|
|
errors.Add(
|
|
$"{TelemetryOptions.SectionName}:{nameof(TelemetryOptions.ApiKey)} is empty. Under "
|
|
+ $"{TelemetryOptions.ModeGrpc} the shared bearer key is the whole authentication "
|
|
+ "boundary to this node's telemetry surface; without it every dial is rejected. "
|
|
+ $"Supply it via the environment ({TelemetryOptions.SectionName}__ApiKey).");
|
|
}
|
|
}
|
|
|
|
return errors.Count == 0
|
|
? ValidateOptionsResult.Success
|
|
: ValidateOptionsResult.Fail(string.Join(" ", errors));
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// Fails the host at startup on a <see cref="TelemetryDialOptions"/> shape that would leave
|
|
/// central unable to dial a node's live-telemetry stream.
|
|
/// </summary>
|
|
public sealed class TelemetryDialOptionsValidator : IValidateOptions<TelemetryDialOptions>
|
|
{
|
|
/// <inheritdoc />
|
|
public ValidateOptionsResult Validate(string? name, TelemetryDialOptions options)
|
|
{
|
|
ArgumentNullException.ThrowIfNull(options);
|
|
|
|
var errors = new List<string>();
|
|
|
|
var isDps = string.Equals(
|
|
options.Mode, TelemetryDialOptions.ModeDps, StringComparison.OrdinalIgnoreCase);
|
|
var isGrpc = string.Equals(
|
|
options.Mode, TelemetryDialOptions.ModeGrpc, StringComparison.OrdinalIgnoreCase);
|
|
|
|
if (!isDps && !isGrpc)
|
|
{
|
|
errors.Add(
|
|
$"{TelemetryDialOptions.SectionName}:{nameof(TelemetryDialOptions.Mode)} is "
|
|
+ $"'{options.Mode}'. Expected '{TelemetryDialOptions.ModeDps}' or "
|
|
+ $"'{TelemetryDialOptions.ModeGrpc}'.");
|
|
}
|
|
|
|
if (isGrpc && string.IsNullOrEmpty(options.ApiKey))
|
|
{
|
|
errors.Add(
|
|
$"{TelemetryDialOptions.SectionName}:{nameof(TelemetryDialOptions.ApiKey)} is empty. "
|
|
+ $"Under {TelemetryDialOptions.ModeGrpc} the shared bearer key is the whole "
|
|
+ "authentication boundary to a node's telemetry surface; without it every dial is "
|
|
+ $"rejected. Supply it via the environment ({TelemetryDialOptions.SectionName}__ApiKey).");
|
|
}
|
|
|
|
return errors.Count == 0
|
|
? ValidateOptionsResult.Success
|
|
: ValidateOptionsResult.Fail(string.Join(" ", errors));
|
|
}
|
|
}
|