48dc2120c6
The known-cluster gate published only under Mode=Grpc, but the interceptor it feeds is registered on every admin node and 1B maps CentralCommandService by LISTENER PORT — neither consults the mode. A dark-switch rollback to ClusterClient or Dps would therefore have left that endpoint serving with the gate permanently inactive, silently restoring both the timing oracle and the pre-auth secret-store/audit flood, as a side effect of a rollback nobody would connect to authentication. The gate is central's cluster inventory, not transport state, so it is now decoupled from the mode entirely: - SetKnownClusters + InvalidateAll run on EVERY successful contact load in EVERY mode, and the contact-refresh timer starts in all three. Under Dps this is a ConfigDb query every ContactRefreshSeconds that did not run before; central always has a ConfigDb, and one indexed read a minute is the cheaper half of the trade. - The GATE set is built from ALL ClusterNode rows. The eligibility filter moved out of the query's WHERE and into the loop, so the FAN set is byte-for-byte what it was (enabled && !maintenance) while the gate widens. This also dissolves the maintenance coupling flagged in 1A.2: a cluster whose nodes are all disabled or in maintenance can still ack, and refusing it would make maintenance mode an authentication switch. Central still refuses clusters it has never heard of. - HandleContactsLoaded keeps a documented order — targets adopted, gate published, cache dropped — now pinned by test. Guards: PreStart mirrors the null-dialer error for a null PSK provider (silent absence is the worst outcome there — nothing malfunctions, so nothing is investigated, while the gate never publishes and keys never rotate); a failed contact load now says out loud that the gate is stale and, before any load has succeeded, inactive; MeshGrpcCommandDialer clamps a non-positive DispatchTimeoutSeconds to 10 with a construction Warning, because the validator for that key is driver-role-scoped and a pure-admin central reads it unvalidated; the dialer's local PSK refusal was shortened so its remediation survives the actor's 200-char truncation of remote-authored status detail; the portless-node warn-once set is rebuilt each refresh, so a node that regains then loses a port warns again — the regression is the occurrence worth seeing. Docs/pins: dialer disposal comment no longer claims one-way; the two interceptor-test comments claiming Matches has a default implementation corrected (it deliberately has none); new pins for per-refresh invalidation, the SetKnownClusters-before-InvalidateAll order, and one channel per endpoint via a new CachedChannelCount. The_known_cluster_set_is_not_published_by_the_other_two_modes is replaced — its premise is obsolete — by three facts: every mode publishes, gate set is a strict superset of the fan set, and the handler ordering. ControlPlane.Tests 230 -> 236, Host.Tests 92, Cluster.Tests 184, 0 warnings.
332 lines
16 KiB
C#
332 lines
16 KiB
C#
using System.Diagnostics.CodeAnalysis;
|
|
using Grpc.Core;
|
|
using Microsoft.Extensions.Logging.Abstractions;
|
|
using Shouldly;
|
|
using Xunit;
|
|
using ZB.MOM.WW.OtOpcUa.ControlPlane.Communication;
|
|
using ZB.MOM.WW.OtOpcUa.Host.Configuration;
|
|
|
|
namespace ZB.MOM.WW.OtOpcUa.Host.Tests.Configuration;
|
|
|
|
/// <summary>
|
|
/// ClusterClient→gRPC Phase 0 — the central-side ack endpoint's inbound gate. Mapping
|
|
/// <c>CentralCommandService</c> onto the pipeline is a Phase 1 task; this test covers only the
|
|
/// interceptor's own authorization behavior.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// Differs from <see cref="MeshCommandAuthInterceptor"/> in the one way central differs from a
|
|
/// node: there is no single expected key. The caller names its cluster in
|
|
/// <c>x-otopcua-cluster</c>, the bearer is verified against <i>that</i> cluster's PSK, and a
|
|
/// cluster with no resolvable key is denied — so a caller cannot pick a cluster to be graded
|
|
/// against, only claim one it must then prove.
|
|
/// </remarks>
|
|
public sealed class CentralCommandAuthInterceptorTests
|
|
{
|
|
private const string ReportApplyMethod = "/mesh_transport.v1.CentralCommandService/ReportApply";
|
|
private const string OtherMethod = "/deployment_artifact.v1.DeploymentArtifactService/Fetch";
|
|
|
|
private static CentralCommandAuthInterceptor CreateInterceptor(params (string Cluster, string Psk)[] keys)
|
|
=> new(new FakeMeshPskProvider(keys), NullLogger<CentralCommandAuthInterceptor>.Instance);
|
|
|
|
private static ServerCallContext CreateContext(
|
|
string method, string? authorizationHeader, string? clusterHeader)
|
|
=> CreateContext(
|
|
method,
|
|
authorizationHeader is null ? [] : [authorizationHeader],
|
|
clusterHeader);
|
|
|
|
private static ServerCallContext CreateContext(
|
|
string method, string[] authorizationHeaders, string? clusterHeader)
|
|
{
|
|
var headers = new Metadata();
|
|
foreach (var value in authorizationHeaders)
|
|
headers.Add("authorization", value);
|
|
if (clusterHeader is not null)
|
|
headers.Add("x-otopcua-cluster", clusterHeader);
|
|
|
|
return new FakeServerCallContext(method, headers);
|
|
}
|
|
|
|
/// <summary>Invokes the interceptor's unary path (how <c>ReportApply</c> actually runs).</summary>
|
|
private static Task Invoke(CentralCommandAuthInterceptor interceptor, ServerCallContext context)
|
|
=> interceptor.UnaryServerHandler<string, string>(
|
|
"request", context, (_, _) => Task.FromResult("ok"));
|
|
|
|
[Fact]
|
|
public async Task NonCentralCommandMethod_PassesThrough_EvenWithNoKeysAtAll()
|
|
{
|
|
// The interceptor shares the AddGrpc pipeline with config-serve (and, on a fused node, sync
|
|
// and telemetry), so it sees every call. It must be scoped strictly to its own service.
|
|
var interceptor = CreateInterceptor();
|
|
var context = CreateContext(OtherMethod, authorizationHeader: null, clusterHeader: null);
|
|
|
|
await Invoke(interceptor, context); // no throw
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_WithCorrectBearerForItsCluster_PassesThrough()
|
|
{
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer psk-site-a", "SITE-A");
|
|
|
|
await Invoke(interceptor, context); // no throw
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_WithNoClusterHeader_IsDenied()
|
|
{
|
|
// Without the cluster there is no key to grade against. Falling back to "try them all" would
|
|
// make one compromised site's key valid for forging every other site's acks.
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer psk-site-a", clusterHeader: null);
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_WithBlankClusterHeader_IsDenied()
|
|
{
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer psk-site-a", " ");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_ForAClusterWithNoResolvableKey_IsDenied()
|
|
{
|
|
// Fail-closed on the provider's "no key" — an unseeded or unreachable key must cost the ack,
|
|
// never authentication.
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer anything-at-all", "SITE-Z");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_WithAnotherClustersBearer_IsDenied()
|
|
{
|
|
// The isolation proof: SITE-B's key does not authenticate a call claiming to be SITE-A.
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"), ("SITE-B", "psk-site-b"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer psk-site-b", "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_WithNoBearerToken_IsDenied()
|
|
{
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, authorizationHeader: null, clusterHeader: "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_WithMalformedAuthorizationHeader_IsDenied()
|
|
{
|
|
// No "Bearer " prefix at all — must be treated the same as a missing header.
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "psk-site-a", "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_TokenComparison_IsNotAPrefixMatch()
|
|
{
|
|
// A prefix/StartsWith comparison would accept a truncated key and make the secret
|
|
// recoverable one character at a time.
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer psk-site-", "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ReportApply_WithAJunkAuthorizationHeaderBeforeTheRealOne_IsDenied()
|
|
{
|
|
// First entry wins, and it must parse. Scanning past a non-Bearer header for a later one
|
|
// would let a caller smuggle its credential behind a header an intermediary may have
|
|
// inspected or rewritten, and would make WHICH credential is checked depend on ordering
|
|
// nothing guarantees. Pinned identically on MeshCommandAuthInterceptor.
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(
|
|
ReportApplyMethod, ["junk", "Bearer psk-site-a"], "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() => Invoke(interceptor, context));
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task EveryDenialArm_ReturnsAnIdenticalDetail()
|
|
{
|
|
// The anti-enumeration mechanism IS this identity: the caller supplies the cluster id, so a
|
|
// reply distinguishing "no key for that cluster" from "wrong key" would let an
|
|
// unauthenticated caller read central's cluster set off the wire. Asserting only StatusCode
|
|
// would let a well-meaning "more helpful message" edit break it silently.
|
|
// (Content-level only — the class remark records the timing oracle that survives until the
|
|
// Phase 1A known-cluster-set gate.)
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
|
|
var noHeader = await Should.ThrowAsync<RpcException>(() => Invoke(
|
|
interceptor, CreateContext(ReportApplyMethod, "Bearer psk-site-a", clusterHeader: null)));
|
|
var unknownCluster = await Should.ThrowAsync<RpcException>(() => Invoke(
|
|
interceptor, CreateContext(ReportApplyMethod, "Bearer psk-site-a", "SITE-Z")));
|
|
var wrongToken = await Should.ThrowAsync<RpcException>(() => Invoke(
|
|
interceptor, CreateContext(ReportApplyMethod, "Bearer wrong", "SITE-A")));
|
|
var noToken = await Should.ThrowAsync<RpcException>(() => Invoke(
|
|
interceptor, CreateContext(ReportApplyMethod, authorizationHeader: null, clusterHeader: "SITE-A")));
|
|
|
|
unknownCluster.Status.Detail.ShouldBe(noHeader.Status.Detail);
|
|
wrongToken.Status.Detail.ShouldBe(noHeader.Status.Detail);
|
|
noToken.Status.Detail.ShouldBe(noHeader.Status.Detail);
|
|
noHeader.Status.Detail.ShouldNotBeNullOrWhiteSpace();
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ServerStreamingPath_IsAlsoGated()
|
|
{
|
|
// ReportApply is unary today, but gating only that path would leave any future streaming
|
|
// method on the same service open. Gate every handler shape.
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer wrong", "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() =>
|
|
interceptor.ServerStreamingServerHandler<string, string>(
|
|
"request", responseStream: null!, context, (_, _, _) => Task.CompletedTask));
|
|
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task ClientStreamingPath_IsAlsoGated()
|
|
{
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer wrong", "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() =>
|
|
interceptor.ClientStreamingServerHandler<string, string>(
|
|
requestStream: null!, context, (_, _) => Task.FromResult("ok")));
|
|
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task DuplexStreamingPath_IsAlsoGated()
|
|
{
|
|
var interceptor = CreateInterceptor(("SITE-A", "psk-site-a"));
|
|
var context = CreateContext(ReportApplyMethod, "Bearer wrong", "SITE-A");
|
|
|
|
var ex = await Should.ThrowAsync<RpcException>(() =>
|
|
interceptor.DuplexStreamingServerHandler<string, string>(
|
|
requestStream: null!, responseStream: null!, context, (_, _, _) => Task.CompletedTask));
|
|
|
|
ex.StatusCode.ShouldBe(StatusCode.PermissionDenied);
|
|
}
|
|
|
|
[Fact]
|
|
public void HasExactlyOnePublicConstructor()
|
|
{
|
|
// Grpc.AspNetCore silently stops invoking an interceptor with more than one public ctor —
|
|
// this pins the shape so a well-meaning overload doesn't quietly disable auth in production.
|
|
var publicCtors = typeof(CentralCommandAuthInterceptor).GetConstructors();
|
|
|
|
publicCtors.Length.ShouldBe(1);
|
|
}
|
|
|
|
[Fact]
|
|
public async Task Verification_GoesThroughMatches_AndTheInterceptorNeverHandlesTheKeyItself()
|
|
{
|
|
// Phase 1A security register item 2. The interceptor must not fetch the raw key and compare
|
|
// it — that is how a constant-time comparison eventually becomes '=='. Matches owns the
|
|
// comparison; TryGetPsk survives only for the DIAL side, which has to send the key.
|
|
// (The production provider's Matches calls TryGetPsk internally; this fake implements the two
|
|
// independently, so a zero TryGetPsk count here means the INTERCEPTOR never asked for the key.)
|
|
var provider = new FakeMeshPskProvider(("SITE-A", "psk-site-a"));
|
|
var interceptor = new CentralCommandAuthInterceptor(
|
|
provider, NullLogger<CentralCommandAuthInterceptor>.Instance);
|
|
|
|
await Invoke(interceptor, CreateContext(ReportApplyMethod, "Bearer psk-site-a", "SITE-A"));
|
|
|
|
provider.MatchesCalls.ShouldBe(1);
|
|
provider.TryGetPskCalls.ShouldBe(0);
|
|
}
|
|
|
|
/// <summary>An <see cref="IMeshPskProvider"/> over a fixed key set; unknown clusters fail closed.</summary>
|
|
private sealed class FakeMeshPskProvider(params (string Cluster, string Psk)[] keys) : IMeshPskProvider
|
|
{
|
|
private readonly Dictionary<string, string> _keys =
|
|
keys.ToDictionary(k => k.Cluster, k => k.Psk, StringComparer.OrdinalIgnoreCase);
|
|
|
|
/// <summary>How many times the raw key was asked for — must stay 0 on the verify path.</summary>
|
|
public int TryGetPskCalls { get; private set; }
|
|
|
|
/// <summary>How many times verification went through the provider's comparison.</summary>
|
|
public int MatchesCalls { get; private set; }
|
|
|
|
public bool TryGetPsk(string clusterId, [NotNullWhen(true)] out string? psk)
|
|
{
|
|
TryGetPskCalls++;
|
|
return _keys.TryGetValue(clusterId, out psk);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Implemented independently of <see cref="TryGetPsk"/> so the two calls are separately
|
|
/// countable — the interface declares this with no default implementation, deliberately.
|
|
/// Semantics match the production provider's: no key, or a key that is not exactly the
|
|
/// presented token.
|
|
/// </summary>
|
|
public bool Matches(string clusterId, string presented)
|
|
{
|
|
MatchesCalls++;
|
|
return !string.IsNullOrWhiteSpace(presented)
|
|
&& _keys.TryGetValue(clusterId, out var expected)
|
|
&& string.Equals(expected, presented, StringComparison.Ordinal);
|
|
}
|
|
|
|
/// <summary>No-op: the gate is the production provider's behaviour, pinned in its own tests.</summary>
|
|
public void SetKnownClusters(IReadOnlyCollection<string> clusterIds) { }
|
|
|
|
public void Invalidate(string clusterId) { }
|
|
|
|
public void InvalidateAll() { }
|
|
}
|
|
|
|
/// <summary>
|
|
/// Minimal <see cref="ServerCallContext"/> carrying just a method name and request headers —
|
|
/// the only two things the interceptor reads. Hand-rolled because
|
|
/// <c>Grpc.Core.Testing.TestServerCallContext</c> ships only in the retired native package.
|
|
/// </summary>
|
|
private sealed class FakeServerCallContext(string method, Metadata requestHeaders)
|
|
: ServerCallContext
|
|
{
|
|
protected override string MethodCore => method;
|
|
protected override string HostCore => "localhost";
|
|
protected override string PeerCore => "ipv4:127.0.0.1:12345";
|
|
protected override DateTime DeadlineCore => DateTime.UtcNow.AddMinutes(1);
|
|
protected override Metadata RequestHeadersCore => requestHeaders;
|
|
protected override CancellationToken CancellationTokenCore => CancellationToken.None;
|
|
protected override Metadata ResponseTrailersCore { get; } = [];
|
|
protected override Status StatusCore { get; set; }
|
|
protected override WriteOptions? WriteOptionsCore { get; set; }
|
|
|
|
protected override AuthContext AuthContextCore { get; } =
|
|
new(null, new Dictionary<string, List<AuthProperty>>());
|
|
|
|
protected override ContextPropagationToken CreatePropagationTokenCore(
|
|
ContextPropagationOptions? options)
|
|
=> throw new NotSupportedException();
|
|
|
|
protected override Task WriteResponseHeadersAsyncCore(Metadata responseHeaders)
|
|
=> Task.CompletedTask;
|
|
}
|
|
}
|