dohertj2/mxaccessgw
1
0
Fork 0
Code Issues 3 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

fix(SEC-31,SEC-32): re-partition the API-key failure limiter on (peer, key id) with probe admission

Browse Source 
The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).

ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.

SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.

Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.

Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).
This commit is contained in:
Joseph Doherty
2026-08-07 05:39:10 -04:00
parent ead921cace
commit df710e18a9
16 changed files with 1148 additions and 137 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/GatewayGrpcAuthorizationInterceptor.cs
+66 -32
View File
@@ -3,6 +3,7 @@ using Grpc.Core.Interceptors;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.Auth.Abstractions.ApiKeys;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using ZB.MOM.WW.MxGateway.Server.Security.Authentication;
// The handler pushes the gateway's constraint-bearing identity; alias away the shared library's
@@ -16,8 +17,13 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
GatewayGrpcScopeResolver scopeResolver,
IGatewayRequestIdentityAccessor identityAccessor,
IOptions<GatewayOptions> options,
ApiKeyFailureLimiter failureLimiter) : Interceptor
ApiKeyFailureLimiter failureLimiter,
GatewayMetrics metrics) : Interceptor
{
// Generated key ids are far shorter; the cap only exists to stop an invented "key id" of
// arbitrary length from becoming a limiter partition.
private const int MaxKeyIdLength = 64;
/// <inheritdoc />
public override async Task<TResponse> UnaryServerHandler<TRequest, TResponse>(
TRequest request,
@@ -64,14 +70,19 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
string? authorizationHeader = context.RequestHeaders.GetValue("authorization");
// Short-circuit a peer that has already failed too many times inside the sliding
// window BEFORE the verification store read, so online guessing cannot spend a SQLite read
// per attempt. The peer key prefers the presented key id over the transport address (NAT
// caveat). ResourceExhausted signals throttling without revealing whether any particular
// secret was valid.
string peerKey = ResolvePeerKey(authorizationHeader, context);
if (failureLimiter.IsBlocked(peerKey))
// Short-circuit a partition that has already failed too many times inside the sliding window
// BEFORE the verification store read, so online guessing cannot spend a SQLite read per
// attempt. The partition is the composite (transport peer, key id) — never the key id alone,
// which is public and would let any peer deny a key it does not own — plus a cross-peer
// aggregate for the key id. An over-limit state still admits one probe per interval, so the
// holder of the correct secret always reaches the verifier and resets the state.
// ResourceExhausted signals throttling without revealing whether any secret was valid.
ApiKeyThrottlePartition throttlePartition = ResolveThrottlePartition(authorizationHeader, context);
ApiKeyThrottleDecision decision = failureLimiter.Check(throttlePartition);
if (decision is ApiKeyThrottleDecision.ThrottledByPeer or ApiKeyThrottleDecision.ThrottledByAggregate)
{
metrics.RecordAuthThrottled(
decision == ApiKeyThrottleDecision.ThrottledByPeer ? "peer" : "aggregate");
throw new RpcException(new Status(
StatusCode.ResourceExhausted,
"Too many authentication attempts. Try again later."));
@@ -86,15 +97,17 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
if (!verification.Succeeded || verification.Identity is null)
{
failureLimiter.RecordFailure(peerKey);
failureLimiter.RecordFailure(throttlePartition);
throw new RpcException(new Status(
StatusCode.Unauthenticated,
"Missing or invalid API key."));
}
// Successful authentication clears the peer's failure count so a legitimate client that
// fat-fingered a few attempts is not penalised once it recovers.
failureLimiter.Reset(peerKey);
// Successful authentication clears both limiter layers, so a legitimate client that
// fat-fingered a few attempts is not penalised once it recovers — and, because the check
// above admits a probe rather than blocking absolutely, this reset stays reachable while the
// key is under an active spray.
failureLimiter.Reset(throttlePartition);
ApiKeyIdentity identity = GatewayApiKeyIdentityMapper.ToGatewayIdentity(verification.Identity);
@@ -109,29 +122,50 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
return identity;
}
// Resolves the failure-limiter partition key: the presented key id (token shape
// mxgw_<keyId>_<secret>) when the header parses, otherwise the transport peer address. Keying on
// key id throttles a single abusive credential without locking out co-located clients behind NAT.
private static string ResolvePeerKey(string? authorizationHeader, ServerCallContext context)
// Resolves the failure-limiter partition: always the transport peer, plus the presented key id
// when — and only when — the token is validly shaped. Both halves are unauthenticated input, so
// the peer half is what keeps a throttle bound to the address that earned it.
private static ApiKeyThrottlePartition ResolveThrottlePartition(
string? authorizationHeader,
ServerCallContext context)
{
if (!string.IsNullOrWhiteSpace(authorizationHeader))
{
ReadOnlySpan<char> header = authorizationHeader.AsSpan().Trim();
const string bearer = "Bearer ";
ReadOnlySpan<char> token = header.StartsWith(bearer, StringComparison.OrdinalIgnoreCase)
? header[bearer.Length..].Trim()
: header;
return new ApiKeyThrottlePartition(context.Peer, TryResolveKeyId(authorizationHeader));
}
// mxgw_<keyId>_<secret>: the key id is the second underscore-delimited segment. The
// secret may itself contain underscores, but the key id is unaffected.
string tokenText = token.ToString();
string[] parts = tokenText.Split('_');
if (parts.Length >= 3 && parts[0].Length > 0 && parts[1].Length > 0)
{
return "key:" + parts[1];
}
// Token shape mxgw_<keyId>_<secret>: the key id is the second underscore-delimited segment (the
// secret may itself contain underscores, which does not affect the key id). The shape is checked
// before a key-id partition is minted so a spray of invented tokens cannot mint one tracked
// partition each and flush the limiter's bounded map (SEC-32). Anything that fails the check
// falls back to the sender's transport-peer partition.
private static string? TryResolveKeyId(string? authorizationHeader)
{
if (string.IsNullOrWhiteSpace(authorizationHeader))
{
return null;
}
return "peer:" + context.Peer;
ReadOnlySpan<char> header = authorizationHeader.AsSpan().Trim();
const string bearer = "Bearer ";
ReadOnlySpan<char> token = header.StartsWith(bearer, StringComparison.OrdinalIgnoreCase)
? header[bearer.Length..].Trim()
: header;
string[] parts = token.ToString().Split('_');
if (parts.Length < 3)
{
return null;
}
if (!string.Equals(parts[0], AuthStoreServiceCollectionExtensions.TokenPrefix, StringComparison.Ordinal))
{
return null;
}
if (parts[1].Length == 0 || parts[1].Length > MaxKeyIdLength || parts[2].Length == 0)
{
return null;
}
return parts[1];
}
}