fix(SEC-31,SEC-32): re-partition the API-key failure limiter on (peer, key id) with probe admission

The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).

ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.

SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.

Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.

Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).

src/ZB.MOM.WW.MxGateway.Server/Configuration/GatewayOptionsValidator.cs

@@ -87,6 +87,18 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
             options.ApiKeyFailureTrackedPeers,
             "MxGateway:Security:ApiKeyFailureTrackedPeers must be greater than zero.",
             builder);
+
+        // The two-layer limiter knobs (SEC-31) accept 0 as "disable this layer": a zero aggregate
+        // limit turns off cross-peer counting, and a zero probe interval restores absolute blocking.
+        // Negatives express no intent.
+        AddIfNegative(
+            options.ApiKeyFailureAggregateLimit,
+            "MxGateway:Security:ApiKeyFailureAggregateLimit must be greater than or equal to zero (0 disables the per-key aggregate layer).",
+            builder);
+        AddIfNegative(
+            options.ApiKeyFailureProbeIntervalSeconds,
+            "MxGateway:Security:ApiKeyFailureProbeIntervalSeconds must be greater than or equal to zero (0 blocks absolutely instead of admitting probes).",
+            builder);
     }
 
     private static void ValidateAuthentication(AuthenticationOptions options, ValidationBuilder builder)

src/ZB.MOM.WW.MxGateway.Server/Configuration/SecurityOptions.cs

@@ -43,23 +43,49 @@ public sealed class SecurityOptions
     public int LoginRateLimitWindowSeconds { get; init; } = 60;
 
     /// <summary>
-    /// Gets the number of consecutive failed API-key verifications, per peer, within
-    /// <see cref="ApiKeyFailureWindowSeconds"/> that trips the in-process short-circuit. Once tripped,
-    /// the gRPC auth path rejects further attempts before the store read; a successful verification
-    /// resets the peer's counter. Default is 10.
+    /// Gets the number of failed API-key verifications, per <c>(transport peer, key id)</c>
+    /// partition, within <see cref="ApiKeyFailureWindowSeconds"/> that trips the in-process
+    /// short-circuit. Once tripped, the gRPC auth path rejects further attempts from that partition
+    /// before the store read, except for the probe admitted every
+    /// <see cref="ApiKeyFailureProbeIntervalSeconds"/>; a successful verification resets the
+    /// partition. Default is 10.
     /// </summary>
     public int ApiKeyFailureLimit { get; init; } = 10;
 
     /// <summary>
     /// Gets the sliding-window length, in seconds, over which API-key verification failures are
-    /// counted per peer. Default is 60 seconds.
+    /// counted (for both the per-partition and the per-key-id aggregate layer). Default is 60
+    /// seconds.
     /// </summary>
     public int ApiKeyFailureWindowSeconds { get; init; } = 60;
 
     /// <summary>
-    /// Gets the maximum number of distinct peers tracked by the API-key failure counter. The counter
-    /// is a bounded LRU so a spray of unique peer keys cannot grow memory without limit. Default is
-    /// 4096.
+    /// Gets the number of failed API-key verifications for one key id, counted across <em>all</em>
+    /// transport peers within <see cref="ApiKeyFailureWindowSeconds"/>, that puts the key id into
+    /// probe mode. This second layer bounds a distributed or source-rotating sprayer that never
+    /// trips any single <c>(peer, key id)</c> partition. Set to <c>0</c> to disable the aggregate
+    /// layer. Default is 30.
+    /// </summary>
+    public int ApiKeyFailureAggregateLimit { get; init; } = 30;
+
+    /// <summary>
+    /// Gets the minimum interval, in seconds, between probe admissions for an over-limit partition
+    /// or key-id aggregate. An over-limit state is a valve rather than a wall: at most one request
+    /// per interval is admitted through to the real verifier, so the holder of the correct secret
+    /// can always reach the constant-time compare (and reset the state) while an attacker is
+    /// spraying. Set to <c>0</c> to block absolutely instead — not recommended, because an
+    /// unauthenticated peer can then deny the key to its legitimate holder for the whole window.
+    /// Default is 5 seconds.
+    /// </summary>
+    public int ApiKeyFailureProbeIntervalSeconds { get; init; } = 5;
+
+    /// <summary>
+    /// Gets the maximum number of distinct partitions tracked by the API-key failure counter. The
+    /// counter is a bounded LRU so a spray of unique tokens cannot grow memory without limit, and —
+    /// since only a validly shaped token mints a key-id partition, capped per transport peer —
+    /// cannot be flushed to clear an active block either: eviction prefers fully expired windows and
+    /// never removes a partition that is currently over its limit, up to a transient overshoot
+    /// ceiling of twice this value. Default is 4096.
     /// </summary>
     public int ApiKeyFailureTrackedPeers { get; init; } = 4096;
 }