Files
mxaccessgw/docs/Authorization.md
T
Joseph Doherty df710e18a9 fix(SEC-31,SEC-32): re-partition the API-key failure limiter on (peer, key id) with probe admission
The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).

ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.

SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.

Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.

Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).
2026-08-07 05:39:10 -04:00

294 lines
19 KiB
Markdown

# Gateway gRPC Authorization
The authorization subsystem has two layers. The gRPC interceptor enforces the
verb scope required by the RPC. Service-layer constraint checks then narrow
what an authenticated API key can browse, read, or write inside the Galaxy.
## Overview
Authorization runs as a single gRPC server interceptor registered for every call on the gateway. It pulls the authenticated identity for the current request, derives the scope that the request type requires, and either lets the call continue or fails the call with a gRPC status. The pipeline keeps service classes free of cross-cutting checks, which matches the `gateway.md` "thin gRPC layer" rule that service handlers translate between contracts and domain code without owning policy.
The participating types live under `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/`:
- `GatewayGrpcAuthorizationInterceptor` runs the authenticate-then-authorize pipeline for unary and server-streaming calls.
- `GatewayGrpcScopeResolver` maps a request message (and, for `MxCommandRequest`, the inner `MxCommandKind`) to the scope string that must be present on the caller.
- `GatewayScopes` exposes the canonical scope constants used by the resolver and any downstream consumer.
- `GatewayRequestIdentityAccessor` and `IGatewayRequestIdentityAccessor` expose the verified identity to handlers and any service code that runs inside the call.
- `IConstraintEnforcer` applies optional API-key constraints against the
cached Galaxy hierarchy from service bodies.
- `GrpcAuthorizationServiceCollectionExtensions` wires the components into the DI container and the gRPC pipeline.
The `ApiKeyIdentity` consumed here is produced by the authentication layer; see [Authentication](./Authentication.md) for how it is built and how scopes are persisted.
## Why an Interceptor
Centralizing the policy in `GatewayGrpcAuthorizationInterceptor` produces three concrete benefits:
1. Every RPC defined in `MxAccessGatewayService` is covered by construction. A new RPC inherits the check the moment its request type is added to `GatewayGrpcScopeResolver`, instead of relying on each service method to remember to call an authorization helper.
2. Verb-scope policy stays centralized. Request-specific constraints still run
in service bodies because they need command payloads, item handles, and
Galaxy metadata that the interceptor should not inspect.
3. Authentication and authorization happen in one place, so the gRPC `Status` mapping is consistent. A failed key check always returns `Unauthenticated`, and a missing scope always returns `PermissionDenied` with the offending scope name.
## Interceptor Flow
`GatewayGrpcAuthorizationInterceptor` overrides both `UnaryServerHandler` and `ServerStreamingServerHandler`. Both call the same private `AuthenticateAndAuthorizeAsync` helper before invoking the continuation, then push the resolved identity onto the accessor for the duration of the call.
```csharp
public override async Task<TResponse> UnaryServerHandler<TRequest, TResponse>(
TRequest request,
ServerCallContext context,
UnaryServerMethod<TRequest, TResponse> continuation)
{
ApiKeyIdentity? identity = await AuthenticateAndAuthorizeAsync(request, context).ConfigureAwait(false);
IDisposable? identityScope = identity is null ? null : identityAccessor.Push(identity);
using (identityScope)
{
return await continuation(request, context).ConfigureAwait(false);
}
}
```
The shared helper performs the actual decision:
```csharp
if (options.Value.Authentication.Mode == AuthenticationMode.Disabled)
{
return null;
}
string? authorizationHeader = context.RequestHeaders.GetValue("authorization");
ApiKeyVerificationResult verificationResult = await apiKeyVerifier
.VerifyAsync(authorizationHeader, context.CancellationToken)
.ConfigureAwait(false);
if (!verificationResult.Succeeded || verificationResult.Identity is null)
{
throw new RpcException(new Status(
StatusCode.Unauthenticated,
"Missing or invalid API key."));
}
string requiredScope = scopeResolver.ResolveRequiredScope(request);
if (!verificationResult.Identity.Scopes.Contains(requiredScope))
{
throw new RpcException(new Status(
StatusCode.PermissionDenied,
$"API key is missing required scope '{requiredScope}'."));
}
return verificationResult.Identity;
```
The flow is:
1. If `GatewayOptions.Authentication.Mode` is `AuthenticationMode.Disabled`, the helper returns `null` immediately. No identity is pushed onto the accessor and the continuation runs without scope enforcement. This matches the `AuthenticationMode` enum, which only defines `ApiKey` and `Disabled`.
2. Otherwise, the `authorization` request header is read directly off `ServerCallContext.RequestHeaders` and handed to `IApiKeyVerifier.VerifyAsync`. A failed verification or a missing identity throws `RpcException` with `StatusCode.Unauthenticated`.
3. `GatewayGrpcScopeResolver.ResolveRequiredScope(request)` produces the scope string. If the identity's `Scopes` set does not contain it, the helper throws `RpcException` with `StatusCode.PermissionDenied` and embeds the missing scope name in `Status.Detail` so callers can diagnose the failure.
4. On success, the verified `ApiKeyIdentity` is returned and pushed onto `IGatewayRequestIdentityAccessor` for the lifetime of the call.
The status codes are deliberately distinct: `Unauthenticated` signals "we do not know who you are," and `PermissionDenied` signals "we know who you are, but you cannot do this." Treating the two as the same code would make troubleshooting harder for client implementations.
### Rate limiting the auth surface (SEC-11, SEC-31, SEC-32)
Before the verification store read, the helper asks a cheap in-process failure counter (`ApiKeyFailureLimiter`) whether the attempt may proceed, so online guessing of API-key secrets cannot spend a SQLite read (and, in a naive design, a cache miss) per attempt. The counter has two layers over one sliding `ApiKeyFailureWindowSeconds` window:
- **Composite `(transport peer, key id)` partitions.** Reaching `MxGateway:Security:ApiKeyFailureLimit` failures binds the throttle to the address that produced them. The key id alone is never the partition: key ids are not secret — they ride in every token and are listed on the dashboard — so keying on them let any network peer deny a key to its legitimate holder. The key id joins the partition only after a token-shape check (literal `mxgw` prefix, at least three non-empty `_` segments, key id of at most 64 characters), and one address may mint at most 32 key-id partitions before the overflow collapses onto that address's fallback partition.
- **A per-key-id aggregate** across all peers (`ApiKeyFailureAggregateLimit`, default 30), which bounds a distributed or source-rotating sprayer that never trips any single partition.
An over-limit state is a valve, not a wall: one request per `ApiKeyFailureProbeIntervalSeconds` (default 5 s) is admitted through to the real verifier, and everything else is refused with `StatusCode.ResourceExhausted` before the store read. A successful verification resets both layers — which is why the reset path stays reachable while a key is under active spray. The tracked partitions form a bounded LRU (`ApiKeyFailureTrackedPeers`) whose eviction prefers fully expired windows and never removes an over-limit partition below a 2x transient overshoot ceiling, so the cap bounds memory without becoming a reset button for an active block. `ResourceExhausted` reveals only that throttling is in effect, not whether any particular secret was valid, preserving the opaque-failure property. Refusals increment `mxgateway.auth.throttled`, tagged `stage=peer|aggregate` and nothing else — `/metrics` is unauthenticated, so neither key ids nor peer addresses may appear there.
The dashboard login surface is throttled independently: `POST /auth/login` carries a fixed-window ASP.NET Core rate-limiter policy keyed per remote IP (`MxGateway:Security:LoginRateLimit*`), rejecting a burst with HTTP 429 before the LDAP bind is relayed to the directory. See [GatewayConfiguration](./GatewayConfiguration.md#security-options).
## Scope Resolution
`GatewayGrpcScopeResolver` is a stateless singleton that switches on the runtime request type. Top-level RPC requests map directly:
```csharp
public string ResolveRequiredScope(object request)
{
return request switch
{
OpenSessionRequest => GatewayScopes.SessionOpen,
CloseSessionRequest => GatewayScopes.SessionClose,
StreamEventsRequest => GatewayScopes.EventsRead,
MxCommandRequest commandRequest => ResolveCommandScope(commandRequest.Command?.Kind ?? MxCommandKind.Unspecified),
AcknowledgeAlarmRequest => GatewayScopes.InvokeWrite,
StreamAlarmsRequest => GatewayScopes.EventsRead,
QueryActiveAlarmsRequest => GatewayScopes.EventsRead,
TestConnectionRequest or
GetLastDeployTimeRequest or
DiscoverHierarchyRequest or
WatchDeployEventsRequest => GatewayScopes.MetadataRead,
_ => GatewayScopes.Admin
};
}
```
The `_ => GatewayScopes.Admin` fallback is intentional: any future request type that the resolver does not recognize fails closed, requiring the strongest scope until the resolver is updated. `AcknowledgeAlarm` is treated as a write — it mutates alarm state, mirroring `MxCommandKind.Write*` — and `StreamAlarms` and `QueryActiveAlarms` share the alarm/event surface with `StreamEvents` and `MxCommandKind.DrainEvents`, so they carry `events:read` (the active-alarm snapshot is the same data reachable through the event surface). All three alarm RPCs are session-less: the scope check is the only authorization gate, since there is no per-session ownership to enforce.
`MxCommandRequest` is special because it multiplexes many MxAccess operations through a single RPC. The resolver inspects the embedded `MxCommandKind` so each operation gets its own scope:
```csharp
private static string ResolveCommandScope(MxCommandKind kind)
{
return kind switch
{
MxCommandKind.Write or
MxCommandKind.Write2 or
MxCommandKind.WriteBulk or
MxCommandKind.Write2Bulk => GatewayScopes.InvokeWrite,
MxCommandKind.WriteSecured or
MxCommandKind.WriteSecured2 or
MxCommandKind.WriteSecuredBulk or
MxCommandKind.WriteSecured2Bulk or
MxCommandKind.AuthenticateUser => GatewayScopes.InvokeSecure,
MxCommandKind.ArchestraUserToId or
MxCommandKind.GetSessionState or
MxCommandKind.GetWorkerInfo => GatewayScopes.MetadataRead,
MxCommandKind.DrainEvents => GatewayScopes.EventsRead,
MxCommandKind.ShutdownWorker => GatewayScopes.Admin,
_ => GatewayScopes.InvokeRead
};
}
```
Reads (`Register`, `AddItem`, `Advise`, `ReadBulk`, and any other unspecified kind) fall through to `InvokeRead`, which keeps the matrix small while still separating reads from writes, secured writes, metadata lookups, event drains, and worker shutdown. The four bulk-write families (`WriteBulk`, `Write2Bulk`, `WriteSecuredBulk`, `WriteSecured2Bulk`) are mapped explicitly so a missing arm cannot silently demote a bulk write to a read scope.
## Constraint Enforcement
`ApiKeyIdentity.Constraints` is optional. Empty constraints preserve the
previous behavior: the key is authorized only by its verb scopes. Non-empty
constraints are stored as JSON in `api_keys.constraints` and are applied by
`IConstraintEnforcer` after the interceptor succeeds.
Supported constraints are:
| Constraint | Meaning |
|------------|---------|
| `read_subtrees` | Contained-path globs allowed for read/subscription commands. |
| `write_subtrees` | Contained-path globs allowed for write commands. |
| `read_tag_globs` | Tag-address globs allowed for read/subscription commands. |
| `write_tag_globs` | Tag-address globs allowed for write commands. |
| `max_write_classification` | Maximum Galaxy attribute `security_classification` a key may write. |
| `browse_subtrees` | Contained-path globs used to filter Galaxy browse results and deploy-event counts. |
| `read_alarm_only` | Read/subscription commands must target objects with alarm-bearing attributes. |
| `read_historized_only` | Read/subscription commands must target objects with historized attributes. |
Glob matching is anchored, case-insensitive, and supports `*` and `?`.
Subtree and tag glob lists are alternatives: matching either list allows that
scope dimension. Empty lists mean unconstrained for that dimension.
Constraints are set when a key is created — through the `apikey create-key`
flags (see [Authentication](./Authentication.md)) or the dashboard API Keys
page create dialog (see
[Gateway Dashboard Design](./GatewayDashboardDesign.md#api-keys-page)). The
dashboard API Keys page also renders each key's effective constraints.
The service checks read constraints for `AddItem`, `AddItem2`, `AddItemBulk`,
`SubscribeBulk`, `AdviseItemBulk`, and `ReadBulk`. It checks write constraints
for `Write`, `Write2`, `WriteSecured`, `WriteSecured2`, `WriteBulk`,
`Write2Bulk`, `WriteSecuredBulk`, and `WriteSecured2Bulk`. Bulk commands run
through `BulkConstraintPlan` (`ReadBulkConstraintPlan`,
`WriteBulkConstraintPlan`, `SubscribeBulkConstraintPlan`), which preserves the
caller's input order: each entry is evaluated against the constraint surface,
and `BulkConstraintPlan.MergeDeniedInto` re-merges denied entries back into
their original index positions so the reply slot at `entries[i]` always
corresponds to the request slot at `entries[i]`. Successful item registrations
are tracked per session so later item-handle commands resolve back to the
original tag address. If a constrained key presents an unknown item handle,
the gateway fails closed.
Non-bulk constraint failures return gRPC `PermissionDenied`. Bulk read
commands preserve input order and return a failed `SubscribeResult` for each
denied item while still forwarding allowed items to the worker. Every denial
adds an `api_key_audit` entry with the key id, command kind, target, and
blocking constraint; secured values and raw credentials are never logged.
## Scope Catalog
`GatewayScopes` is the single source of truth for scope strings. Every entry is currently mapped by either the resolver or another security component:
| Constant | Value | Required For |
|----------|-------|--------------|
| `SessionOpen` | `session:open` | `OpenSessionRequest` |
| `SessionClose` | `session:close` | `CloseSessionRequest` |
| `EventsRead` | `events:read` | `StreamEventsRequest`, `StreamAlarmsRequest`, `QueryActiveAlarmsRequest`, `MxCommandKind.DrainEvents` |
| `InvokeRead` | `invoke:read` | `MxCommandRequest` for read-style command kinds (`Register`, `AddItem`, `Advise`, `ReadBulk`, and any kind not otherwise mapped) |
| `InvokeWrite` | `invoke:write` | `AcknowledgeAlarmRequest`, `MxCommandKind.Write`, `MxCommandKind.Write2`, `MxCommandKind.WriteBulk`, `MxCommandKind.Write2Bulk` |
| `InvokeSecure` | `invoke:secure` | `MxCommandKind.WriteSecured`, `MxCommandKind.WriteSecured2`, `MxCommandKind.WriteSecuredBulk`, `MxCommandKind.WriteSecured2Bulk`, `MxCommandKind.AuthenticateUser` |
| `MetadataRead` | `metadata:read` | `MxCommandKind.ArchestraUserToId`, `MxCommandKind.GetSessionState`, `MxCommandKind.GetWorkerInfo`, `GalaxyRepository.TestConnection`, `GalaxyRepository.GetLastDeployTime`, `GalaxyRepository.DiscoverHierarchy`, `GalaxyRepository.WatchDeployEvents` |
| `Admin` | `admin` | `MxCommandKind.ShutdownWorker`, the default for any unrecognized request type, and the dashboard authorization policy |
The `Admin` constant is also referenced by `DashboardAuthenticator` and `DashboardAuthorizationHandler` so that the dashboard and the gRPC layer agree on what "admin" means.
## Identity Access for Downstream Layers
Once authorization passes, `GatewayGrpcAuthorizationInterceptor` calls `identityAccessor.Push(identity)` and disposes the returned scope when the continuation completes. `GatewayRequestIdentityAccessor` stores the active identity in an `AsyncLocal<ApiKeyIdentity?>`, so the value flows across `await` boundaries and child tasks belonging to the same request.
```csharp
public sealed class GatewayRequestIdentityAccessor : IGatewayRequestIdentityAccessor
{
private readonly AsyncLocal<ApiKeyIdentity?> currentIdentity = new();
public ApiKeyIdentity? Current => currentIdentity.Value;
public IDisposable Push(ApiKeyIdentity identity)
{
ArgumentNullException.ThrowIfNull(identity);
ApiKeyIdentity? previousIdentity = currentIdentity.Value;
currentIdentity.Value = identity;
return new IdentityScope(this, previousIdentity);
}
}
```
The returned `IdentityScope` restores the previous value on dispose rather than clearing it. This makes the accessor safe for nested pushes, even though the current interceptor only pushes once per call. Disposing twice is a no-op because of the `disposed` guard inside `IdentityScope`.
Downstream code consumes the accessor through the `IGatewayRequestIdentityAccessor` interface:
```csharp
public interface IGatewayRequestIdentityAccessor
{
ApiKeyIdentity? Current { get; }
IDisposable Push(ApiKeyIdentity identity);
}
```
`MxAccessGatewayService` takes `IGatewayRequestIdentityAccessor` as a constructor dependency and reads `Current` whenever it needs to attach the calling identity to a domain operation, which keeps the service free of header parsing or scope checks.
When `AuthenticationMode.Disabled` is configured, no identity is pushed, so `Current` returns `null`. Downstream code must tolerate that, just as it tolerates the absence of a scope check.
## Registration
`GrpcAuthorizationServiceCollectionExtensions.AddGatewayGrpcAuthorization` is the single entry point that registers every component and inserts the interceptor into the gRPC pipeline:
```csharp
public static IServiceCollection AddGatewayGrpcAuthorization(this IServiceCollection services)
{
services.AddSingleton<GatewayGrpcScopeResolver>();
services.AddSingleton<IGatewayRequestIdentityAccessor, GatewayRequestIdentityAccessor>();
services.AddSingleton<GatewayGrpcAuthorizationInterceptor>();
services.AddGrpc(options => options.Interceptors.Add<GatewayGrpcAuthorizationInterceptor>());
return services;
}
```
Singleton lifetimes are appropriate because none of the three classes hold per-request state on instance fields; the request-scoped value lives inside the `AsyncLocal` on `GatewayRequestIdentityAccessor`. `GatewayApplication` calls `builder.Services.AddGatewayGrpcAuthorization()` during startup, and the call also performs `AddGrpc`, so the gateway never registers gRPC without the interceptor attached.
## Related Documentation
- [Authentication](./Authentication.md)
- [Gateway Dashboard Design](./GatewayDashboardDesign.md)
- [Grpc](./Grpc.md)
- [GatewayConfiguration](./GatewayConfiguration.md)
- [Galaxy Repository Browse](./GalaxyRepository.md)