mxaccessgw/docs/Authentication.md

# Gateway Authentication

The gateway authenticates inbound gRPC callers with API keys: a bearer token is
parsed, its secret is hashed with a peppered HMAC and compared in constant time
against a stored hash, and administrative and verification events are recorded to
an audit trail.

The peppered-HMAC pipeline itself — token parsing, secret generation, hashing,
constant-time compare, the SQLite schema, the key store, the verifier, and schema
migration — lives in the shared **`ZB.MOM.WW.Auth.ApiKeys`** package, of which
this gateway is the donor. The gateway does not reimplement or fork those types;
it binds the library through `AddZbApiKeyAuth` and layers gateway-specific
concerns on top: constraint enforcement, the gRPC authorization interceptor,
hot-path decorators, the admin CLI, the dashboard, and a canonical audit store
that supersedes the library's own audit table. This document describes the
consumer side — the token format, the options the gateway binds, the pieces it
adds, and where the library boundary sits. For the library internals (the concrete
`ApiKeyVerifier`, the SQLite stores, the schema and migrator), read the
`ZB.MOM.WW.Auth.ApiKeys` sources; they are not duplicated in this repository.

## Token Format

API keys travel in the HTTP `Authorization` header as a bearer token shaped
`mxgw_<keyId>_<secret>`. The `mxgw_` prefix scopes parsing to gateway tokens, the
`<keyId>` segment is the public identifier used for lookup, and `<secret>` is the
high-entropy portion verified against a stored hash. The prefix and the pepper
configuration key the gateway pins are constants on
`AuthStoreServiceCollectionExtensions`
(`TokenPrefix = "mxgw"`, `PepperSecretName = "MxGateway:ApiKeyPepper"`); they are
supplied to the library at registration so the library's parser and pepper
provider use the gateway's contract. The library parser rejects a malformed token
before any database round-trip, and only a well-formed `mxgw_<keyId>_<secret>`
token reaches the store lookup.

## Secrets And Peppered Hashing

New secret material is high-entropy: the library generates 32 random bytes and
encodes them URL-safe base64 (no padding) so a secret embeds in a header without
escaping. The gateway never persists a plaintext secret — only its hash.

Secrets are hashed with `HMAC-SHA256` keyed by a server-side **pepper**. The
pepper lives outside the database and is resolved from configuration under the
`MxGateway:ApiKeyPepper` key (the library's pepper provider reads it). Keeping the
pepper out of the SQLite file means an attacker who exfiltrates only the database
holds the hashes but lacks the keying material to brute-force candidate secrets,
even if the hash algorithm is known.

When the pepper is not configured, the library surfaces the failure as an
`InvalidOperationException` whose message reports the pepper is unavailable rather
than persisting a key with an unkeyed hash. The dashboard management path
(`DashboardApiKeyManagementService`) catches that condition and returns the
friendly "API key pepper is not configured." result instead of faulting the Blazor
circuit; it currently matches on the message text, so a library wording change
would need to be reflected there (a typed pepper-unavailable exception is a pending
library improvement).

## Verification

The gateway consumes the library's `IApiKeyVerifier` from
`GatewayGrpcAuthorizationInterceptor`. The verifier's flow is:

1. Parse the `Authorization` header into the key id and presented secret.
2. Look up the stored key record by key id.
3. Reject a revoked record, and reject an expired record whose `ExpiresUtc` is in
   the past. Expiry is opt-in — keys created without an expiry never expire; an
   expired key fails opaquely, indistinguishable to the client from any other auth
   failure.
4. Hash the presented secret with the configured pepper.
5. Compare hashes in constant time to avoid a timing oracle.
6. Stamp a `LastUsedUtc` timestamp and return a shared `ApiKeyIdentity` carrying
   the key id, key prefix, display name, scopes, and the opaque constraints JSON.

A verification failure is opaque to the client: the interceptor returns
`Unauthenticated`/`PermissionDenied` without disclosing which check failed, while
the failure detail is available server-side for audit.

`GatewayApiKeyIdentityMapper.ToGatewayIdentity` maps the library's shared
`ApiKeyIdentity` onto the gateway's own `ApiKeyIdentity`
(`Security/Authentication/ApiKeyIdentity.cs`), which exposes the deserialized
`ApiKeyConstraints` — parsed from the opaque constraints JSON via
`ApiKeyConstraintSerializer` — that the downstream `ConstraintEnforcer` and the
request-identity accessor enforce. The gateway identity exposes only non-secret
fields (`KeyId`, `KeyPrefix`, `DisplayName`, `Scopes`, `Constraints`).

### Hot-path caching and last-used coalescing

Left unmediated, every authenticated gRPC call costs a SQLite read plus a
`last_used_utc` **write** (the library verifier couples `MarkUsed` into
`VerifyAsync`), which makes the auth store the throughput ceiling on the
bulk-read workload. The gateway layers two decorators over the shared library's
registrations (in `AuthStoreServiceCollectionExtensions`) — it does not edit the
library:

- **`CachingApiKeyVerifier`** wraps the library `IApiKeyVerifier` with an
  `IMemoryCache` entry per successful verification, keyed on a SHA-256 hash of the
  presented token (never the plaintext secret). A cache hit within
  `MxGateway:Security:ApiKeyVerificationCacheSeconds` (default 15 s) returns the
  cached result without touching the store, so both the read and the coupled write
  are skipped. Only successes are cached; failures always reach the inner verifier.
  On a gateway-initiated revoke/rotate/delete the dashboard admin service calls
  `IApiKeyCacheInvalidator.Invalidate(keyId)`, evicting the cached entry
  immediately. `Invalidate` bumps a per-key generation counter **before** it evicts,
  and `VerifyAsync` snapshots that generation before the inner verify and re-checks
  it after writing the cache entry (set-then-recheck); a revoke that lands while a
  verification is still in flight in the inner library therefore discards that
  verification's repopulation instead of re-caching the just-revoked identity for a
  full TTL (SEC-34). The short TTL remains the backstop for two bounded-staleness
  windows it cannot close directly: (1) out-of-band mutations (a direct DB edit, or a
  revoke run by the separate `apikey` CLI process, whose in-memory cache is not the
  running gateway's cache); and (2) a key whose `ExpiresUtc` passes while cached keeps
  authenticating until the entry's TTL elapses — expiry is enforced by the inner
  library verifier, which a cache hit never reaches, and the verification identity the
  library returns carries no expiry timestamp, so the cache cannot cap an entry at the
  key's expiry (capping it needs the donor library to surface expiry on the
  verification identity). The default 15 s TTL bounds both windows.
- **`CoalescingMarkApiKeyStore`** wraps the library `IApiKeyStore` and forwards at
  most one `MarkUsed` write per key per
  `MxGateway:Security:ApiKeyLastUsedCoalesceSeconds` (default 60 s), so even under a
  cache miss the `last_used_utc` write is bounded to roughly one per key per minute
  rather than one per RPC. `last_used_utc` is a coarse staleness hint, not an audit
  record (audit rows are written separately), so bounded staleness of up to one
  window is acceptable.

`GatewayApiKeyIdentityMapper` additionally memoizes the constraints-JSON
deserialization by blob, so the per-call parse on the mapped identity collapses to
a dictionary lookup. Both windows are configurable and may be set to `0` to disable
the respective mechanism; see
[GatewayConfiguration](./GatewayConfiguration.md).

Failures are never cached — a wrong secret always reaches the store — so the
failure path is shielded by `ApiKeyFailureLimiter` instead, consulted before
`VerifyAsync` runs. It counts failures over one sliding
`MxGateway:Security:ApiKeyFailureWindowSeconds` window in two layers: a composite
`(transport peer, key id)` partition capped at `ApiKeyFailureLimit`, and a
per-key-id aggregate across all peers capped at `ApiKeyFailureAggregateLimit`. The
key id never partitions on its own — it is public, so an attacker-supplied one
would otherwise let any peer throttle a key it does not hold — and it joins the
partition only when the presented token is validly shaped
(`mxgw_<keyId>_<secret>`, key id at most 64 characters), with at most 32 key-id
partitions per address before the overflow collapses onto that address's fallback
partition. An over-limit state admits one probe per
`ApiKeyFailureProbeIntervalSeconds` through to the real verifier and refuses
everything else with `ResourceExhausted` before the store read, so a legitimate
holder presenting the correct secret always reaches the constant-time compare and
resets both layers; the counter's LRU eviction (`ApiKeyFailureTrackedPeers`)
prefers expired windows and will not drop an over-limit partition below a 2x
overshoot ceiling, so the memory bound cannot be turned into a way to clear an
active block. See [Authorization](./Authorization.md) for the enforcement path.

## Storage

API-key state lives in a dedicated SQLite database owned by the shared library.
SQLite is sufficient because credential volume is small, the gateway runs as a
single process, and the file is straightforward to back up and rotate independently
of the main application data.

The database path is `GatewayOptions.Authentication.SqlitePath`. Its code default
is derived from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)`
(`C:\ProgramData\MxGateway\gateway-auth.db` on Windows,
`/usr/share/MxGateway/gateway-auth.db` or the container equivalent elsewhere) so the
credential store is never written relative to the launch working directory on a
non-Windows host. `appsettings.json` no longer ships an explicit path (SEC-33): the
removed Windows literal matched the Windows code default and, being non-rooted on a
Unix host, would have resolved against the CWD there; deployed hosts override it
through the NSSM environment (`MxGateway__Authentication__SqlitePath`).
`GatewayOptionsValidator` rejects a `SqlitePath` that is not rooted **on the host
running the gateway** (`Path.IsPathRooted`, current OS) — a relative filename or a
foreign-platform literal fails fast at startup rather than scattering the store by
launch CWD (SEC-01, SEC-33).

The library owns the SQLite schema and connection factory. The `api_keys` table
carries the key id, key prefix, secret-hash blob, display name, serialized scopes,
optional serialized constraints, and the `created_utc`, `last_used_utc`,
`revoked_utc`, and `expires_utc` timestamps. Because the schema, stores, and migrator
belong to `ZB.MOM.WW.Auth.ApiKeys`, this document does not restate their column
readers or SQL; consult the library for that detail.

### Audit trail

The library emits its own API-key audit entries (from the admin verbs — create,
revoke, rotate, `init-db`, and constraint denials), but the gateway **overrides**
the library's `IApiKeyAuditStore` registration with
`CanonicalForwardingApiKeyAuditStore`. That adapter canonicalizes every
library-emitted `ApiKeyAuditEntry` onto the gateway's `AuditEvent` shape and routes
it through `IAuditWriter` (`CanonicalAuditWriter`) into `SqliteCanonicalAuditStore`,
which persists to a single **`audit_event`** table (columns `event_id`,
`occurred_at_utc`, `actor`, `action`, `outcome`, `category`, `target`,
`source_node`, `correlation_id`, `details_json`). Reads for the dashboard "recent
audit" view go back through the same adapter, which maps `audit_event` rows back to
`ApiKeyAuditEntry` values so the existing view keeps working unchanged.

Consequently the library's own `api_key_audit` table is left in place but
**unused** after adoption — nothing writes to it once the override is registered.
The canonical `audit_event` table is the single durable record of both API-key
administrative actions and the dashboard's own audit vocabulary
(`dashboard-create-key`, `dashboard-rotate-key`, `dashboard-revoke-key`,
`dashboard-delete-key`, and the session Close/Kill actions). This is why any prose
that describes credential audits as landing in `api_key_audit` is stale: the
canonical store is `audit_event`.

## Registration

`AuthStoreServiceCollectionExtensions.AddSqliteAuthStore(IConfiguration)` wires the
whole subsystem. It does not register the library types directly — it delegates to
the shared provider and then layers the gateway concerns:

```csharp
public static IServiceCollection AddSqliteAuthStore(
    this IServiceCollection services,
    IConfiguration configuration)
{
    // Pin the gateway's token prefix ("mxgw") and pepper key ("MxGateway:ApiKeyPepper")
    // as fallback defaults UNDER the supplied configuration, then register the shared
    // provider: it binds ApiKeyOptions from MxGateway:Authentication and wires the SQLite
    // stores, the configuration-backed pepper provider, the verifier, the migrator, and
    // the migration hosted service.
    services.AddZbApiKeyAuth(effectiveConfig, AuthenticationSectionPath);

    // SEC-08 hot-path decorators layered over the library registrations.
    services.AddMemoryCache();
    // CoalescingMarkApiKeyStore decorates IApiKeyStore; CachingApiKeyVerifier decorates
    // IApiKeyVerifier and also serves as IApiKeyCacheInvalidator.

    // Canonical audit: override the library's IApiKeyAuditStore so every API-key audit
    // event is forwarded through IAuditWriter into the audit_event table.
    services.AddSingleton<IApiKeyAuditStore, CanonicalForwardingApiKeyAuditStore>();

    // The shared admin command set (ApiKeyAdminCommands) and the gateway CLI runner.
    services.AddSingleton<ApiKeyAdminCliRunner>();

    return services;
}
```

The decorators wrap the library's last registration for each interface rather than
replacing the library types, preserving singleton semantics; the audit override is
registered after `AddZbApiKeyAuth` so it wins as the resolved `IApiKeyAuditStore`.

## Admin CLI

`ApiKeyAdminCommandLineParser.Parse` recognises a leading `apikey` argument and
dispatches to one of the subcommands declared by `ApiKeyAdminCommandKind`. Each
parsed invocation produces an `ApiKeyAdminCommand` (or an `ApiKeyAdminParseResult`
carrying an error). `ApiKeyAdminCliRunner` then runs the migrator, invokes the shared
`ApiKeyAdminCommands` verb, and writes text or JSON output via `ApiKeyAdminOutput`.
The returned `ApiKeyAdminListedKey` projection deliberately omits the secret hash so
listing a database never surfaces hash material.

The supported subcommands match `ApiKeyAdminCommandKind` exactly:

| Subcommand | Required options | Behaviour |
|------------|------------------|-----------|
| `init-db` | none | Runs the migrator and records an audit entry. |
| `create-key` | `--key-id`, `--display-name` | Generates a new secret, stores its peppered hash and optional constraints, and prints the assembled `mxgw_<keyId>_<secret>` token. Optional `--expires` sets an expiry (absolute ISO-8601 UTC, or a relative `<N>d`/`<N>h` from now); omit it for a non-expiring key. |
| `list-keys` | none | Lists every stored key with its scopes, constraints, revocation state, and expiry (`active`/`expired`/`revoked`). |
| `revoke-key` | `--key-id` | Marks the key revoked if it is currently active. |
| `rotate-key` | `--key-id` | Replaces the secret hash and prints the new token. |

Examples:

```bash
mxgateway apikey init-db
mxgateway apikey create-key --key-id ops.alice --display-name "Alice (ops)" --scopes read,write
mxgateway apikey create-key --key-id area1.reader --display-name "Area 1 reader" --scopes invoke:read,metadata:read --read-subtree "Area1/*" --browse-subtree "Area1/*"
mxgateway apikey create-key --key-id ops.temp --display-name "Temp contractor" --scopes invoke:read --expires 90d
mxgateway apikey create-key --key-id team-a.svc --display-name "Team A service" --scopes session:open,invoke:read --dashboard-tags team-a
mxgateway apikey create-key --key-id ops.audit --display-name "Audit window" --scopes metadata:read --expires 2027-01-01T00:00:00Z
mxgateway apikey list-keys --json
mxgateway apikey revoke-key --key-id ops.alice
mxgateway apikey rotate-key --key-id ops.alice
```

Constraint flags are optional. `--read-subtree`, `--write-subtree`,
`--read-tag-glob`, `--write-tag-glob`, and `--browse-subtree` are repeatable.
`--max-write-classification` accepts one integer. `--read-alarm-only` and
`--read-historized-only` are boolean flags. `--dashboard-tags` takes a
comma-separated list (`--dashboard-tags team-a,team-b`) and is repeatable; its
segments are trimmed and de-duplicated ordinal-ignore-case, and an empty segment
is rejected rather than dropped so a stray comma cannot silently persist a grant
the operator did not write. It is the **only constraint flag** that splits its
value on commas (`--scopes`, which is not a constraint, is the other flag that
does): the repeatable subtree and glob flags each take exactly one value per
occurrence, so `--read-subtree "Area1/*,Area2/*"` is a single literal pattern
containing a comma, not two patterns. Repeat the flag instead. Existing rows with null constraints remain fully
unconstrained after migration; rows written before `--dashboard-tags` existed
deserialize as untagged, unchanged in every other respect.

`--dashboard-tags` is *not* a data-access constraint — it only labels the key for
dashboard event visibility, and sessions the key opens inherit it. See
[Authorization](./Authorization.md#constraint-enforcement).

`list-keys` prints the tags as a trailing tab-separated column (`-` when
untagged); the values are operator-chosen labels, not key material.

Key ids are restricted by the parser to ASCII letters, digits, periods, and hyphens
so they remain safe to embed in the token format and in URL paths used by
administrative tooling.

The CLI is not the only management surface: the dashboard API Keys page creates,
rotates, revokes, and deletes (revoked-only) keys through the same shared admin
command set. Every destructive dashboard action is gated by a confirmation dialog
and emits its own audit event (`dashboard-create-key`, `dashboard-rotate-key`,
`dashboard-revoke-key`, `dashboard-delete-key`) into the canonical `audit_event`
store. The page also surfaces expiry: each row shows an `Expires` column (`Never`
when unset) and a status badge that reads `Expired`, `Expiring` (within seven days),
`Revoked`, or `Active`. This staleness surfacing is display-only; expiry is set at
creation time via `apikey create-key --expires`, not from the dashboard. See
[Gateway Dashboard Design](./GatewayDashboardDesign.md#api-keys-page).

## Related Documentation

- [Gateway Configuration](./GatewayConfiguration.md)
- [Authorization](./Authorization.md)
- [Gateway Dashboard Design](./GatewayDashboardDesign.md)
- [Diagnostics](./Diagnostics.md)