mxaccessgw/docs/Authentication.md

Gateway Authentication

The gateway authentication subsystem verifies inbound API key credentials against a SQLite-backed key store, hashes secrets with a configurable pepper, and records administrative and verification events to an audit trail.

Token Format

API keys travel in the HTTP Authorization header as a bearer token shaped mxgw_<keyId>_<secret>. The mxgw_ prefix scopes parsing to gateway tokens, the <keyId> segment is the public identifier used for lookup, and <secret> is the high-entropy portion that the gateway verifies against a stored hash.

ApiKeyParser enforces the format and rejects malformed tokens before any database round-trip:

public bool TryParseAuthorizationHeader(string? authorizationHeader, out ParsedApiKey? apiKey)
{
    apiKey = null;

    if (string.IsNullOrWhiteSpace(authorizationHeader)
        || !authorizationHeader.StartsWith(BearerPrefix, StringComparison.OrdinalIgnoreCase))
    {
        return false;
    }

    string token = authorizationHeader[BearerPrefix.Length..].Trim();

    if (!token.StartsWith(TokenPrefix, StringComparison.OrdinalIgnoreCase))
    {
        return false;
    }

A successful parse produces a ParsedApiKey(KeyId, Secret) record. The IApiKeyParser interface exists so verification consumers can be tested without depending on header-format details.

Parsing and Secrets

Secret generation

ApiKeySecretGenerator.Generate() is the single source of new secret material. It uses 32 bytes from RandomNumberGenerator.Fill and encodes with URL-safe base64 (no padding) so secrets can be embedded in headers without escaping:

public static string Generate()
{
    Span<byte> bytes = stackalloc byte[32];
    RandomNumberGenerator.Fill(bytes);

    return Convert.ToBase64String(bytes)
        .TrimEnd('=')
        .Replace('+', '-')
        .Replace('/', '_');
}

Peppered hashing

ApiKeySecretHasher (registered behind IApiKeySecretHasher) hashes secrets with HMACSHA256 keyed by a server-side pepper. The pepper lives outside the database and is resolved by IConfiguration lookup against the configured PepperSecretName:

public byte[] HashSecret(string secret)
{
    string pepper = GetPepper();
    byte[] pepperBytes = Encoding.UTF8.GetBytes(pepper);
    byte[] secretBytes = Encoding.UTF8.GetBytes(secret);

    using HMACSHA256 hmac = new(pepperBytes);

    return hmac.ComputeHash(secretBytes);
}

The pepper is intentionally not stored alongside the hash: an attacker who exfiltrates only the SQLite file holds the hashes but lacks the keying material to brute-force candidate secrets, even if the stored hash algorithm and salt scheme are known. If the pepper is missing the hasher throws ApiKeyPepperUnavailableException, which the verifier converts to a distinct failure code rather than treating it as a credential mismatch.

Verification

ApiKeyVerifier (IApiKeyVerifier) implements the verification flow:

1. Parse the Authorization header into a ParsedApiKey.
2. Look up the ApiKeyRecord by KeyId through IApiKeyStore.FindByKeyIdAsync.
3. Reject revoked records (RevokedUtc is not null) and expired records (ExpiresUtc in the past). Expiry is opt-in — keys created without an expiry never expire; an expired key fails opaquely, indistinguishable to the client from any other auth failure.
4. Hash the presented secret with the configured pepper.
5. Compare hashes with CryptographicOperations.FixedTimeEquals to avoid timing oracles.
6. Record a LastUsedUtc timestamp via MarkKeyUsedAsync and return an ApiKeyIdentity.

if (!CryptographicOperations.FixedTimeEquals(presentedHash, storedKey.SecretHash))
{
    return ApiKeyVerificationResult.Fail(ApiKeyVerificationFailure.SecretMismatch);
}

await keyStore.MarkKeyUsedAsync(storedKey.KeyId, DateTimeOffset.UtcNow, cancellationToken)
    .ConfigureAwait(false);

return ApiKeyVerificationResult.Success(new ApiKeyIdentity(
    KeyId: storedKey.KeyId,
    KeyPrefix: storedKey.KeyPrefix,
    DisplayName: storedKey.DisplayName,
    Scopes: storedKey.Scopes,
    Constraints: storedKey.Constraints));

ApiKeyVerificationResult carries either an ApiKeyIdentity or a discriminated ApiKeyVerificationFailure value. The failure enum distinguishes parse errors, missing pepper, missing or revoked keys, and secret mismatch so the calling middleware can emit precise audit detail without leaking which check failed to the client.

ApiKeyIdentity exposes only non-secret fields (KeyId, KeyPrefix, DisplayName, Scopes, and Constraints) and is the type downstream authorization code consumes.

Hot-path caching and last-used coalescing

Left unmediated, every authenticated gRPC call costs a SQLite read plus a last_used_utc write (the library verifier couples MarkKeyUsed into VerifyAsync), which makes the auth store the throughput ceiling on the bulk-read workload. The gateway layers two decorators over the shared library's registrations (in AuthStoreServiceCollectionExtensions) — it does not edit the library:

• CachingApiKeyVerifier wraps IApiKeyVerifier with an IMemoryCache entry per successful verification, keyed on a SHA-256 hash of the presented token (never the plaintext secret). A cache hit within MxGateway:Security:ApiKeyVerificationCacheSeconds (default 15 s) returns the cached result without touching the store, so both the read and the coupled write are skipped. Only successes are cached; failures always reach the inner verifier. On a gateway-initiated revoke/rotate/delete the dashboard admin service calls IApiKeyCacheInvalidator.Invalidate(keyId), evicting the cached entry immediately. The short TTL is the backstop for out-of-band mutations (a direct DB edit, or a revoke run by the separate apikey CLI process, whose in-memory cache is not the running gateway's cache).
• CoalescingMarkApiKeyStore wraps IApiKeyStore and forwards at most one MarkUsed write per key per MxGateway:Security:ApiKeyLastUsedCoalesceSeconds (default 60 s), so even under a cache miss the last_used_utc write is bounded to roughly one per key per minute rather than one per RPC. last_used_utc is a coarse staleness hint, not an audit record (audit rows are written separately), so bounded staleness of up to one window is acceptable.

GatewayApiKeyIdentityMapper additionally memoizes the constraints-JSON deserialization by blob, so the per-call parse on the mapped identity collapses to a dictionary lookup. Both windows are configurable and may be set to 0 to disable the respective mechanism; see GatewayConfiguration.

Storage

The gateway keeps API key state in a dedicated SQLite database. SQLite is sufficient because credential volume is small, the gateway runs as a single process, and the file is straightforward to back up and rotate independently of the main application data.

The database path is GatewayOptions.Authentication.SqlitePath. Its code default is derived from Environment.GetFolderPath(SpecialFolder.CommonApplicationData) (C:\ProgramData\MxGateway\gateway-auth.db on Windows, /usr/share/MxGateway/gateway-auth.db or the container equivalent elsewhere) so the credential store is never written relative to the launch working directory on a non-Windows host. The production hosts pin the explicit Windows path in appsettings.json. GatewayOptionsValidator rejects a non-rooted (relative) SqlitePath so a bad override fails fast at startup rather than scattering the store by launch CWD (SEC-01).

Connection factory

AuthSqliteConnectionFactory reads GatewayOptions.Authentication.SqlitePath, ensures the parent directory exists, and builds a connection string in ReadWriteCreate mode so first-run installations can create the file without manual provisioning. Connection pooling is enabled and the connection string carries a non-zero DefaultTimeout:

SqliteConnectionStringBuilder builder = new()
{
    DataSource = sqlitePath,
    Mode = SqliteOpenMode.ReadWriteCreate,
    Pooling = true,
    DefaultTimeout = (int)BusyTimeout.TotalSeconds,
};

Every store opens its connection through OpenConnectionAsync, which opens the connection and then applies PRAGMA journal_mode=WAL and PRAGMA busy_timeout. WAL is a persistent database-level setting so re-applying it per connection is a cheap no-op; busy_timeout is per-connection state. Because MarkKeyUsedAsync runs on every authenticated request and SqliteApiKeyAuditStore appends on every denial, this lets concurrent readers and writers retry briefly instead of surfacing SQLITE_BUSY as a hard failure on the request path.

Schema

SqliteAuthSchema declares table names and the current schema version as constants. Three tables are involved:

• api_keys stores key_id, key_prefix, the secret_hash blob, display_name, serialized scopes, optional serialized constraints, and the created_utc, last_used_utc, and revoked_utc timestamps.
• api_key_audit is an append-only log keyed by an autoincrement audit_id with key_id, event_type, remote_address, created_utc, and details columns.
• schema_version carries a single row whose version column is matched against SqliteAuthSchema.CurrentVersion.

Read paths

SqliteApiKeyStore (IApiKeyStore) handles the two reads needed at request time: FindByKeyIdAsync returns any record (so revoked keys can be reported distinctly) and FindActiveByKeyIdAsync filters to non-revoked rows. MarkKeyUsedAsync updates last_used_utc only for non-revoked rows so a freshly revoked key cannot have its timestamp refreshed by a racing verification.

ApiKeyRecord is the in-memory projection. ApiKeyRecordReader.Read is shared by every read path so column ordering is defined in one place:

public static ApiKeyRecord Read(SqliteDataReader reader)
{
    return new ApiKeyRecord(
        KeyId: reader.GetString(0),
        KeyPrefix: reader.GetString(1),
        SecretHash: (byte[])reader["secret_hash"],
        DisplayName: reader.GetString(3),
        Scopes: ApiKeyScopeSerializer.Deserialize(reader.GetString(4)),
        Constraints: ApiKeyConstraintSerializer.Deserialize(reader.IsDBNull(5) ? null : reader.GetString(5)),
        CreatedUtc: DateTimeOffset.Parse(reader.GetString(6), System.Globalization.CultureInfo.InvariantCulture),
        LastUsedUtc: ReadNullableDateTimeOffset(reader, 7),
        RevokedUtc: ReadNullableDateTimeOffset(reader, 8));
}

Write paths

SqliteApiKeyAdminStore (IApiKeyAdminStore) implements administrative mutations: CreateAsync accepts an ApiKeyCreateRequest, RevokeAsync sets revoked_utc only when not already revoked, RotateAsync replaces secret_hash, clears last_used_utc, and clears revoked_utc so a rotated key is immediately usable, and DeleteAsync permanently removes a row but only when revoked_utc IS NOT NULL — active keys are untouched (returns false) so the revoke event lands in the audit log before the row disappears.

Because RotateAsync clears revoked_utc, rotating a previously revoked key reactivates it. The dashboard API Keys page therefore offers the Rotate (and Revoke) actions only for keys whose status is Active; revoked keys instead show a Delete action that calls DeleteAsync, so an operator can permanently remove a revoked row without ever risking un-revocation as a side effect of a rotation.

The dashboard API Keys page also surfaces expiry: each row shows an Expires column (Never when unset) and a status badge that reads Expired (past expiry, red), Expiring (within seven days, amber), Revoked, or Active. This is display-only staleness surfacing; expiry is set at creation time via the apikey create-key --expires CLI, not from the dashboard.

Audit trail

SqliteApiKeyAuditStore (IApiKeyAuditStore) appends ApiKeyAuditEntry values to the api_key_audit table and stamps each row with a UTC timestamp inside the store rather than trusting the caller. ListRecentAsync returns the most recent rows ordered by audit_id descending and projects them into ApiKeyAuditRecord. Rows are kept even after the referenced key is revoked because the audit history is the durable record of administrative action; the key_id column is nullable to accommodate non-key-scoped events such as init-db.

Migration

Schema bring-up is centralised behind IAuthStoreMigrator. SqliteAuthStoreMigrator executes the migration inside a single transaction so a partial failure leaves the database untouched, refuses to start when the on-disk schema version is newer than the binary supports, and idempotently creates the v1 schema:

if (existingVersion > SqliteAuthSchema.CurrentVersion)
{
    throw new AuthStoreMigrationException(
        $"Auth database schema version {existingVersion} is newer than supported version {SqliteAuthSchema.CurrentVersion}.");
}

await ApplyVersionOneAsync(connection, transaction, cancellationToken).ConfigureAwait(false);

await transaction.CommitAsync(cancellationToken).ConfigureAwait(false);

AuthStoreMigrationHostedService runs the migrator at startup, but only when API-key authentication is enabled and RunMigrationsOnStartup is true. Operators who manage schema out-of-band can disable the hosted run and use the admin CLI's init-db command instead.

AuthStoreMigrationException is a sealed InvalidOperationException so it can be caught precisely without swallowing unrelated failures.

Admin CLI

ApiKeyAdminCommandLineParser.Parse recognises a leading apikey argument and dispatches to one of the subcommands declared by ApiKeyAdminCommandKind. Each parsed invocation produces an ApiKeyAdminCommand (or an ApiKeyAdminParseResult carrying an error). ApiKeyAdminCliRunner then executes the command, runs the migrator first, calls the relevant store method, appends an audit row, and writes either text or JSON output via ApiKeyAdminOutput. The returned ApiKeyAdminListedKey projection deliberately omits the secret_hash so listing a database does not surface hash material.

The supported subcommands match ApiKeyAdminCommandKind exactly:

Subcommand	Required options	Behaviour
init-db	none	Runs the migrator and records an audit entry.
create-key	--key-id, --display-name	Generates a new secret, stores its peppered hash and optional constraints, and prints the assembled mxgw_<keyId>_<secret> token. Optional --expires sets an expiry (absolute ISO-8601 UTC, or a relative <N>d/<N>h from now); omit it for a non-expiring key.
list-keys	none	Lists every stored key with its scopes, constraints, revocation state, and expiry (active/expired/revoked).
revoke-key	--key-id	Sets revoked_utc if the key is currently active.
rotate-key	--key-id	Replaces the secret hash and prints the new token.

Examples:

mxgateway apikey init-db
mxgateway apikey create-key --key-id ops.alice --display-name "Alice (ops)" --scopes read,write
mxgateway apikey create-key --key-id area1.reader --display-name "Area 1 reader" --scopes invoke:read,metadata:read --read-subtree "Area1/*" --browse-subtree "Area1/*"
mxgateway apikey create-key --key-id ops.temp --display-name "Temp contractor" --scopes invoke:read --expires 90d
mxgateway apikey create-key --key-id ops.audit --display-name "Audit window" --scopes metadata:read --expires 2027-01-01T00:00:00Z
mxgateway apikey list-keys --json
mxgateway apikey revoke-key --key-id ops.alice
mxgateway apikey rotate-key --key-id ops.alice

Constraint flags are optional. --read-subtree, --write-subtree, --read-tag-glob, --write-tag-glob, and --browse-subtree are repeatable. --max-write-classification accepts one integer. --read-alarm-only and --read-historized-only are boolean flags. Existing rows with null constraints remain fully unconstrained after migration.

Key ids are restricted by the parser to ASCII letters, digits, periods, and hyphens so they remain safe to embed in the token format and in URL paths used by administrative tooling.

The CLI is not the only management surface: the dashboard API Keys page creates, rotates, revokes, and deletes (revoked-only) keys through the same IApiKeyAdminStore. Every destructive dashboard action is gated by a confirmation dialog and emits its own audit event (dashboard-create-key, dashboard-rotate-key, dashboard-revoke-key, dashboard-delete-key). See Gateway Dashboard Design.

Scope Serialization

Scopes are persisted as a single TEXT column rather than a join table because the set is small, never queried by membership at the database level, and changes atomically with the owning row. ApiKeyScopeSerializer.Serialize writes a JSON array sorted with StringComparer.Ordinal so equivalent scope sets produce byte-identical column values, which makes audit diffing and database comparisons deterministic:

public static string Serialize(IReadOnlySet<string> scopes)
{
    return JsonSerializer.Serialize(scopes.Order(StringComparer.Ordinal));
}

public static IReadOnlySet<string> Deserialize(string value)
{
    if (string.IsNullOrWhiteSpace(value))
    {
        return new HashSet<string>(StringComparer.Ordinal);
    }

    string[]? scopes = JsonSerializer.Deserialize<string[]>(value);

    return new HashSet<string>(scopes ?? [], StringComparer.Ordinal);
}

Deserialize tolerates an empty column by returning an empty set so older rows or hand-edited records do not crash the verifier.

Registration

AuthStoreServiceCollectionExtensions.AddSqliteAuthStore wires every service in this subsystem as a singleton and registers the migration hosted service:

public static IServiceCollection AddSqliteAuthStore(this IServiceCollection services)
{
    services.AddSingleton<IApiKeyParser, ApiKeyParser>();
    services.AddSingleton<IApiKeySecretHasher, ApiKeySecretHasher>();
    services.AddSingleton<IApiKeyVerifier, ApiKeyVerifier>();
    services.AddSingleton<ApiKeyAdminCliRunner>();
    services.AddSingleton<AuthSqliteConnectionFactory>();
    services.AddSingleton<IAuthStoreMigrator, SqliteAuthStoreMigrator>();
    services.AddSingleton<IApiKeyStore, SqliteApiKeyStore>();
    services.AddSingleton<IApiKeyAdminStore, SqliteApiKeyAdminStore>();
    services.AddSingleton<IApiKeyAuditStore, SqliteApiKeyAuditStore>();
    services.AddHostedService<AuthStoreMigrationHostedService>();

    return services;
}

Singletons are safe because each operation opens its own short-lived SqliteConnection through the factory; there is no shared mutable state inside the services.

Related Documentation

• Gateway Configuration
• Authorization
• Gateway Dashboard Design
• Diagnostics