mxaccessgw/docs/implementation-plan-gateway.md

# Gateway Implementation Plan

This plan implements the .NET 10 gateway process first. It covers contracts,
configuration, API-key authentication, worker lifecycle, gRPC APIs, event
streaming, metrics, dashboard, tests, and operational hooks.

Primary designs:

- `docs/gateway-process-design.md`
- `docs/gateway-dashboard-design.md`
- `docs/design-decisions.md`
- `docs/toolchain-links.md`

## Milestone: gateway-foundation

Goal: create the solution, shared contracts, configuration model, logging, and
test scaffolding that all later work depends on.

### Issue: Scaffold Gateway Solution And Projects

Labels: `area:gateway`, `type:infra`, `priority:p0`

Deliverables:

- create `src/MxGateway.sln`,
- create `src/MxGateway.Contracts`,
- create `src/MxGateway.Server`,
- create `src/MxGateway.Tests`,
- create `src/MxGateway.IntegrationTests`,
- target `MxGateway.Server` to `net10.0`,
- add shared C# build settings in `Directory.Build.props`,
- add baseline tests.

Acceptance criteria:

- `dotnet build src/MxGateway.sln` succeeds,
- `dotnet test src/MxGateway.sln` succeeds,
- gateway project does not reference MXAccess COM.

### Issue: Define Protobuf Contracts

Labels: `area:contracts`, `type:feature`, `priority:p0`

Deliverables:

- `src/MxGateway.Contracts/Protos/mxaccess_gateway.proto`,
- `src/MxGateway.Contracts/Protos/mxaccess_worker.proto`,
- `MxAccessGateway` service with `OpenSession`, `CloseSession`, `Invoke`, and
  `StreamEvents`,
- `WorkerEnvelope` and worker IPC messages,
- `MxValue`, `MxArray`, `MxStatusProxy`, `MxEvent`, and first-slice command
  payloads,
- generated C# code.

Acceptance criteria:

- generated code builds,
- worker envelopes include protocol version, session id, sequence, and
  correlation id,
- command replies preserve protocol status, HRESULT, return value, out params,
  and status arrays.

Tests:

- protobuf generation smoke,
- serialization round-trip for command, reply, event, value, and status.

### Issue: Add Gateway Configuration And Validation

Labels: `area:gateway`, `type:feature`, `priority:p0`

Deliverables:

- typed options for authentication, worker, sessions, events, dashboard, and
  protocol,
- startup validation,
- defaults matching design docs,
- redacted effective-configuration model.

Acceptance criteria:

- invalid worker path, invalid queue capacity, invalid auth config, and invalid
  dashboard config fail startup clearly,
- redacted config never includes API key pepper or raw secrets.

Tests:

- options binding,
- validation,
- redaction.

### Issue: Add Structured Logging And Metrics Foundation

Labels: `area:gateway`, `type:infra`, `priority:p0`

Deliverables:

- logging scopes for session id, worker process id, correlation id, command
  method, and client identity,
- counters/gauges/histograms for sessions, workers, commands, events, queues,
  and faults,
- redaction helpers.

Acceptance criteria:

- common logs include correlation fields,
- API keys and credential-bearing values are not logged,
- metrics can feed dashboard snapshots.

Tests:

- log redaction,
- metric update tests.

## Milestone: gateway-auth

Goal: implement API-key authentication backed by SQLite.

### Issue: Implement SQLite Auth Store And Migrations

Labels: `area:auth`, `type:feature`, `priority:p0`

Deliverables:

- SQLite schema for `schema_version`, `api_keys`, and `api_key_audit`,
- idempotent startup migrations,
- newer-schema startup block,
- key lookup and audit services.

Acceptance criteria:

- empty DB initializes,
- existing DB migrates,
- newer DB version blocks startup,
- revoked keys cannot authenticate.

Tests:

- temp SQLite migration tests,
- key lookup tests,
- revoked key tests.

### Issue: Implement API Key Hashing And Verification

Labels: `area:auth`, `type:feature`, `priority:p0`

Deliverables:

- parse `mxgw_<key-id>_<secret>` format,
- HMAC-SHA256 with gateway-local pepper or accepted Argon2id dependency,
- constant-time hash comparison,
- key id/display name/scopes identity model.

Acceptance criteria:

- raw secrets are never stored,
- malformed keys fail unauthenticated,
- valid keys authenticate,
- revoked keys fail.

Tests:

- parse tests,
- hash verification,
- redaction,
- scope extraction.

### Issue: Implement Local API Key Admin CLI

Labels: `area:auth`, `type:feature`, `priority:p1`

Deliverables:

- local admin CLI or gateway subcommand,
- `init-db`,
- `create-key`,
- `list-keys`,
- `revoke-key`,
- `rotate-key`,
- JSON output option.

Acceptance criteria:

- created key can authenticate,
- listed keys never show raw secret,
- revoked key fails authentication,
- raw secret is printed exactly once on create/rotate.

Tests:

- CLI parser,
- temp DB command tests,
- JSON redaction.

### Issue: Add gRPC Authentication And Scope Authorization

Labels: `area:auth`, `area:gateway`, `type:feature`, `priority:p0`

Deliverables:

- gRPC auth middleware/interceptor,
- request identity context,
- scope checks for sessions, invoke, secure invoke, events, metadata, and
  admin actions.

Acceptance criteria:

- missing/invalid key returns unauthenticated,
- valid key with missing scope returns permission denied,
- auth applies to unary and streaming calls.

Tests:

- unary auth,
- streaming auth,
- scope mapping.

## Milestone: gateway-sessions-ipc

Goal: create, supervise, and communicate with per-session workers.

### Issue: Implement Worker Frame Protocol

Labels: `area:gateway`, `area:contracts`, `type:feature`, `priority:p0`

Deliverables:

- little-endian uint32 length-prefixed frame reader/writer,
- max message size enforcement,
- protobuf envelope validation,
- protocol violation errors.

Acceptance criteria:

- valid frames round-trip,
- partial reads are handled,
- oversized frames fail before allocation,
- wrong protocol/session id is detected.

Tests:

- round-trip,
- partial read,
- malformed length,
- max size,
- wrong protocol/session.

### Issue: Implement Worker Process Launcher

Labels: `area:gateway`, `type:feature`, `priority:p0`

Deliverables:

- worker executable validation,
- process launch with session id, pipe name, protocol version,
- nonce via environment,
- startup timeout handling,
- failed-startup cleanup.

Acceptance criteria:

- command line contains no secrets,
- nonce is not logged,
- failed startup kills worker and disposes pipe,
- process id is recorded.

Tests:

- fake worker success/failure,
- timeout kill,
- command-line redaction.

### Issue: Implement Gateway WorkerClient

Labels: `area:gateway`, `type:feature`, `priority:p0`

Deliverables:

- named-pipe server,
- `GatewayHello`/`WorkerHello` handshake,
- read loop,
- write loop,
- pending command dictionary,
- event channel,
- heartbeat tracking,
- terminal fault handling.

Acceptance criteria:

- worker ready establishes `Ready` state,
- command reply completes matching pending command,
- worker events enter channel in order,
- pipe disconnect faults session.

Tests:

- fake worker protocol,
- command correlation,
- late reply,
- pipe disconnect,
- heartbeat expiration.

### Issue: Implement Session Manager And Registry

Labels: `area:gateway`, `type:feature`, `priority:p0`

Deliverables:

- session state machine,
- registry keyed by session id,
- `OpenSession` orchestration,
- `CloseSession` idempotency,
- lease hooks,
- gateway shutdown cleanup.

Acceptance criteria:

- only `Ready` sessions accept commands,
- close is idempotent,
- faulted sessions reject new commands,
- shutdown terminates workers.

Tests:

- state transitions,
- close idempotency,
- open failure cleanup,
- shutdown cleanup.

## Milestone: gateway-grpc-events-dashboard

Goal: expose the public API, stream events, and provide the dashboard.

### Issue: Implement Public gRPC Service

Labels: `area:gateway`, `type:feature`, `priority:p0`

Deliverables:

- `MxAccessGatewayService`,
- `OpenSession`,
- `CloseSession`,
- `Invoke`,
- `StreamEvents`,
- request validation,
- public-to-worker mappers.

Acceptance criteria:

- missing session fails clearly,
- method-specific payloads map correctly,
- HRESULT/status survives in replies,
- transport errors are separate from command replies.

Tests:

- service unit tests,
- mapper tests,
- validation tests,
- reply/error mapping.

### Issue: Implement Event Streaming And Backpressure

Labels: `area:gateway`, `type:feature`, `priority:p0`

Deliverables:

- one active subscriber per session,
- second-subscriber rejection,
- ordered event streaming,
- fail-fast queue overflow,
- terminal fault propagation,
- event-rate metrics.

Acceptance criteria:

- event order preserved,
- stream cancellation detaches subscriber,
- queue overflow faults session,
- `OperationComplete` is not synthesized by gateway.

Tests:

- order,
- single-subscriber enforcement,
- cancellation,
- overflow.

### Issue: Implement Dashboard Snapshot Service

Labels: `area:dashboard`, `type:feature`, `priority:p1`

Deliverables:

- immutable dashboard snapshot DTOs,
- session summaries,
- worker summaries,
- metric summaries,
- fault summaries,
- `WatchSnapshotsAsync`.

Acceptance criteria:

- snapshot reads do not mutate session/worker state,
- secrets and credential values are redacted,
- subscribers dispose cleanly.

Tests:

- projection,
- redaction,
- subscription disposal,
- empty/active/faulted states.

### Issue: Implement Blazor Server Dashboard

Labels: `area:dashboard`, `type:feature`, `priority:p1`

Deliverables:

- Blazor Server hosting,
- Bootstrap CSS/JS assets,
- layout/nav,
- home page,
- sessions page,
- workers page,
- events page,
- settings page,
- real-time refresh.

Acceptance criteria:

- Bootstrap/local CSS only,
- no MudBlazor or other Blazor UI libraries,
- pages update without manual refresh,
- dashboard can be disabled by config.

Tests:

- snapshot service tests,
- component tests if bUnit is added,
- disabled-dashboard behavior.

### Issue: Implement Dashboard Authentication

Labels: `area:dashboard`, `area:auth`, `type:feature`, `priority:p1`

Deliverables:

- `/dashboard/login`,
- API-key validation with `admin` scope,
- HTTP-only secure cookie,
- logout,
- anti-forgery protection,
- optional explicit anonymous-localhost dev mode defaulting false.

Acceptance criteria:

- unauthenticated access is denied/redirected,
- non-admin key is denied,
- admin key logs in,
- cookies use secure settings,
- API keys never appear in query strings or logs.

Tests:

- auth decisions,
- non-admin denial,
- cookie properties,
- redaction.

## Milestone: integration-and-parity

Goal: prove gateway behavior with fake workers before depending on live
MXAccess.

### Issue: Build Fake Worker Test Harness

Labels: `area:tests`, `area:gateway`, `type:test`, `priority:p0`

Deliverables:

- fake worker executable or in-process transport,
- scripted hello/ready/reply/event/fault behavior,
- malformed protocol scenarios,
- slow/hung worker scenarios.

Acceptance criteria:

- gateway tests do not require installed MXAccess,
- fake worker simulates startup success/failure,
- fake worker emits ordered events and faults.

### Issue: Gateway End-To-End Smoke With Fake Worker

Labels: `area:tests`, `area:gateway`, `type:test`, `priority:p0`

Deliverables:

- open session,
- invoke `Register`, `AddItem`, `Advise`,
- stream one event,
- close session,
- verify metrics/dashboard snapshot changed.

Acceptance criteria:

- smoke passes without live MXAccess,
- worker exits,
- artifacts stay in temp directories.