Compare commits

..

21 Commits

Author SHA1 Message Date
dohertj2 69ea7937ca Merge pull request 'Architecture remediation: P1 tier (process & hardening)' (#121) from fix/archreview-p1 into main
ci / windows (push) Waiting to run
ci / live-mxaccess (push) Waiting to run
ci / java (push) Failing after 40s
ci / portable (push) Failing after 46s
2026-07-09 09:59:48 -04:00
dohertj2 b75bfbe8f4 Merge pull request 'fix(archreview): P0 tier remediation (8 findings)' (#120) from fix/archreview-p0 into main 2026-07-09 09:58:55 -04:00
Joseph Doherty a038363e74 feat(security): apikey --expires CLI flag + dashboard expiry/staleness badge (SEC-10 polish)
ci / windows (push) Waiting to run
ci / live-mxaccess (push) Waiting to run
ci / windows (pull_request) Waiting to run
ci / live-mxaccess (pull_request) Waiting to run
ci / portable (push) Failing after 44s
ci / java (push) Failing after 50s
ci / java (pull_request) Failing after 37s
ci / portable (pull_request) Failing after 55s
Gateway-side follow-up to the shared auth-lib expiry core (delivered via G-2):

- apikey create-key gains optional --expires — absolute ISO-8601 UTC or relative
  <N>d/<N>h from now; omitted means non-expiring (opt-in, unchanged default).
  Threaded ApiKeyAdminCommand -> parser -> runner into the library's
  CreateKeyAsync(..., expiresUtc, ...). list-keys shows an expiry column and an
  active/expired/revoked status.
- Dashboard API Keys page surfaces expiry: an Expires column (Never when unset)
  and a status badge reading Expired (red) / Expiring (<=7d, amber) / Revoked /
  Active. DashboardApiKeySummary.ExpiresUtc projected in DashboardSnapshotService;
  StatusBadge maps the new states.

Tests: parser (absolute/relative/invalid/none) + end-to-end past-expiry rejection
through the live verifier; dashboard summary suite green. Docs: Authentication.md
(verification-flow expiry step, CLI table + examples, dashboard badge).

Closes the tracked SEC-10 polish (SEC-10 core was already Done).
2026-07-09 09:48:09 -04:00
Joseph Doherty de57d834b1 docs(archreview-p1): mark GWC-04 Done — Wave 3 complete
ci / windows (push) Waiting to run
ci / live-mxaccess (push) Waiting to run
ci / windows (pull_request) Waiting to run
ci / live-mxaccess (pull_request) Waiting to run
ci / portable (push) Failing after 3m27s
ci / java (push) Failing after 3m30s
ci / java (pull_request) Failing after 46s
ci / portable (pull_request) Failing after 51s
2026-07-09 09:28:39 -04:00
Joseph Doherty 32d6b48910 feat(gateway): decouple worker event enqueue from the read loop (GWC-04)
The read loop awaited EnqueueWorkerEventAsync inline, which blocks in the bounded
event channel's timed WriteAsync (up to EventChannelFullModeTimeout, default 5s)
when the channel is full with no/slow consumer. A WorkerCommandReply or heartbeat
queued behind an event frame was then stalled, so an in-flight InvokeAsync could
hit CommandTimeout even though the worker replied in time.

Mirror the existing outbound WriteLoopAsync: add an unbounded event staging
channel and a dedicated EventWriteLoopAsync. DispatchEnvelope is now fully
synchronous — the WorkerEvent branch hands the event off with a non-blocking
TryWrite and the read loop never awaits. The event write loop owns the timed
WriteAsync into the bounded channel and the sustained-overflow ProtocolViolation
fault (unchanged contract). Registered in WaitForBackgroundTasks + completed on
close/fault/dispose.

Test: reply arriving after events with a full, consumer-less event channel is
dispatched promptly (no CommandTimeout) — pipe-harness, verified on windev.
Docs: GatewayProcessDesign read/write/event-loop section.
2026-07-09 09:28:13 -04:00
Joseph Doherty bfbfcdd112 docs(archreview-p1): mark IPC-02/03/04 + WRK-04/07 Done (Wave 3, windev-verified) 2026-07-09 09:21:48 -04:00
Joseph Doherty 309296f467 test(ipc): refresh proto descriptor set + MaxMessageBytes default expectation
Follow-up to the IPC-02 proto change and IPC-03 headroom default:
- regenerate clients/proto/descriptors/mxaccessgw-client-v1.protoset with the
  pinned protoc 34.1 so it carries GatewayHello.max_frame_bytes (IPC-01 descriptor
  refresh; ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField).
- update GatewayOptionsTests design-default assertion to 16 MiB + 64 KiB reserve.
2026-07-09 09:16:31 -04:00
Joseph Doherty ebe6aeac98 feat(worker): adopt negotiated frame max, bound drain, priority write scheduler (IPC-02/04 + WRK-04/07 worker half)
Worker half of the Wave 3 size/backpressure + write-ordering pass:

- IPC-02: the worker adopts GatewayHello.max_frame_bytes during the handshake
  (WorkerFrameProtocolOptions.AdoptNegotiatedMaxMessageBytes) instead of a
  hard-coded default; 0 keeps the default, a value above a 256 MiB ceiling is
  rejected. Reader and writer share the options instance, applied before the
  message loop.
- IPC-04: DrainEvents caps each reply at MaxDrainEventsPerReply (10_000) and
  treats max_events = 0 as that cap rather than 'drain the entire queue', so one
  diagnostic drain cannot pack a session-killing reply frame.
- WRK-04: WorkerFrameWriter stamps the envelope Sequence at the actual point of
  writing (under the write lock) instead of at envelope creation, so the on-wire
  order and the stamped sequence always agree under concurrent producers.
- WRK-07: the writer is now a cooperative priority scheduler — callers enqueue at
  Control or Event priority and the draining lock-holder writes all control
  frames before any event frame, so replies/faults/heartbeats jump ahead of an
  event backlog. Per-frame validation/size rejections fail only that frame; a
  stream write failure fails all queued frames.

Tests: monotonic gap-free sequence under concurrency, control-before-event
priority (gated stream), negotiated-max adoption, DrainEvents zero-bound.
Worker builds x86 only — verified on windev.
2026-07-09 09:09:14 -04:00
Joseph Doherty c8b3a2281a feat(ipc): negotiate worker frame max, add gRPC headroom, bound DrainEvents (IPC-02/03/04 gateway half)
Proto foundation + gateway-side of the size/backpressure pass:

- IPC-02: add GatewayHello.max_frame_bytes (regen Generated/); gateway sends
  its negotiated worker-frame max in the handshake so the worker can adopt it
  instead of a hard-coded default. Worker read-half lands separately.
- IPC-03: give the pipe frame max envelope-overhead headroom above the public
  gRPC cap (WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes = 64 KiB;
  default Worker.MaxMessageBytes bumped to 16 MiB + reserve), cross-validate the
  headroom at startup, and pre-check command envelope size in WorkerClient so an
  oversized command fails only that correlation (ResourceExhausted) instead of
  faulting the whole session.
- IPC-04: reject DrainEvents max_events above a public ceiling in the request
  validator (worker per-reply cap lands with the worker half).

Docs: GatewayConfiguration, WorkerFrameProtocol, gateway.md.
Tests: headroom validation, DrainEvents bound, oversized-command per-command
failure (pipe-harness, verified on windev).
2026-07-09 08:49:59 -04:00
Joseph Doherty e1a505d662 docs(archreview-p1): record Windows temp-file-lock flakiness fix + windev verification
3 consecutive windev full-suite runs: 793 passed / 2 failed, 0 orphans (was flaky
2-4 fails). The 2 stable failures are pre-existing windev-environmental (cert SAN
machine-name assertion; event-stream concurrency timing) and pass on macOS.
2026-07-09 08:20:43 -04:00
Joseph Doherty 11a716a07f test+fix: eliminate Windows full-suite temp-file-lock flakiness
Full-host tests passed in isolation but failed 2-4/run under parallel execution on
Windows with 'the process cannot access the file ... because it is being used by
another process' (macOS never sees it — Unix deletes open files). Two shared-state
parallel collisions, discovered while verifying TST-08:

1. Self-signed cert generation. GatewayTlsBootstrapTests sets process-global
   Kestrel/TLS env vars that GatewayApplication.Build reads; a parallel host-building
   test inherits them mid-run and both generate a cert at the same path, racing on the
   fixed-name '<path>.tmp'. Fixes:
   - SelfSignedCertificateProvider: stage the PFX in a unique '<path>.<guid>.tmp' instead
     of a fixed name, so concurrent/interrupted writers never collide on the temp file
     (real robustness: two instances or a restart-during-write no longer clash). The
     final atomic Move (last-writer-wins) still yields an equivalent cert.
   - TestHostEnvironmentInitializer: default MxGateway__Tls__SelfSignedCertPath to a
     per-process temp path so the suite never writes the shared ProgramData default or
     fights the deployed service; first host-building test generates, the rest load it.
   - GatewayTlsBootstrapTests: [Collection] with DisableParallelization so its global
     env-var mutation cannot bleed into parallel tests (new GlobalEnvironmentCollection).

2. AuthStoreHealthCheckTests opened a real SQLite file; Microsoft.Data.Sqlite's
   connection pool keeps the .db handle open after 'await using', so the finally
   File.Delete threw. Reuse the existing TempDatabaseDirectory helper, which calls
   SqliteConnection.ClearAllPools() before deleting.

Server build clean; affected classes 17/17 on macOS. Windows full-suite verified separately.
2026-07-09 08:14:55 -04:00
Joseph Doherty 8c06635ee5 docs(archreview-p1): TST-08 orphaned-testhost leak does not reproduce
Empirically verified the full ZB.MOM.WW.MxGateway.Tests suite exits cleanly:
- macOS: 0 surviving testhost after a full run.
- windev (origin/main worktree): 0 new testhost and 0 new worker processes
  after a full run, isolated by process StartTime.

Static audit confirms the prime-suspect fixtures already dispose deterministically
(WorkerClient cancels _stopCts + awaits its read/write/heartbeat tasks with a 5s
timeout; GatewaySession disposes the client; harness/E2E fixtures use await using).

Drop the stale 'leaves orphaned testhost processes' rationale from CLAUDE.md's
Source Update Workflow; keep filtered-run guidance as a speed guideline only.

Separately discovered (tracked in 00-tracking.md change log, NOT part of TST-08):
Windows-only temp-file-lock flakiness in ~4 full-host test classes (Sqlite
connection-pool retaining a temp .db handle; SelfSignedCertificateProvider's
fixed-name gw.pfx.tmp racing teardown) that macOS never surfaces.
2026-07-09 08:02:33 -04:00
Joseph Doherty 74e815c4d7 fix(archreview-p1): SEC-02/12/20 dashboard + observability hardening
- SEC-02: DashboardAuthorizationHandler restricts the loopback and
  Authentication:Mode=Disabled bypasses to read-only. They now satisfy only a
  Viewer-bearing requirement (AnyDashboardRole), never AdminOnly, so anonymous
  localhost can view the dashboard but cannot reach API-key CRUD or session
  Close/Kill at the policy layer (previously guarded only by service re-checks).
- SEC-12: DashboardSessionAdminService emits canonical AuditEvents through
  IAuditWriter (actions dashboard-close-session / dashboard-kill-worker, category
  SessionAdmin) on Success/Failure/Denied, mirroring the API-key audit path so
  destructive session actions leave durable, queryable rows.
- SEC-20: drop the unbounded session_id tag from the exported
  mxgateway.heartbeats.failed counter (per-session detail stays in the snapshot/log).

Docs updated same-change: CLAUDE.md (read-only loopback + 5-min bearer),
GatewayDashboardDesign.md (bypass scoping + session-admin audit), Metrics.md.
Server build clean; 30/30 targeted + 295/295 Dashboard/Security/App/Metrics sweep.
2026-07-09 07:47:51 -04:00
Joseph Doherty c2df4a0aae docs(archreview): P1 Wave 2b status (SEC-05/07/08/11 Done) 2026-07-09 07:32:14 -04:00
Joseph Doherty 17f16ea181 fix(archreview): security authz+hub cluster (SEC-07/05/08/11) + validation hardening
SEC-07: add QueryActiveAlarmsRequest -> events:read scope arm; fix two tests that
  constructed StreamAlarmsRequest instead of QueryActiveAlarmsRequest.
SEC-05: shorten hub-token lifetime 30m -> 5m; document that the ?access_token= query
  carriage must never be request-logged.
SEC-08: gateway-side CachingApiKeyVerifier (short TTL, keyed on a hash of the presented
  secret) skips the per-call store read+last_used write; CoalescingMarkApiKeyStore bounds
  last_used writes to <=1/key/min; identity constraints are cached. Invalidation is wired
  at the gateway admin sites (revoke/rotate/delete); short TTL backstops out-of-process CLI.
SEC-11: fixed-window rate limit on POST /auth/login + a per-peer (key-id) failure limiter
  checked before VerifyAsync; new MxGateway:Security options bound + validated.

Also fixes regressions from the SEC-01/04/06 commit (c185f62) that a narrow test filter
missed (all now covered by a full-suite checkpoint):
- Rooting check is cross-platform: accepts Windows C:\/UNC forms on Unix so the shipped
  appsettings path validates on the macOS dev box, still rejecting bare filenames.
- AddGatewayConfiguration TryAdds a non-production IHostEnvironment fallback so the validator
  resolves in minimal test/tooling containers; the real host + apikey CLI register the actual
  environment first (TryAdd no-op there).
- Test assembly defaults ASPNETCORE_ENVIRONMENT=Development (ModuleInitializer) so full-host
  tests exercise wiring instead of tripping the SEC-04/06 production guards.
- GatewayOptionsTests asserts the SEC-01 CommonApplicationData-derived default (platform-correct).

archreview: SEC-07/05/08/11 (P1). Verified: NonWindows build clean; full gateway suite
747 passed / 42 failed, where all 42 are the pre-existing macOS named-pipe-harness failures
(Unix-socket path limit) and 0 are validation/regression failures.
2026-07-09 07:31:35 -04:00
Joseph Doherty 970613eebd docs(archreview): P1 Wave 2a status (SEC-01/04/06 Done; SEC-10 Done via G-2 Auth 0.1.4) 2026-07-09 06:41:40 -04:00
Joseph Doherty c185f621a4 fix(archreview): security config guards (SEC-01/04/06)
- SEC-01: derive AuthenticationOptions.SqlitePath / TlsOptions.SelfSignedCertPath
  defaults from SpecialFolder.CommonApplicationData (was Windows-literal strings
  that became CWD-relative files off-Windows); validator rejects non-rooted
  overrides; delete the stray C:\ProgramData\...gateway-auth.db files that
  materialized under src/; add a tree-hygiene test failing on any *.db under src/.
- SEC-04: GatewayOptionsValidator fails startup when IsProduction() && DisableLogin.
- SEC-06: validator rejects LDAP Transport=None in Production; document the
  MxGateway__Ldap__ServiceAccountPassword env override + dev-credential rotation.

Galaxy SnapshotCachePath rooting could not be validator-enforced (bound by the
external GalaxyRepository package, not GatewayOptions) — documented instead.

archreview: SEC-01/04/06 (P1). Verified on the Auth-0.1.4 baseline: Server build
clean, GatewayOptionsValidator + GatewayTreeHygiene tests 53/53.
2026-07-09 06:40:57 -04:00
Joseph Doherty 197731ae2d chore(auth): consume ZB.MOM.WW.Auth 0.1.4 — gain API-key expiry enforcement (archreview G-2)
Bump all four ZB.MOM.WW.Auth.* package refs 0.1.2 -> 0.1.4. The shared
ApiKeyVerifier now rejects any key whose ExpiresUtc is in the past;
existing keys have NULL expiry (never expire), so nothing breaks, and the
auth SQLite DB auto-migrates to schema v3 (nullable expires_utc column) on
first boot. Implement the two IApiKeyAdminStore members added in 0.1.3
(SetScopesAsync/SetEnabledAsync) on the test FakeApiKeyAdminStore.

Build green; no new test failures (the macOS worker-pipe IPC failures are
pre-existing/environmental, identical to the pre-bump baseline).
2026-07-09 06:37:10 -04:00
Joseph Doherty 8fb1a814d4 docs(archreview): P1 Wave 1 status (IPC-01/09/19/20, CLI-02 Done; TST-03 In review) 2026-07-09 06:18:03 -04:00
Joseph Doherty d5248f61a2 fix(archreview): CI pipeline + codegen freshness guards (IPC-01/09/19/20, TST-03)
- IPC-01: regenerate the stale client descriptor set and add ClientProtoInputTests
  (semantic, protoc-free) so a missing contract symbol fails the build.
- IPC-20: make publish-client-proto-inputs.ps1 -Check source_code_info-normalized
  so it is protoc-version tolerant.
- IPC-19: document + guard the "Generated/ must be committed for net48" rule
  (docs/Contracts.md, csproj comment, check-codegen.ps1).
- IPC-09: harden the per-client generate-proto scripts (PATH resolution + pinned
  grpcio/protobuf/java version asserts) so regeneration is reproducible.
- TST-03: add .gitea/workflows/ci.yml (portable/java/windows/live jobs) running the
  build, tests, client checks, and the codegen guards.
- Also: check-codegen.ps1 Check 3 guards the CLI-02 vendored Rust protos against drift.

archreview: IPC-01/09/19/20 Done, TST-03 In review (pipeline authored + validated,
not yet run on a Gitea runner). Verified on macOS: NonWindows build clean,
ClientProtoInputTests 5/5, -Check exit 0.
2026-07-09 06:17:56 -04:00
Joseph Doherty 219fa6ddb4 fix(CLI-02): make Rust crate buildable outside the repository
build.rs resolved the .proto files via a monorepo-relative path and packaging
relied on `cargo publish --no-verify` to hide the resulting failure. Vendor the
three canonical protos into clients/rust/protos/, resolve them via
CARGO_MANIFEST_DIR (in-repo path preferred, vendored fallback for the packaged
crate), ship them via Cargo.toml `include`, and drop `--no-verify` from
pack-clients.ps1.

archreview: CLI-02 (P1). Verified: cargo fmt/check/test (28) + clippy -D warnings,
and `cargo package` (no --no-verify) compiles standalone in the isolated staging
dir — the out-of-repo proof. Vendored-proto drift is guarded by check-codegen.ps1.
2026-07-09 06:17:56 -04:00
97 changed files with 5608 additions and 265 deletions
+124
View File
@@ -0,0 +1,124 @@
# Continuous integration for mxaccessgw (Gitea Actions; origin is Gitea at gitea.dohertylan.com).
#
# Scope note: the x86 Worker (src/ZB.MOM.WW.MxGateway.Worker) and Worker.Tests target
# .NET Framework 4.8 / x86 and need MXAccess COM installed, so they build ONLY on a Windows host.
# They are out of scope for this Linux `portable` job — the `windows` job below (self-hosted windev
# runner) covers them, and live-MXAccess integration runs are scheduled-only so they never gate a push.
name: ci
on:
push:
pull_request:
schedule:
# Nightly live-MXAccess smoke (windev). Push/PR runs skip the live-mxaccess job.
- cron: '0 6 * * *'
jobs:
portable:
# Any Linux runner: no MXAccess, no x86. Covers the NonWindows solution, gateway fake-worker
# tests, the codegen/descriptor freshness guards, and the clients that build on Linux.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
- uses: actions/setup-go@v5
with:
go-version: '1.26'
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy, rustfmt
# protoc is pinned to 34.1 (see docs/ToolchainLinks.md) so the committed client descriptor
# regenerates reproducibly. The freshness check normalizes source_code_info, so it tolerates
# patch drift, but keep the CI toolchain on the pin.
- name: Install protoc 34.1
run: |
curl -sSL -o /tmp/protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v34.1/protoc-34.1-linux-x86_64.zip
sudo unzip -o /tmp/protoc.zip -d /usr/local bin/protoc 'include/*'
protoc --version
- name: Build NonWindows solution
run: dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx -c Release
# IPC-01 / IPC-19 / IPC-20: descriptor set + Contracts/Generated must match the current protos.
- name: Codegen / descriptor freshness
shell: pwsh
run: ./scripts/check-codegen.ps1
- name: Gateway fake-worker tests
run: dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj -c Release --no-build
- name: .NET client
run: dotnet build clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx -c Release
- name: Go client
working-directory: clients/go
run: |
test -z "$(gofmt -l .)" || (gofmt -l . && echo 'gofmt needed' && exit 1)
go build ./...
go test ./...
- name: Rust client
working-directory: clients/rust
run: |
cargo fmt --all -- --check
cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
- name: Python client
working-directory: clients/python
run: |
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
python -m pytest
java:
# Java client runs on a JDK-17 Linux runner (the macOS dev box has no JRE). The protobuf gradle
# plugin rewrites MxaccessGateway.java with spurious protobuf-runtime-version churn on every
# build; when no .proto changed, revert that one file so checkGeneratedClean / a dirty tree does
# not fail the build (repo memory project_java_generated_churn).
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'
- name: Gradle test
working-directory: clients/java
run: gradle test
- name: Revert spurious protobuf-version churn (no .proto changed)
run: git checkout -- clients/java/src/main/generated/main/java/mxaccess_gateway/v1/MxaccessGateway.java || true
- name: Verify generated tree is clean
run: git diff --exit-code -- clients/java/src/main/generated
windows:
# Self-hosted windev runner (10.100.0.48): the only host that can build the x86 / net48 Worker
# and MXAccess-adjacent code. Not required for a green portable build; runs where a runner exists.
runs-on: [self-hosted, windows]
steps:
- uses: actions/checkout@v4
- name: Build x86 Worker
run: dotnet build src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj -p:Platform=x86
- name: Worker tests
run: dotnet test src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj -p:Platform=x86
live-mxaccess:
# Opt-in, scheduled only — needs installed MXAccess COM + live provider state. Never gates a push.
if: ${{ github.event_name == 'schedule' }}
runs-on: [self-hosted, windows, mxaccess]
env:
MXGATEWAY_RUN_LIVE_MXACCESS_TESTS: '1'
steps:
- uses: actions/checkout@v4
- name: Live MXAccess smoke
run: dotnet test src/ZB.MOM.WW.MxGateway.IntegrationTests/ZB.MOM.WW.MxGateway.IntegrationTests.csproj --filter FullyQualifiedName~WorkerLiveMxAccessSmokeTests
+2 -2
View File
@@ -88,7 +88,7 @@ powershell -ExecutionPolicy Bypass -File scripts/run-client-e2e-tests.ps1
When source code changes, build and test the affected component before reporting work done. If the change crosses component boundaries, build each affected component — don't rely on a single top-level build:
**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow and leaves orphaned testhost processes — run it at most once per phase (after a related batch of tasks lands), not after every task.
**Run targeted tests per task, never the full suite each time.** When executing a plan task-by-task, run only the tests that exercise the code that task touched (`dotnet test --filter "FullyQualifiedName~<TestClass>"`, or the per-task test named in the plan). The full gateway suite is slow — run it at most once per phase (after a related batch of tasks lands), not after every task. This is a speed guideline, not a correctness one: the suite exits cleanly (verified on macOS and the Windows dev box — 0 surviving `testhost`/worker processes after a full run), so filtered runs are about turnaround, not about avoiding a process leak.
| Changed area | Required verification |
|---|---|
@@ -123,7 +123,7 @@ Gateway gRPC clients authenticate with an API key in metadata: `authorization: B
Session event streaming is **owner-scoped**: the API key that opened a session is recorded on the session, and every `StreamEvents` attach/reattach is rejected with `PermissionDenied` unless the caller's key id matches the owner. Possessing the `event` scope and knowing a session id is not sufficient — this closes the reconnect/fan-out trust boundary (detach-grace and replay retention are on by default) so an `event`-scoped key cannot attach to another key's retained session.
Dashboard auth is LDAP-backed (separate from the gRPC API-key model). `/login` binds against `MxGateway:Ldap` and maps the user's LDAP groups to `Admin` or `Viewer` via `MxGateway:Dashboard:GroupToRole`, then issues an HTTP-only secure `__Host-MxGatewayDashboard` cookie. SignalR hubs at `/hubs/{snapshot,alarms,events}` accept either the cookie or a 30-minute bearer minted at `/hubs/token`. `Dashboard:AllowAnonymousLocalhost` bypasses auth on loopback when enabled. `Dashboard:DisableLogin` (default `false`) auto-authenticates every dashboard request — including remote browsers — as `Dashboard:AutoLoginUser` (default `multi-role`) with both Admin and Viewer roles; dev/test only, never enable in production.
Dashboard auth is LDAP-backed (separate from the gRPC API-key model). `/login` binds against `MxGateway:Ldap` and maps the user's LDAP groups to `Admin` or `Viewer` via `MxGateway:Dashboard:GroupToRole`, then issues an HTTP-only secure `__Host-MxGatewayDashboard` cookie. SignalR hubs at `/hubs/{snapshot,alarms,events}` accept either the cookie or a 5-minute bearer minted at `/hubs/token`. `Dashboard:AllowAnonymousLocalhost` (default `true`) grants **read-only** loopback access — it satisfies the Viewer requirement but never the Admin-only requirement, so anonymous localhost can view the dashboard but not reach API-key CRUD or session Close/Kill (`Authentication:Mode=Disabled` is scoped the same way). `Dashboard:DisableLogin` (default `false`) auto-authenticates every dashboard request — including remote browsers — as `Dashboard:AutoLoginUser` (default `multi-role`) with both Admin and Viewer roles; dev/test only, never enable in production.
## Process / Platform Notes
+33 -24
View File
@@ -62,7 +62,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| GWC-01 | Critical | P0 | M | — | Done | Alarm monitor and distributor pump both drain the single worker event channel |
| GWC-02 | High | P0 | M | — | Done | Faulted sessions are never swept |
| GWC-03 | High | P0 | S | — | Done | Documented sparse-array max-length bound is unimplemented |
| GWC-04 | Medium | P1 | M | — | Not started | Full event channel stalls the worker read loop behind command replies |
| GWC-04 | Medium | P1 | M | — | Done | Full event channel stalls the worker read loop behind command replies |
| GWC-05 | Medium | — | S | — | Not started | Worker pipe created with no ACL / no CurrentUserOnly |
| GWC-06 | Medium | P2 | S | GWC-07,08 | Not started | Stopwatch allocated per streamed event |
| GWC-07 | Medium | P2 | S | GWC-06,08 | Not started | Every mapped event is deep-cloned |
@@ -90,10 +90,10 @@ Full design + implementation for each row lives in the linked domain doc under i
| WRK-01 | High | P0 | M | — | Done | Long `ReadBulk` self-faults as `StaHung`; all replies then dropped |
| WRK-02 | Medium | — | M | — | Not started | STA thread death after startup is silent; future work hangs forever |
| WRK-03 | Medium | — | S | WRK-02 | Not started | Commands after shutdown starts are dropped with no reply |
| WRK-04 | Medium | — | M | — | Not started | Envelope `sequence` can appear out of order on the wire |
| WRK-04 | Medium | — | M | — | Done | Envelope `sequence` can appear out of order on the wire |
| WRK-05 | Medium | — | M | — | Not started | One transient alarm-poll failure kills the whole session |
| WRK-06 | Medium | P2 | S | — | Not started | `MXSTATUS_PROXY` conversion reflects per field, per event |
| WRK-07 | Medium | P1 | M | WRK-04 | Not started | Documented outbound write priority not implemented |
| WRK-07 | Medium | P1 | M | WRK-04 | Done | Documented outbound write priority not implemented |
| WRK-08 | Low | — | S | — | Not started | Residual event queue discarded at graceful shutdown, undocumented |
| WRK-09 | Low | — | S | — | Not started | No `AppDomain.UnhandledException` hook |
| WRK-10 | Low | — | S | — | Not started | Top-level catch logs exception type but never message |
@@ -112,15 +112,15 @@ Full design + implementation for each row lives in the linked domain doc under i
| ID | Sev | Tier | Eff | Dep | Status | Title |
|---|---|:-:|:-:|---|---|---|
| IPC-01 | High | P1 | M | — | Not started | Published client descriptor set 7 weeks stale; nothing enforces freshness |
| IPC-02 | Medium | P1 | M | IPC-03 | Not started | Worker max frame size hard-coded, cannot follow gateway config |
| IPC-03 | Medium | P1 | M | IPC-02 | Not started | gRPC max == pipe max with zero headroom; oversized write faults whole session |
| IPC-04 | Medium | P1 | S | IPC-03 | Not started | `DrainEvents max_events=0` packs entire queue into one reply frame |
| IPC-01 | High | P1 | M | — | Done | Published client descriptor set 7 weeks stale; nothing enforces freshness |
| IPC-02 | Medium | P1 | M | IPC-03 | Done | Worker max frame size hard-coded, cannot follow gateway config |
| IPC-03 | Medium | P1 | M | IPC-02 | Done | gRPC max == pipe max with zero headroom; oversized write faults whole session |
| IPC-04 | Medium | P1 | S | IPC-03 | Done | `DrainEvents max_events=0` packs entire queue into one reply frame |
| IPC-05 | Medium | P2 | S | — | Not started | Redundant deep copies on command and event hot paths |
| IPC-06 | Medium | P2 | S | — | Not started | `gateway.md` Worker Envelope sketch no longer matches the contract |
| IPC-07 | Medium | P2 | S | — | Not started | `docs/Grpc.md` says six RPCs; there are seven |
| IPC-08 | Medium | — | M | — | Not started | `WorkerCancel` defined and handled but never sent |
| IPC-09 | Medium | P1 | M | IPC-01 | Not started | Known codegen fragilities have no in-repo guards |
| IPC-09 | Medium | P1 | M | IPC-01 | Done | Known codegen fragilities have no in-repo guards |
| IPC-10 | Low | — | S | — | Not started | Envelope `sequence` is write-only; monotonicity never validated |
| IPC-11 | Low | — | S | — | Not started | No protocol version negotiation despite `supported_protocol_version` name |
| IPC-12 | Low | — | S | — | Not started | Gateway frame writer has no write lock; integrity rests on undocumented invariant |
@@ -130,8 +130,8 @@ Full design + implementation for each row lives in the linked domain doc under i
| IPC-16 | Low | — | S | — | N/A | Correlation id carried twice per reply (Info) |
| IPC-17 | Low | P2 | S | — | Not started | Two docs name the wrong Python generated-output directory |
| IPC-18 | Low | — | S | IPC-01 | N/A | Generated-code hygiene verified good — preserve under change (positive) |
| IPC-19 | Low | P1 | S | IPC-01 | Not started | Generated/-must-be-committed rule for net48 undocumented and unguarded |
| IPC-20 | Low | P1 | S | IPC-01 | Not started | Descriptor `-Check` byte-compares source-info bytes; protoc-version-sensitive |
| IPC-19 | Low | P1 | S | IPC-01 | Done | Generated/-must-be-committed rule for net48 undocumented and unguarded |
| IPC-20 | Low | P1 | S | IPC-01 | Done | Descriptor `-Check` byte-compares source-info bytes; protoc-version-sensitive |
| IPC-21 | Low | P2 | S | IPC-06 | Not started | `gateway.md` presents unimplemented `Session` RPC as live |
| IPC-22 | Low | — | S | — | N/A | Public error-detail model stops at status codes plus prose (Info) |
@@ -139,18 +139,18 @@ Full design + implementation for each row lives in the linked domain doc under i
| ID | Sev | Tier | Eff | Dep | Status | Title |
|---|---|:-:|:-:|---|---|---|
| SEC-01 | Medium | P1 | M | — | Not started | Windows-absolute default paths become relative files off-Windows |
| SEC-02 | Medium | P1 | S | — | Not started | `AllowAnonymousLocalhost` satisfies AdminOnly, not just Viewer |
| SEC-01 | Medium | P1 | M | — | Done | Windows-absolute default paths become relative files off-Windows |
| SEC-02 | Medium | P1 | S | — | Done | `AllowAnonymousLocalhost` satisfies AdminOnly, not just Viewer |
| SEC-03 | Medium | P2 | S | — | Not started | Dashboard cookie lost its `__Host-` prefix; four docs still promise it |
| SEC-04 | Medium | P1 | S | SEC-03 | Not started | `DisableLogin` has no production guard |
| SEC-05 | Medium | P1 | M | — | Not started | Hub bearer tokens irrevocable for 30 min, carried in query string |
| SEC-06 | Medium | P1 | M | — | Not started | LDAP plaintext-by-default with a committed service password |
| SEC-07 | Medium | P1 | S | — | Not started | `QueryActiveAlarmsRequest` missing from scope resolver; tests mislabeled |
| SEC-08 | Medium | P1 | L | — | Not started | Per-RPC SQLite read + `last_used_utc` write; no verification cache |
| SEC-04 | Medium | P1 | S | SEC-03 | Done | `DisableLogin` has no production guard |
| SEC-05 | Medium | P1 | M | — | Done | Hub bearer tokens irrevocable for 30 min, carried in query string |
| SEC-06 | Medium | P1 | M | — | Done | LDAP plaintext-by-default with a committed service password |
| SEC-07 | Medium | P1 | S | — | Done | `QueryActiveAlarmsRequest` missing from scope resolver; tests mislabeled |
| SEC-08 | Medium | P1 | L | — | Done | Per-RPC SQLite read + `last_used_utc` write; no verification cache |
| SEC-09 | Medium | P2 | S | — | Not started | Dashboard design-doc `GroupToRole` sample now fails startup validation |
| SEC-10 | Medium | P1 | L | — | Not started | No API-key expiry |
| SEC-11 | Medium | P1 | M | SEC-08 | Not started | No rate limiting or lockout on either auth surface |
| SEC-12 | Medium | P1 | M | — | Not started | Dashboard Close/Kill bypass the canonical audit store |
| SEC-10 | Medium | P1 | L | — | Done | No API-key expiry |
| SEC-11 | Medium | P1 | M | SEC-08 | Done | No rate limiting or lockout on either auth surface |
| SEC-12 | Medium | P1 | M | — | Done | Dashboard Close/Kill bypass the canonical audit store |
| SEC-13 | Low | — | S | SEC-30 | Not started | Redactor's credential-command list omits secured-bulk variants |
| SEC-14 | Low | — | S | SEC-02,20 | Not started | `/metrics` + `/health` unauthenticated; a metric leaks session ids |
| SEC-15 | Low | — | S | — | Not started | GET `/logout` skips antiforgery |
@@ -158,7 +158,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| SEC-17 | Low | — | S | — | Not started | Interceptor does not override client-streaming/duplex handlers |
| SEC-18 | Low | — | S | — | Not started | Pepper-unavailable detection matches library message text |
| SEC-19 | Low | — | S | — | Not started | Canonical audit store re-issues `CREATE TABLE` per write/read |
| SEC-20 | Low | P1 | S | — | Not started | `session_id` metric tag is unbounded cardinality |
| SEC-20 | Low | P1 | S | — | Done | `session_id` metric tag is unbounded cardinality |
| SEC-21 | Low | — | M | — | Not started | Snapshot publisher works every second regardless of audience |
| SEC-22 | Low | P2 | M | — | Not started | `docs/Authentication.md` documents types no longer in this repo |
| SEC-23 | Low | — | S | SEC-03,04 | Not started | Validator ignores `DisableLogin`/`AutoLoginUser`/`RequireHttpsCookie`/`CookieName` |
@@ -175,7 +175,7 @@ Full design + implementation for each row lives in the linked domain doc under i
| ID | Sev | Tier | Eff | Dep | Status | Title |
|---|---|:-:|:-:|---|---|---|
| CLI-01 | High | P0 | M | — | Done | Go `Session.Events()` silently closes stream on 16-slot overflow |
| CLI-02 | High | P1 | M | — | Not started | Rust crate unbuildable outside the repo (`build.rs` path + `--no-verify`) |
| CLI-02 | High | P1 | M | — | Done | Rust crate unbuildable outside the repo (`build.rs` path + `--no-verify`) |
| CLI-03 | High | P0 | M | — | Done | Rust `invoke` never validates HRESULT / MXSTATUS_PROXY |
| CLI-04 | High | P2 | L | CLI-15 | Not started | Typed-command parity gap across all five clients |
| CLI-05 | Medium | — | S | — | Not started | .NET session cannot be re-attached to an existing session id |
@@ -215,12 +215,12 @@ Full design + implementation for each row lives in the linked domain doc under i
|---|---|:-:|:-:|---|---|---|
| TST-01 | High | P2 | L | TST-04 | Not started | Reconnect/replay has no e2e test and no client consumer |
| TST-02 | High | P0 | M | TST-04 | Done | Reconnect owner re-validation not implemented |
| TST-03 | High | P1 | M | — | Not started | No CI exists |
| TST-03 | High | P1 | M | — | In review | No CI exists |
| TST-04 | High | P2 | L | — | Not started | Session-resilience epic 16/28 tasks unfinished |
| TST-05 | Medium | P1 | S | TST-03 | Not started | Real-worker control/COM paths verified opt-in only |
| TST-06 | Medium | — | M | — | Not started | Dashboard live-data path untested |
| TST-07 | Medium | — | S | — | Not started | Real-clock sleeps with negative assertions are latent flakes |
| TST-08 | Medium | P1 | M | — | Not started | Full-suite orphaned testhost processes |
| TST-08 | Medium | P1 | M | — | Done | Full-suite orphaned testhost processes (does not reproduce; doc de-stale) |
| TST-09 | Medium | — | M | — | Not started | Health checks cover only the auth store |
| TST-10 | Medium | — | M | — | Not started | Deployment/upgrade undocumented and hand-rolled |
| TST-11 | Medium | P2 | M | — | Not started | No version discipline on server; client versions drift |
@@ -255,4 +255,13 @@ Findings the review flagged as one coordinated design pass — sequence them tog
|---|---|
| 2026-07-09 | Initial tracking doc generated from the six domain remediation designs. All 153 findings `Not started`. |
| 2026-07-09 | P0 tier executed via parallel agents. GWC-01/02/03, TST-02, TST-12, CLI-01, CLI-03 → `Done` (NonWindows build clean; gateway 135/135 targeted tests pass, Go `go test ./...` + `-race` pass, Rust `test`/`clippy -D warnings` pass). |
| 2026-07-09 | Windows full-suite temp-file-lock flakiness (discovered via TST-08) → **fixed** (commit `11a716a`). Two shared-state parallel collisions that macOS never surfaces: (1) self-signed cert generation — `SelfSignedCertificateProvider` now stages the PFX in a unique `<path>.<guid>.tmp` (was a fixed name that concurrent/interrupted writers collided on); `TestHostEnvironmentInitializer` defaults the cert path to a per-process temp dir (so the suite never writes shared `ProgramData` or fights the deployed service); `GatewayTlsBootstrapTests` moved to a `DisableParallelization` collection so its process-global env-var mutation can't bleed into parallel tests. (2) `AuthStoreHealthCheckTests` — reuse the existing `TempDatabaseDirectory` helper (`SqliteConnection.ClearAllPools()` before delete) so the Microsoft.Data.Sqlite pool releases the temp `.db` handle. **windev-verified:** 3 consecutive full-suite runs all **793 passed / 2 failed, 0 new orphans** (was flaky 24 fails, count-varying). The 2 stable remaining failures are pre-existing windev-environmental (`SelfSignedCertificateProviderTests.GenerateCertificate_HasExpectedSansEkuAndValidity` machine-name/FQDN SAN assertion; `EventStreamServiceTests…TracksAggregateQueueDepth` timing) — both pass on macOS, unrelated to this change. |
| 2026-07-09 | P1 TST-08 (orphaned testhost) → `Done` by evidence: the claimed leak does **not** reproduce. Full `ZB.MOM.WW.MxGateway.Tests` suite exits cleanly on macOS (0 survivors) and on the Windows dev box (windev, `origin/main` worktree: 0 new `testhost` and 0 new `ZB.MOM.WW.MxGateway.Worker` processes after a full run, isolated by StartTime). Static audit confirms the prime-suspect fixtures already dispose deterministically (WorkerClient cancels its `_stopCts` + awaits read/write/heartbeat tasks with a 5 s timeout; GatewaySession disposes the client; E2E/harness fixtures use `await using`). CLAUDE.md "Source Update Workflow" updated to drop the stale "leaves orphaned testhost processes" rationale (filtered runs kept as a speed guideline). **Discovered separately (NOT TST-08, still open):** Windows-only temp-file-lock flakiness in ~4 full-host test classes — `AuthStoreHealthCheckTests` (Microsoft.Data.Sqlite connection-pool holds the temp `.db` handle after `await using`, so the finally `File.Delete` throws "used by another process"), `GatewayTlsBootstrapTests`/`DashboardCookieOptionsTests`/`DashboardHubsRegistrationTests` (a `gw.pfx.tmp` handle from `SelfSignedCertificateProvider`'s fixed-name `path+".tmp"` atomic-write racing teardown). Count varies run-to-run (24 fails); macOS never sees it (Unix allows deleting open files). Fix = `SqliteConnection.ClearAllPools()` before temp delete + a unique/guarded `.tmp` name; verify on windev. |
| 2026-07-09 | SEC-10 polish (gateway-side follow-up to the G-2 auth-lib core) → done. `apikey create-key` gains an optional `--expires` (absolute ISO-8601 UTC or relative `<N>d`/`<N>h`; omit = non-expiring, opt-in) threaded through `ApiKeyAdminCommand`/parser/runner into the library's `CreateKeyAsync(..., expiresUtc, ...)`; `list-keys` shows an expiry column + `active`/`expired`/`revoked` status. Dashboard API Keys page surfaces expiry: an `Expires` column (`Never` when unset) and a status badge reading `Expired` (red) / `Expiring` (≤7 days, amber) / `Revoked` / `Active` (`DashboardApiKeySummary.ExpiresUtc` projected in `DashboardSnapshotService`; `StatusBadge` maps the new states). Server build clean; targeted tests 32/32 (parser abs/relative/invalid + end-to-end past-expiry rejection via the live verifier) + dashboard 28/28. Docs: `docs/Authentication.md` (verification flow expiry step, CLI table + examples, dashboard badge). SEC-10 was already `Done` (core); this closes the tracked polish. |
| 2026-07-09 | P1 Wave 3 (event-channel decoupling). GWC-04 → `Done`. `WorkerClient`'s read loop awaited `EnqueueWorkerEventAsync` inline, which blocks in the bounded event channel's timed `WriteAsync` (≤ `EventChannelFullModeTimeout`, 5 s) when the channel is full with a slow/absent `StreamEvents` consumer — stalling any `WorkerCommandReply`/heartbeat queued behind an event frame, so an in-flight `InvokeAsync` could hit `CommandTimeout` despite a timely worker reply. Fix mirrors the existing outbound `WriteLoopAsync`: an unbounded event **staging** channel + a dedicated `EventWriteLoopAsync`; `DispatchEnvelope` is now fully synchronous and the `WorkerEvent` branch hands off with a non-blocking `TryWrite`, so the read loop never awaits event enqueue. The event write loop owns the timed write + the sustained-overflow `ProtocolViolation` fault (unchanged contract); registered in `WaitForBackgroundTasks`, completed on close/fault/dispose. Server build clean (0 warnings); non-pipe event tests 37/38 (the 1 is the known parallel-load `EventStreamServiceTests…TracksAggregateQueueDepth` timing flake — passes in isolation). New pipe-harness test (reply after events with a full consumer-less channel dispatches without `CommandTimeout`) verified on windev. Docs: GatewayProcessDesign read/write/event-loop section. Commit `32d6b48`. **Wave 3 complete.** |
| 2026-07-09 | P1 Wave 3 (size/backpressure topology + write ordering). IPC-02/03/04 + WRK-04/07 → `Done`. **Size negotiation (IPC-02):** added `GatewayHello.max_frame_bytes` (regen `Generated/` + `clients/proto` descriptor refresh with pinned protoc 34.1); the gateway sends its negotiated worker-frame max and the worker adopts it (`WorkerFrameProtocolOptions.AdoptNegotiatedMaxMessageBytes`, 0 = keep default, >256 MiB rejected) instead of a hard-coded default. **Headroom (IPC-03):** the pipe frame max now sits `EnvelopeOverheadReserveBytes` (64 KiB) above the public gRPC cap (default `Worker.MaxMessageBytes` 16 MiB→16 MiB+64 KiB), cross-validated at startup; `WorkerClient` pre-checks command envelope size and fails only the offending correlation (`ResourceExhausted`) instead of `SetFaulted`ing the session. **Drain bound (IPC-04):** gateway request validator rejects `DrainEvents max_events` above 10 000; the worker caps each reply at `MaxDrainEventsPerReply` (10 000) and treats `max_events = 0` as that cap, not "drain all". **Sequence (WRK-04):** `WorkerFrameWriter` stamps the envelope `Sequence` at the point of writing under the write lock, so wire order and stamped sequence always agree under concurrent producers. **Priority (WRK-07):** the worker writer is now a cooperative priority scheduler — control frames (reply/fault/heartbeat/shutdown-ack) drain ahead of event frames; per-frame validation/size rejections fail only that frame, a stream failure fails all queued. Docs same-change (GatewayConfiguration, WorkerFrameProtocol, gateway.md). **Verified:** macOS NonWindows build clean + validator/grpc tests green; **windev** x86 worker builds clean, `Worker.Tests` 352 passed / 0 failed / 11 skipped (incl. new monotonic-sequence, control-before-event priority, negotiated-max, drain-bound tests), gateway `Tests` 799 passed / 3 failed — all 3 pre-existing windev-environmental (SelfSigned SAN + 2 `EventStreamServiceTests` timing, both pass in isolation). Commits `c8b3a22` (gateway half), `ebe6aea` (worker half), `309296f` (descriptor + default-expectation refresh). GWC-04 (event-channel decoupling) is the remaining Wave 3 item. |
| 2026-07-09 | P1 S-misc (dashboard/observability hardening). SEC-02/12/20 → `Done`. SEC-02: `DashboardAuthorizationHandler` restricts the loopback + `Authentication:Mode=Disabled` bypasses to read-only (they satisfy a Viewer-bearing requirement but never `AdminOnly`), closing the policy-layer gap where anonymous localhost was authorized for Admin surfaces. SEC-12: `DashboardSessionAdminService` now emits canonical `AuditEvent`s (`dashboard-close-session`/`dashboard-kill-worker`, category `SessionAdmin`) through `IAuditWriter` on Success/Failure/Denied, so Close/Kill land durable audit rows. SEC-20: dropped the unbounded `session_id` tag from the exported `mxgateway.heartbeats.failed` counter. Docs updated same-change (CLAUDE.md, GatewayDashboardDesign.md, Metrics.md). Server build clean (0 warnings); targeted classes 30/30 pass; broader Dashboard+Security+GatewayApplication+Metrics sweep 295/295 pass. |
| 2026-07-09 | P1 Wave 2b (security authz+hub). SEC-05/07/08/11 → `Done` (hub-token lifetime; QueryActiveAlarms scope arm; gateway-side verification cache + last-used coalescing; login rate limit + per-peer gRPC failure limiter). Full-suite checkpoint caught + fixed regressions the earlier narrow SEC-01/04/06 filter missed: cross-platform path-rooting, an `IHostEnvironment` fallback for minimal DI containers, a test-assembly `ASPNETCORE_ENVIRONMENT=Development` default, and a platform-correct default-path assertion. Suite: 747 passed / 42 failed, all 42 pre-existing macOS named-pipe-harness env failures. |
| 2026-07-09 | P1 Wave 2a (security). SEC-01/04/06 → `Done` (config path-rooting + production validator guards; Server build clean, validator+hygiene tests 53/53). SEC-10 → `Done`: the shared `ZB.MOM.WW.Auth.ApiKeys` gained optional `ExpiresUtc` (expired keys rejected, auth DB auto-migrates to schema v3) via a concurrent HistorianGateway-remediation session's "G-2"; this repo consumes it by bumping the four `Auth.*` refs 0.1.2→0.1.4 (commit 197731a). Remaining SEC-10 polish (`apikey create --expires` + dashboard staleness badge) tracked as a small follow-up. |
| 2026-07-09 | P1 Wave 1 (CI + codegen freshness + Rust buildability) via parallel agents. IPC-01/09/19/20, CLI-02 → `Done`; TST-03 → `In review` (CI pipeline authored + YAML/layout-validated, but not yet executed on a Gitea runner — proven on first push). Verified on macOS: NonWindows build clean, `ClientProtoInputTests` 5/5, `publish-client-proto-inputs.ps1 -Check` exit 0, `cargo package` (no `--no-verify`) compiles standalone. Added a vendored-Rust-proto drift guard (Check 3) to `check-codegen.ps1` closing the CLI-02 static-copy risk. IPC-09 script guards not executed end-to-end (need pinned python/JRE toolchains); logic is PATH-resolution + version assertion. |
| 2026-07-09 | WRK-01 → `Done`. Verified on Windows host (windev) via an isolated `origin/main` worktree: worker builds x86 clean, `StaRuntimeTests`+`WorkerPipeSessionTests` 33/33 pass. Fixed an `xUnit1030` build error (the new worker test used `.ConfigureAwait(false)` in `[Fact]` bodies) that the macOS tree could not surface. Also ran GWC-01's Windows-only `WorkerClientTests` on windev: 18/18 pass (incl. `ReadEventsAsync_SecondEnumerator_Throws`). All 8 P0 findings now `Done`. Not yet committed. |
+39 -11
View File
@@ -5,25 +5,46 @@ $repoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$protoRoot = Join-Path $repoRoot 'src\ZB.MOM.WW.MxGateway.Contracts\Protos'
$outputRoot = Join-Path $PSScriptRoot 'internal\generated'
$modulePath = 'gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/internal/generated'
$protoc = 'C:\Users\dohertj2\AppData\Local\Microsoft\WinGet\Packages\Google.Protobuf_Microsoft.Winget.Source_8wekyb3d8bbwe\bin\protoc.exe'
$goPluginPath = 'C:\Users\dohertj2\go\bin'
if (-not (Test-Path $protoc)) {
throw "protoc was not found at $protoc. See docs/ToolchainLinks.md."
}
function Resolve-Tool {
# Resolve a codegen tool from PATH first (portable), then the documented Windows fallbacks,
# instead of the previous hard-coded per-machine paths. See docs/ToolchainLinks.md.
param(
[string[]]$Names,
[string[]]$FallbackPaths
)
foreach ($pluginName in @('protoc-gen-go.exe', 'protoc-gen-go-grpc.exe')) {
$pluginPath = Join-Path $goPluginPath $pluginName
if (-not (Test-Path $pluginPath)) {
throw "$pluginName was not found at $pluginPath. See docs/ToolchainLinks.md."
foreach ($name in $Names) {
$cmd = Get-Command $name -ErrorAction SilentlyContinue
if ($null -ne $cmd) {
return $cmd.Source
}
}
foreach ($fallback in $FallbackPaths) {
if ($fallback -and (Test-Path $fallback)) {
return $fallback
}
}
throw "Could not find $($Names -join '/') on PATH. See docs/ToolchainLinks.md."
}
$wingetProtoc = if ($env:LOCALAPPDATA) {
Join-Path $env:LOCALAPPDATA 'Microsoft\WinGet\Packages\Google.Protobuf_Microsoft.Winget.Source_8wekyb3d8bbwe\bin\protoc.exe'
} else { $null }
$goBin = if ($env:USERPROFILE) { Join-Path $env:USERPROFILE 'go\bin' } elseif ($env:HOME) { Join-Path $env:HOME 'go/bin' } else { $null }
$protoc = Resolve-Tool -Names @('protoc', 'protoc.exe') -FallbackPaths @($wingetProtoc)
$protocGenGo = Resolve-Tool -Names @('protoc-gen-go', 'protoc-gen-go.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go' }))
$protocGenGoGrpc = Resolve-Tool -Names @('protoc-gen-go-grpc', 'protoc-gen-go-grpc.exe') -FallbackPaths @((if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc.exe' }), (if ($goBin) { Join-Path $goBin 'protoc-gen-go-grpc' }))
# protoc discovers the plugins on PATH; prepend the directories the resolved plugins live in.
$env:Path = (Split-Path $protocGenGo -Parent) + [System.IO.Path]::PathSeparator + (Split-Path $protocGenGoGrpc -Parent) + [System.IO.Path]::PathSeparator + $env:Path
New-Item -ItemType Directory -Path $outputRoot -Force | Out-Null
Get-ChildItem -Path $outputRoot -Filter '*.pb.go' -File | Remove-Item
$env:Path = "$goPluginPath;$env:Path"
& $protoc `
--proto_path=$protoRoot `
--go_out=$outputRoot `
@@ -35,6 +56,10 @@ $env:Path = "$goPluginPath;$env:Path"
mxaccess_worker.proto `
galaxy_repository.proto
if ($LASTEXITCODE -ne 0) {
throw "protoc (go) failed with exit code $LASTEXITCODE."
}
& $protoc `
--proto_path=$protoRoot `
--go-grpc_out=$outputRoot `
@@ -44,3 +69,6 @@ $env:Path = "$goPluginPath;$env:Path"
mxaccess_gateway.proto `
galaxy_repository.proto
if ($LASTEXITCODE -ne 0) {
throw "protoc (go-grpc) failed with exit code $LASTEXITCODE."
}
@@ -57,3 +57,35 @@ protobuf {
}
}
}
// IPC-09 guard: the generated Java tree is tracked (generatedFilesBaseDir above points at the
// committed src/main/generated). `gradle build` regenerates it, so an un-regenerated .proto change,
// or a plugin/protobuf version bump, silently drifts the committed output. checkGeneratedClean
// fails when the regenerated tree differs from what is committed.
//
// Caveat (repo memory project_java_generated_churn): the protobuf gradle plugin also rewrites
// MxaccessGateway.java with a spurious protobuf-runtime-version delta on every build even when no
// .proto changed. CI reverts that one file (git checkout) before invoking this task; locally, do the
// same when you did not touch a .proto. See docs/GatewayTesting.md "Continuous Integration".
tasks.register('checkGeneratedClean') {
group = 'verification'
description = 'Fails if the committed generated Java tree differs from a fresh regeneration.'
dependsOn 'generateProto'
doLast {
def generatedDir = 'clients/java/src/main/generated'
def stdout = new ByteArrayOutputStream()
def result = exec {
workingDir = rootProject.projectDir.parentFile.parentFile
commandLine 'git', 'status', '--porcelain', '--', generatedDir
standardOutput = stdout
ignoreExitValue = true
}
def dirty = stdout.toString().trim()
if (!dirty.isEmpty()) {
throw new GradleException(
"Generated Java is stale or churned:\n${dirty}\n" +
"Regenerate and commit after a .proto change, or 'git checkout' the spurious " +
"MxaccessGateway.java protobuf-version churn when no .proto changed.")
}
}
}
+44 -3
View File
@@ -1,15 +1,52 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
# Pinned generator baseline. The committed Python bindings are stamped with
# GRPC_GENERATED_VERSION = 1.80.0 and "Protobuf Python Version: 6.31.1". A newer grpcio-tools
# stamps a higher GRPC_GENERATED_VERSION than the pinned grpcio runtime (pyproject.toml:
# grpcio>=1.80,<2), which makes _pb2_grpc import raise at runtime and breaks pytest. Regeneration
# MUST use this exact grpcio-tools version. See docs/ClientProtoGeneration.md and the repo memory
# project_python_client_regen_pin.
$PinnedGrpcioToolsVersion = '1.80.0'
$repoRoot = Resolve-Path (Join-Path $PSScriptRoot '..\..')
$protoRoot = Join-Path $repoRoot 'src\ZB.MOM.WW.MxGateway.Contracts\Protos'
$outputRoot = Join-Path $PSScriptRoot 'src\zb_mom_ww_mxgateway\generated'
$python = 'C:\Users\dohertj2\AppData\Local\Programs\Python\Python312\python.exe'
if (-not (Test-Path $python)) {
throw "Python was not found at $python. See docs/ToolchainLinks.md."
function Resolve-Python {
# Prefer python on PATH (python on Windows, python3 on Linux/macOS), then the documented
# Windows install location. Avoids the previous hard-coded per-machine path.
foreach ($name in @('python', 'python3', 'python.exe')) {
$cmd = Get-Command $name -ErrorAction SilentlyContinue
if ($null -ne $cmd) {
return $cmd.Source
}
}
if ($env:LOCALAPPDATA) {
$documentedPath = Join-Path $env:LOCALAPPDATA 'Programs\Python\Python312\python.exe'
if (Test-Path $documentedPath) {
return $documentedPath
}
}
throw 'Could not find python on PATH. See docs/ToolchainLinks.md.'
}
function Assert-GrpcioToolsVersion {
param([string]$Python)
$version = (& $Python -c 'import grpc_tools; from importlib.metadata import version; print(version("grpcio-tools"))').Trim()
if ($version -ne $PinnedGrpcioToolsVersion) {
throw "grpcio-tools $version is installed, but regeneration is pinned to $PinnedGrpcioToolsVersion. " +
"Install the pin (python -m pip install 'grpcio-tools==$PinnedGrpcioToolsVersion') before regenerating, " +
"or the generated bindings will stamp a GRPC_GENERATED_VERSION the pinned grpcio runtime rejects."
}
}
$python = Resolve-Python
Assert-GrpcioToolsVersion -Python $python
New-Item -ItemType Directory -Path $outputRoot -Force | Out-Null
Get-ChildItem -Path (Join-Path $outputRoot '*_pb2.py') -File | Remove-Item
Get-ChildItem -Path (Join-Path $outputRoot '*_pb2_grpc.py') -File | Remove-Item
@@ -21,3 +58,7 @@ Get-ChildItem -Path (Join-Path $outputRoot '*_pb2_grpc.py') -File | Remove-Item
mxaccess_gateway.proto `
mxaccess_worker.proto `
galaxy_repository.proto
if ($LASTEXITCODE -ne 0) {
throw "grpc_tools.protoc failed with exit code $LASTEXITCODE."
}
+3
View File
@@ -38,6 +38,9 @@ Issues = "https://gitea.dohertylan.com/dohertj2/mxaccessgw/issues"
[project.optional-dependencies]
dev = [
# Runtime range for local dev, but REGENERATION is pinned to grpcio-tools==1.80.0
# (protobuf 6.31.1): a newer version stamps a GRPC_GENERATED_VERSION the pinned grpcio
# runtime rejects and breaks pytest. generate-proto.ps1 asserts the exact pin.
"grpcio-tools>=1.80,<2",
"pytest>=9,<10",
"pytest-asyncio>=1.3,<2",
+5
View File
@@ -13,6 +13,11 @@ keywords = ["mxaccess", "mxgateway", "grpc", "client", "archestra"]
categories = ["api-bindings", "asynchronous"]
publish = ["dohertj2-gitea"]
build = "build.rs"
# Ship the vendored `.proto` files so `build.rs` can regenerate the tonic/prost
# bindings from the packaged tarball — a consumer building the published crate
# has no access to the in-repo Contracts protos. See CLI-02. Cargo.toml, the
# README, and the license are included automatically.
include = ["src/**/*.rs", "protos/*.proto", "build.rs", "README.md"]
[workspace]
members = ["crates/mxgw-cli"]
+17 -3
View File
@@ -6,11 +6,25 @@ fn main() -> Result<(), Box<dyn Error>> {
configure_protoc();
let manifest_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR")?);
let repo_root = manifest_dir
// Resolve the proto source directory. In-repo builds prefer the canonical
// Contracts protos (two levels above clients/rust) so local `.proto` edits
// are picked up live. When that path is absent — e.g. a consumer building
// the crate unpacked from a published tarball — fall back to the vendored
// copies shipped in `clients/rust/protos/` (see build.rs staleness note and
// CLI-02 in archreview/remediation/50-clients.md). Vendored copies are
// build inputs only; the canonical source remains in Contracts and must be
// refreshed here whenever the `.proto` files change.
let repo_proto_root = manifest_dir
.parent()
.and_then(Path::parent)
.ok_or("clients/rust must live two levels below the repository root")?;
let proto_root = repo_root.join("src/ZB.MOM.WW.MxGateway.Contracts/Protos");
.map(|repo_root| repo_root.join("src/ZB.MOM.WW.MxGateway.Contracts/Protos"));
let vendored_proto_root = manifest_dir.join("protos");
let proto_root = match repo_proto_root {
Some(path) if path.is_dir() => path,
_ => vendored_proto_root,
};
let gateway_proto = proto_root.join("mxaccess_gateway.proto");
let worker_proto = proto_root.join("mxaccess_worker.proto");
let galaxy_proto = proto_root.join("galaxy_repository.proto");
+190
View File
@@ -0,0 +1,190 @@
syntax = "proto3";
package galaxy_repository.v1;
option csharp_namespace = "ZB.MOM.WW.MxGateway.Contracts.Proto.Galaxy";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";
// Wire-compatibility policy (ProtobufStyleGuide): this contract evolves
// additively only. Never renumber or repurpose an existing field number or
// enum value. When a field or enum value is removed, add a `reserved` range
// (and `reserved` name) covering it in the same change so a future editor
// cannot accidentally reuse the retired tag. There are no `reserved`
// declarations today because no field or enum value has ever been removed.
// Read-only browse over the AVEVA System Platform Galaxy Repository (ZB SQL
// database). Lets clients enumerate the deployed object hierarchy and each
// object's dynamic attributes so they know what tag references to subscribe
// to via the MxAccessGateway service.
service GalaxyRepository {
rpc TestConnection(TestConnectionRequest) returns (TestConnectionReply);
rpc GetLastDeployTime(GetLastDeployTimeRequest) returns (GetLastDeployTimeReply);
rpc DiscoverHierarchy(DiscoverHierarchyRequest) returns (DiscoverHierarchyReply);
// Server-stream of deploy events. The server emits the current state immediately
// on subscribe (so clients can bootstrap their cache without waiting for the next
// deploy), then emits one event each time the gateway's hierarchy cache observes
// a new galaxy.time_of_last_deploy. The sequence field is monotonically
// increasing per server start; gaps indicate the per-subscriber buffer dropped
// older events because the client was too slow.
rpc WatchDeployEvents(WatchDeployEventsRequest) returns (stream DeployEvent);
// Returns the direct children of a parent object (or the root objects when
// `parent` is unset). Designed for OPC UA-style lazy expand: clients walk
// one level at a time instead of paging the full hierarchy. Filters mirror
// DiscoverHierarchy exactly. Backed by the same shared hierarchy cache.
rpc BrowseChildren(BrowseChildrenRequest) returns (BrowseChildrenReply);
}
message TestConnectionRequest {}
message TestConnectionReply {
bool ok = 1;
}
message GetLastDeployTimeRequest {}
message GetLastDeployTimeReply {
bool present = 1;
google.protobuf.Timestamp time_of_last_deploy = 2;
}
message DiscoverHierarchyRequest {
// Maximum number of objects to return. The server applies its default when
// unset and rejects non-positive values.
int32 page_size = 1;
// Opaque token returned by a previous DiscoverHierarchy response.
string page_token = 2;
// Optional. When set, return only this object and its descendants.
// Empty = full hierarchy.
oneof root {
int32 root_gobject_id = 3;
string root_tag_name = 4;
string root_contained_path = 5;
}
// Optional. Cap on descendant depth from root. Zero returns only the root.
// Unset means unlimited depth.
google.protobuf.Int32Value max_depth = 6;
// Optional object category id filters.
repeated int32 category_ids = 7;
// Optional case-insensitive substring filters against template names.
repeated string template_chain_contains = 8;
// Optional anchored, case-insensitive glob over object tag_name.
string tag_name_glob = 9;
// Optional. Unset or true includes attributes. False returns object skeletons.
optional bool include_attributes = 10;
// Optional. Return only objects with at least one alarm-bearing attribute.
bool alarm_bearing_only = 11;
// Optional. Return only objects with at least one historized attribute.
bool historized_only = 12;
}
message DiscoverHierarchyReply {
repeated GalaxyObject objects = 1;
// Non-empty when another page is available.
string next_page_token = 2;
// Total number of objects in the cached hierarchy at the time of the call.
int32 total_object_count = 3;
}
message WatchDeployEventsRequest {
// Optional. When set, the bootstrap event is suppressed if the cached deploy
// time matches this value. Future events are still emitted normally.
google.protobuf.Timestamp last_seen_deploy_time = 1;
}
message DeployEvent {
// Monotonically increasing per server start. Gaps indicate dropped events.
uint64 sequence = 1;
// Server wall-clock when the cache observed the deploy.
google.protobuf.Timestamp observed_at = 2;
// Galaxy.time_of_last_deploy. Absent only when the Galaxy table reports null.
google.protobuf.Timestamp time_of_last_deploy = 3;
bool time_of_last_deploy_present = 4;
int32 object_count = 5;
int32 attribute_count = 6;
}
message GalaxyObject {
int32 gobject_id = 1;
string tag_name = 2;
string contained_name = 3;
string browse_name = 4;
int32 parent_gobject_id = 5;
bool is_area = 6;
int32 category_id = 7;
int32 hosted_by_gobject_id = 8;
repeated string template_chain = 9;
repeated GalaxyAttribute attributes = 10;
}
message GalaxyAttribute {
string attribute_name = 1;
string full_tag_reference = 2;
// Raw Galaxy SQL `dbo.data_type` identifier, passed through unchanged.
// This is NOT a member of `mxaccess_gateway.v1.MxDataType` — Galaxy's
// type enumeration is distinct from MXAccess's wire data-type enum and
// the two must not be cast or compared. The GalaxyRepository service is
// metadata-only and deliberately does not share types with
// mxaccess_gateway.proto. See docs/GalaxyRepository.md.
int32 mx_data_type = 3;
// Human-readable name from Galaxy's `dbo.data_type` table (e.g. "Float",
// "Integer", "Boolean"). Free-form Galaxy text; not a stable enum.
string data_type_name = 4;
bool is_array = 5;
int32 array_dimension = 6;
bool array_dimension_present = 7;
// Raw Galaxy SQL attribute-category identifier, passed through unchanged.
// Galaxy-specific; not mapped to any gateway enum. See
// docs/GalaxyRepository.md.
int32 mx_attribute_category = 8;
// Raw Galaxy SQL security-classification identifier, passed through
// unchanged. Galaxy-specific; not mapped to any gateway enum. See
// docs/GalaxyRepository.md.
int32 security_classification = 9;
bool is_historized = 10;
bool is_alarm = 11;
}
message BrowseChildrenRequest {
// Parent selector. Empty oneof returns root objects (parent_gobject_id == 0).
oneof parent {
int32 parent_gobject_id = 1;
string parent_tag_name = 2;
string parent_contained_path = 3;
}
// Maximum number of direct children to return. Server default 500; cap 5000.
int32 page_size = 4;
// Opaque token returned by a previous BrowseChildren response. Bound to the
// cache sequence, parent selector, and the filter set; a mismatch returns
// InvalidArgument.
string page_token = 5;
// --- Filter parity with DiscoverHierarchy. AND-combined. ---
repeated int32 category_ids = 6;
repeated string template_chain_contains = 7;
string tag_name_glob = 8;
optional bool include_attributes = 9;
bool alarm_bearing_only = 10;
bool historized_only = 11;
}
message BrowseChildrenReply {
// Direct children matching the filter, sorted areas-first then by
// case-insensitive display name (same order as the dashboard tree).
repeated GalaxyObject children = 1;
// Non-empty when another page of siblings is available.
string next_page_token = 2;
// Total matching direct children of the parent (post-filter).
int32 total_child_count = 3;
// Parallel array, indexed with `children`. True when the child has at least
// one matching descendant under the same filter set. Lets a UI choose
// whether to draw an expand triangle without an extra round trip.
repeated bool child_has_children = 4;
// Cache sequence this reply was projected from. Clients may pass it back as
// part of the page_token contract. Mismatch on the next page -> InvalidArgument.
uint64 cache_sequence = 5;
}
File diff suppressed because it is too large Load Diff
+132
View File
@@ -0,0 +1,132 @@
syntax = "proto3";
package mxaccess_worker.v1;
option csharp_namespace = "ZB.MOM.WW.MxGateway.Contracts.Proto";
import "google/protobuf/duration.proto";
import "google/protobuf/timestamp.proto";
import "mxaccess_gateway.proto";
// Wire-compatibility policy (ProtobufStyleGuide): this contract evolves
// additively only. Never renumber or repurpose an existing field number or
// enum value. When a field or enum value is removed, add a `reserved` range
// (and `reserved` name) covering it in the same change so a future editor
// cannot accidentally reuse the retired tag. There are no `reserved`
// declarations today because no field or enum value has ever been removed.
// Gateway-to-worker IPC envelope. Named-pipe framing prepends a little-endian
// uint32 payload length to this protobuf payload.
message WorkerEnvelope {
uint32 protocol_version = 1;
string session_id = 2;
uint64 sequence = 3;
string correlation_id = 4;
oneof body {
GatewayHello gateway_hello = 10;
WorkerHello worker_hello = 11;
WorkerReady worker_ready = 12;
WorkerCommand worker_command = 13;
WorkerCommandReply worker_command_reply = 14;
WorkerCancel worker_cancel = 15;
WorkerShutdown worker_shutdown = 16;
WorkerShutdownAck worker_shutdown_ack = 17;
WorkerEvent worker_event = 18;
WorkerHeartbeat worker_heartbeat = 19;
WorkerFault worker_fault = 20;
}
}
message GatewayHello {
uint32 supported_protocol_version = 1;
string nonce = 2;
string gateway_version = 3;
}
message WorkerHello {
uint32 protocol_version = 1;
string nonce = 2;
int32 worker_process_id = 3;
string worker_version = 4;
}
message WorkerReady {
int32 worker_process_id = 1;
string mxaccess_progid = 2;
string mxaccess_clsid = 3;
google.protobuf.Timestamp ready_timestamp = 4;
}
message WorkerCommand {
mxaccess_gateway.v1.MxCommand command = 1;
google.protobuf.Timestamp enqueue_timestamp = 2;
}
message WorkerCommandReply {
mxaccess_gateway.v1.MxCommandReply reply = 1;
google.protobuf.Timestamp completed_timestamp = 2;
}
message WorkerCancel {
string reason = 1;
}
message WorkerShutdown {
google.protobuf.Duration grace_period = 1;
string reason = 2;
}
message WorkerShutdownAck {
mxaccess_gateway.v1.ProtocolStatus status = 1;
}
message WorkerEvent {
mxaccess_gateway.v1.MxEvent event = 1;
}
message WorkerHeartbeat {
int32 worker_process_id = 1;
WorkerState state = 2;
google.protobuf.Timestamp last_sta_activity_timestamp = 3;
uint32 pending_command_count = 4;
uint32 outbound_event_queue_depth = 5;
uint64 last_event_sequence = 6;
string current_command_correlation_id = 7;
}
message WorkerFault {
WorkerFaultCategory category = 1;
string command_method = 2;
optional int32 hresult = 3;
string exception_type = 4;
string diagnostic_message = 5;
mxaccess_gateway.v1.ProtocolStatus protocol_status = 6;
}
enum WorkerState {
WORKER_STATE_UNSPECIFIED = 0;
WORKER_STATE_STARTING = 1;
WORKER_STATE_HANDSHAKING = 2;
WORKER_STATE_INITIALIZING_STA = 3;
WORKER_STATE_READY = 4;
WORKER_STATE_EXECUTING_COMMAND = 5;
WORKER_STATE_SHUTTING_DOWN = 6;
WORKER_STATE_STOPPED = 7;
WORKER_STATE_FAULTED = 8;
}
enum WorkerFaultCategory {
WORKER_FAULT_CATEGORY_UNSPECIFIED = 0;
WORKER_FAULT_CATEGORY_INVALID_ARGUMENTS = 1;
WORKER_FAULT_CATEGORY_GATEWAY_AUTHENTICATION_FAILED = 2;
WORKER_FAULT_CATEGORY_PROTOCOL_MISMATCH = 3;
WORKER_FAULT_CATEGORY_PROTOCOL_VIOLATION = 4;
WORKER_FAULT_CATEGORY_PIPE_DISCONNECTED = 5;
WORKER_FAULT_CATEGORY_MXACCESS_CREATION_FAILED = 6;
WORKER_FAULT_CATEGORY_MXACCESS_COMMAND_FAILED = 7;
WORKER_FAULT_CATEGORY_MXACCESS_EVENT_CONVERSION_FAILED = 8;
WORKER_FAULT_CATEGORY_STA_HUNG = 9;
WORKER_FAULT_CATEGORY_QUEUE_OVERFLOW = 10;
WORKER_FAULT_CATEGORY_SHUTDOWN_TIMEOUT = 11;
}
+41 -3
View File
@@ -73,7 +73,7 @@ The pepper is intentionally not stored alongside the hash: an attacker who exfil
1. Parse the `Authorization` header into a `ParsedApiKey`.
2. Look up the `ApiKeyRecord` by `KeyId` through `IApiKeyStore.FindByKeyIdAsync`.
3. Reject revoked records (`RevokedUtc is not null`).
3. Reject revoked records (`RevokedUtc is not null`) and expired records (`ExpiresUtc` in the past). Expiry is opt-in — keys created without an expiry never expire; an expired key fails opaquely, indistinguishable to the client from any other auth failure.
4. Hash the presented secret with the configured pepper.
5. Compare hashes with `CryptographicOperations.FixedTimeEquals` to avoid timing oracles.
6. Record a `LastUsedUtc` timestamp via `MarkKeyUsedAsync` and return an `ApiKeyIdentity`.
@@ -101,10 +101,44 @@ return ApiKeyVerificationResult.Success(new ApiKeyIdentity(
`DisplayName`, `Scopes`, and `Constraints`) and is the type downstream
authorization code consumes.
### Hot-path caching and last-used coalescing
Left unmediated, every authenticated gRPC call costs a SQLite read plus a
`last_used_utc` **write** (the library verifier couples `MarkKeyUsed` into
`VerifyAsync`), which makes the auth store the throughput ceiling on the
bulk-read workload. The gateway layers two decorators over the shared library's
registrations (in `AuthStoreServiceCollectionExtensions`) — it does not edit the
library:
- **`CachingApiKeyVerifier`** wraps `IApiKeyVerifier` with an `IMemoryCache`
entry per successful verification, keyed on a SHA-256 hash of the presented
token (never the plaintext secret). A cache hit within
`MxGateway:Security:ApiKeyVerificationCacheSeconds` (default 15 s) returns the
cached result without touching the store, so both the read and the coupled
write are skipped. Only successes are cached; failures always reach the inner
verifier. On a gateway-initiated revoke/rotate/delete the dashboard admin
service calls `IApiKeyCacheInvalidator.Invalidate(keyId)`, evicting the cached
entry immediately. The short TTL is the backstop for out-of-band mutations
(a direct DB edit, or a revoke run by the separate `apikey` CLI process, whose
in-memory cache is not the running gateway's cache).
- **`CoalescingMarkApiKeyStore`** wraps `IApiKeyStore` and forwards at most one
`MarkUsed` write per key per `MxGateway:Security:ApiKeyLastUsedCoalesceSeconds`
(default 60 s), so even under a cache miss the `last_used_utc` write is bounded
to roughly one per key per minute rather than one per RPC. `last_used_utc` is a
coarse staleness hint, not an audit record (audit rows are written separately),
so bounded staleness of up to one window is acceptable.
`GatewayApiKeyIdentityMapper` additionally memoizes the constraints-JSON
deserialization by blob, so the per-call parse on the mapped identity collapses to
a dictionary lookup. Both windows are configurable and may be set to `0` to
disable the respective mechanism; see [GatewayConfiguration](./GatewayConfiguration.md).
## Storage
The gateway keeps API key state in a dedicated SQLite database. SQLite is sufficient because credential volume is small, the gateway runs as a single process, and the file is straightforward to back up and rotate independently of the main application data.
The database path is `GatewayOptions.Authentication.SqlitePath`. Its code default is derived from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)` (`C:\ProgramData\MxGateway\gateway-auth.db` on Windows, `/usr/share/MxGateway/gateway-auth.db` or the container equivalent elsewhere) so the credential store is never written relative to the launch working directory on a non-Windows host. The production hosts pin the explicit Windows path in `appsettings.json`. `GatewayOptionsValidator` rejects a non-rooted (relative) `SqlitePath` so a bad override fails fast at startup rather than scattering the store by launch CWD (SEC-01).
### Connection factory
`AuthSqliteConnectionFactory` reads `GatewayOptions.Authentication.SqlitePath`, ensures the parent directory exists, and builds a connection string in `ReadWriteCreate` mode so first-run installations can create the file without manual provisioning. Connection pooling is enabled and the connection string carries a non-zero `DefaultTimeout`:
@@ -159,6 +193,8 @@ public static ApiKeyRecord Read(SqliteDataReader reader)
Because `RotateAsync` clears `revoked_utc`, rotating a previously revoked key reactivates it. The dashboard API Keys page therefore offers the Rotate (and Revoke) actions only for keys whose status is `Active`; revoked keys instead show a Delete action that calls `DeleteAsync`, so an operator can permanently remove a revoked row without ever risking un-revocation as a side effect of a rotation.
The dashboard API Keys page also surfaces expiry: each row shows an `Expires` column (`Never` when unset) and a status badge that reads `Expired` (past expiry, red), `Expiring` (within seven days, amber), `Revoked`, or `Active`. This is display-only staleness surfacing; expiry is set at creation time via the `apikey create-key --expires` CLI, not from the dashboard.
### Audit trail
`SqliteApiKeyAuditStore` (`IApiKeyAuditStore`) appends `ApiKeyAuditEntry` values to the `api_key_audit` table and stamps each row with a UTC timestamp inside the store rather than trusting the caller. `ListRecentAsync` returns the most recent rows ordered by `audit_id` descending and projects them into `ApiKeyAuditRecord`. Rows are kept even after the referenced key is revoked because the audit history is the durable record of administrative action; the `key_id` column is nullable to accommodate non-key-scoped events such as `init-db`.
@@ -192,8 +228,8 @@ The supported subcommands match `ApiKeyAdminCommandKind` exactly:
| Subcommand | Required options | Behaviour |
|------------|------------------|-----------|
| `init-db` | none | Runs the migrator and records an audit entry. |
| `create-key` | `--key-id`, `--display-name` | Generates a new secret, stores its peppered hash and optional constraints, and prints the assembled `mxgw_<keyId>_<secret>` token. |
| `list-keys` | none | Lists every stored key with its scopes, constraints, and revocation state. |
| `create-key` | `--key-id`, `--display-name` | Generates a new secret, stores its peppered hash and optional constraints, and prints the assembled `mxgw_<keyId>_<secret>` token. Optional `--expires` sets an expiry (absolute ISO-8601 UTC, or a relative `<N>d`/`<N>h` from now); omit it for a non-expiring key. |
| `list-keys` | none | Lists every stored key with its scopes, constraints, revocation state, and expiry (`active`/`expired`/`revoked`). |
| `revoke-key` | `--key-id` | Sets `revoked_utc` if the key is currently active. |
| `rotate-key` | `--key-id` | Replaces the secret hash and prints the new token. |
@@ -203,6 +239,8 @@ Examples:
mxgateway apikey init-db
mxgateway apikey create-key --key-id ops.alice --display-name "Alice (ops)" --scopes read,write
mxgateway apikey create-key --key-id area1.reader --display-name "Area 1 reader" --scopes invoke:read,metadata:read --read-subtree "Area1/*" --browse-subtree "Area1/*"
mxgateway apikey create-key --key-id ops.temp --display-name "Temp contractor" --scopes invoke:read --expires 90d
mxgateway apikey create-key --key-id ops.audit --display-name "Audit window" --scopes metadata:read --expires 2027-01-01T00:00:00Z
mxgateway apikey list-keys --json
mxgateway apikey revoke-key --key-id ops.alice
mxgateway apikey rotate-key --key-id ops.alice
+9 -2
View File
@@ -89,6 +89,12 @@ The flow is:
The status codes are deliberately distinct: `Unauthenticated` signals "we do not know who you are," and `PermissionDenied` signals "we know who you are, but you cannot do this." Treating the two as the same code would make troubleshooting harder for client implementations.
### Rate limiting the auth surface (SEC-11)
Before the verification store read, the helper checks a cheap in-process per-peer failure counter (`ApiKeyFailureLimiter`). A peer that has accumulated more than `MxGateway:Security:ApiKeyFailureLimit` failed attempts inside the sliding `ApiKeyFailureWindowSeconds` window is short-circuited with `StatusCode.ResourceExhausted` — so online guessing of API-key secrets cannot spend a SQLite read (and, in a naive design, a cache miss) per attempt. The peer is keyed on the presented key id where the token parses, falling back to the transport peer address; keying on key id throttles a single abusive credential without penalizing co-located clients behind a shared NAT. A successful verification resets the peer's counter. The counter is a bounded LRU (`ApiKeyFailureTrackedPeers`) so it cannot grow without limit. `ResourceExhausted` reveals only that throttling is in effect, not whether any particular secret was valid, preserving the opaque-failure property.
The dashboard login surface is throttled independently: `POST /auth/login` carries a fixed-window ASP.NET Core rate-limiter policy keyed per remote IP (`MxGateway:Security:LoginRateLimit*`), rejecting a burst with HTTP 429 before the LDAP bind is relayed to the directory. See [GatewayConfiguration](./GatewayConfiguration.md#security-options).
## Scope Resolution
`GatewayGrpcScopeResolver` is a stateless singleton that switches on the runtime request type. Top-level RPC requests map directly:
@@ -104,6 +110,7 @@ public string ResolveRequiredScope(object request)
MxCommandRequest commandRequest => ResolveCommandScope(commandRequest.Command?.Kind ?? MxCommandKind.Unspecified),
AcknowledgeAlarmRequest => GatewayScopes.InvokeWrite,
StreamAlarmsRequest => GatewayScopes.EventsRead,
QueryActiveAlarmsRequest => GatewayScopes.EventsRead,
TestConnectionRequest or
GetLastDeployTimeRequest or
DiscoverHierarchyRequest or
@@ -113,7 +120,7 @@ public string ResolveRequiredScope(object request)
}
```
The `_ => GatewayScopes.Admin` fallback is intentional: any future request type that the resolver does not recognize fails closed, requiring the strongest scope until the resolver is updated. `AcknowledgeAlarm` is treated as a write — it mutates alarm state, mirroring `MxCommandKind.Write*` — and `StreamAlarms` shares the alarm/event surface with `StreamEvents` and `MxCommandKind.DrainEvents`, so it carries `events:read`. Both alarm RPCs are session-less: the scope check is the only authorization gate, since there is no per-session ownership to enforce.
The `_ => GatewayScopes.Admin` fallback is intentional: any future request type that the resolver does not recognize fails closed, requiring the strongest scope until the resolver is updated. `AcknowledgeAlarm` is treated as a write — it mutates alarm state, mirroring `MxCommandKind.Write*` — and `StreamAlarms` and `QueryActiveAlarms` share the alarm/event surface with `StreamEvents` and `MxCommandKind.DrainEvents`, so they carry `events:read` (the active-alarm snapshot is the same data reachable through the event surface). All three alarm RPCs are session-less: the scope check is the only authorization gate, since there is no per-session ownership to enforce.
`MxCommandRequest` is special because it multiplexes many MxAccess operations through a single RPC. The resolver inspects the embedded `MxCommandKind` so each operation gets its own scope:
@@ -205,7 +212,7 @@ blocking constraint; secured values and raw credentials are never logged.
|----------|-------|--------------|
| `SessionOpen` | `session:open` | `OpenSessionRequest` |
| `SessionClose` | `session:close` | `CloseSessionRequest` |
| `EventsRead` | `events:read` | `StreamEventsRequest`, `StreamAlarmsRequest`, `MxCommandKind.DrainEvents` |
| `EventsRead` | `events:read` | `StreamEventsRequest`, `StreamAlarmsRequest`, `QueryActiveAlarmsRequest`, `MxCommandKind.DrainEvents` |
| `InvokeRead` | `invoke:read` | `MxCommandRequest` for read-style command kinds (`Register`, `AddItem`, `Advise`, `ReadBulk`, and any kind not otherwise mapped) |
| `InvokeWrite` | `invoke:write` | `AcknowledgeAlarmRequest`, `MxCommandKind.Write`, `MxCommandKind.Write2`, `MxCommandKind.WriteBulk`, `MxCommandKind.Write2Bulk` |
| `InvokeSecure` | `invoke:secure` | `MxCommandKind.WriteSecured`, `MxCommandKind.WriteSecured2`, `MxCommandKind.WriteSecuredBulk`, `MxCommandKind.WriteSecured2Bulk`, `MxCommandKind.AuthenticateUser` |
+36 -6
View File
@@ -48,25 +48,55 @@ session.
## Descriptor Publishing
Run this command after changing either source `.proto` file or the client proto
manifest:
manifest, with the **pinned protoc 34.1** (see [Toolchain Links](./ToolchainLinks.md)):
```powershell
scripts/publish-client-proto-inputs.ps1
pwsh -File scripts/publish-client-proto-inputs.ps1
```
The script writes
`clients/proto/descriptors/mxaccessgw-client-v1.protoset` with imports and
source information included. The descriptor is a generated artifact; do not edit
it by hand.
it by hand. Generating the committed artifact requires the pinned protoc version
so the checked-in bytes are reproducible; the script fails fast on a version
mismatch when generating.
Use the check mode in CI or before committing:
```powershell
scripts/publish-client-proto-inputs.ps1 -Check
pwsh -File scripts/publish-client-proto-inputs.ps1 -Check
```
`-Check` rebuilds the descriptor in a temporary path and fails when the checked
in descriptor is stale.
`-Check` rebuilds the descriptor and fails when the checked-in descriptor is
stale. The comparison normalizes both the committed and freshly built descriptor
through the same protoc with `source_code_info` stripped, so it is tolerant of
protoc-version encoding drift and does not false-fail across protoc releases
(it warns, rather than fails, when protoc is off the pin).
The gateway test project carries an independent, protoc-free freshness guard:
`ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField` reflects
over the in-process contract descriptors and fails if any contract message or
field is missing from the committed protoset. This is the primary CI gate for
descriptor staleness; a red test means "regenerate and commit the protoset."
### Pinned generator versions
Regeneration is reproducible only with the pinned toolchain. Regenerating with a
different version silently produces incompatible or noisy output, so the per-client
scripts assert the pin and resolve tools from `PATH`:
| Generator | Pinned version | Guard |
|-----------|----------------|-------|
| protoc (descriptor set) | 34.1 | version assertion in `scripts/publish-client-proto-inputs.ps1` |
| `Grpc.Tools` (C# `Generated/`) | 2.80.0 (contracts csproj) | `scripts/check-codegen.ps1` git-diff of `Generated/` |
| `grpcio-tools` (Python) | 1.80.0 (protobuf runtime 6.31.1) | version assertion in `clients/python/generate-proto.ps1` |
| protobuf / grpc-java (Java) | `protobufVersion` / `grpcVersion` in `clients/java/build.gradle` | `checkGeneratedClean` gradle task |
A newer `grpcio-tools` stamps a `GRPC_GENERATED_VERSION` above the pinned grpcio
runtime and breaks Python `pytest`; the Java protobuf plugin rewrites
`MxaccessGateway.java` with spurious protobuf-runtime-version churn on every build
(revert that one file when no `.proto` changed — see
[Gateway Testing](./GatewayTesting.md) "Continuous Integration").
## Output Directories
+33 -2
View File
@@ -97,6 +97,24 @@ behavior.
Generated C# output is written to `src/ZB.MOM.WW.MxGateway.Contracts/Generated/`. Do not
hand-edit generated files.
The `Generated/` C# is **tracked and must be committed after any `.proto` change.** The
contracts project `Compile Remove`s `Generated/**/*.cs` and has `Grpc.Tools` regenerate them,
but `Grpc.Tools` skips regeneration when the committed output looks up to date. On the net10
build the freshly regenerated code is compiled, so a stale `Generated/` is invisible there —
but the **net48 worker** consumes the committed `Generated/` and fails to build with `CS0246`
on any new type when the checked-in code lags the `.proto`. So after editing a `.proto` you must
regenerate and commit `Generated/`. If a build does not pick up your proto change, delete the
stale output to force a full regeneration:
```bash
rm src/ZB.MOM.WW.MxGateway.Contracts/Generated/*.cs
dotnet build src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj
```
`scripts/check-codegen.ps1` enforces this in CI (it force-regenerates and fails on any
`git diff` against the committed `Generated/`); the Windows CI job's net48 worker build is the
definitive guard.
Client generation inputs are published through
`clients/proto/proto-inputs.json` and the descriptor set under
`clients/proto/descriptors/`. See
@@ -124,12 +142,25 @@ gateway and test projects:
dotnet build src/ZB.MOM.WW.MxGateway.slnx
```
Regenerate the client descriptor after changing either `.proto` file:
Regenerate the client descriptor after changing either `.proto` file, using the pinned protoc
(34.1, see [Toolchain Links](./ToolchainLinks.md)):
```bash
powershell -ExecutionPolicy Bypass -File scripts/publish-client-proto-inputs.ps1
pwsh -File scripts/publish-client-proto-inputs.ps1
```
Freshness is guarded two ways so a skipped regeneration cannot ship silently:
- `ClientProtoInputTests.Descriptor_ContainsEveryContractMessageAndField` (gateway test project)
reflects over the in-process contract descriptors and fails if any message or field is missing
from the committed protoset. It is semantic (symbol presence), needs no protoc, and runs in the
Linux CI. A red test means "regenerate and commit the protoset."
- `pwsh -File scripts/publish-client-proto-inputs.ps1 -Check` rebuilds the descriptor and compares
it to the committed one. The comparison normalizes both sides through the same protoc with
`source_code_info` stripped, so it does not false-fail across protoc releases.
- `pwsh scripts/check-codegen.ps1` runs both the descriptor check and the `Generated/`-clean check
together; it is the single guard the CI pipeline invokes.
## Related Documentation
- [Client Proto Generation](./ClientProtoGeneration.md)
+63 -10
View File
@@ -29,7 +29,7 @@ paths, timeouts, queue sizes, enum values, or protocol values are invalid.
"ShutdownTimeoutSeconds": 10,
"HeartbeatIntervalSeconds": 5,
"HeartbeatGraceSeconds": 15,
"MaxMessageBytes": 16777216
"MaxMessageBytes": 16842752
},
"Sessions": {
"DefaultCommandTimeoutSeconds": 30,
@@ -93,12 +93,15 @@ Environment variables use the normal .NET double-underscore form. For example,
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Authentication:Mode` | `ApiKey` | Selects public gRPC authentication. Supported values are `ApiKey` and `Disabled`. `Disabled` bypasses API-key verification and is for local development only. |
| `MxGateway:Authentication:SqlitePath` | `C:\ProgramData\MxGateway\gateway-auth.db` | SQLite database path for API-key records and audit rows when API-key authentication is enabled. |
| `MxGateway:Authentication:SqlitePath` | derived from `CommonApplicationData` (`C:\ProgramData\MxGateway\gateway-auth.db` on Windows, `/usr/share/MxGateway/gateway-auth.db` or the container equivalent elsewhere) | SQLite database path for API-key records and audit rows when API-key authentication is enabled. The code default is built from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)` so the credential store never lands in the launch working directory on a non-Windows host; the production hosts pin the explicit Windows path in `appsettings.json`, which overrides the code default. |
| `MxGateway:Authentication:PepperSecretName` | `MxGateway:ApiKeyPepper` | Configuration key used to read the HMAC pepper for API-key secret hashing. The dashboard effective configuration redacts this value. |
| `MxGateway:Authentication:RunMigrationsOnStartup` | `true` | Runs SQLite auth schema migrations at gateway startup when API-key authentication is enabled. |
When `Mode` is `ApiKey`, `SqlitePath` and `PepperSecretName` must be present.
`SqlitePath` must be a valid filesystem path.
`SqlitePath` must be a valid filesystem path and must be **rooted** (absolute):
the validator rejects a non-rooted path so a relative override cannot silently
resolve against the working directory and scatter the credential store by launch
CWD (SEC-01).
## Worker Options
@@ -114,12 +117,16 @@ When `Mode` is `ApiKey`, `SqlitePath` and `PepperSecretName` must be present.
| `MxGateway:Worker:ShutdownTimeoutSeconds` | `10` | Grace period for worker shutdown before the gateway treats shutdown as failed and may kill the worker process tree. |
| `MxGateway:Worker:HeartbeatIntervalSeconds` | `5` | Worker heartbeat send interval and gateway heartbeat check cadence input. |
| `MxGateway:Worker:HeartbeatGraceSeconds` | `15` | Maximum age of the last worker heartbeat before the gateway faults the worker. This must be greater than or equal to `HeartbeatIntervalSeconds`. |
| `MxGateway:Worker:MaxMessageBytes` | `16777216` | Maximum worker IPC frame payload size in bytes. The validator allows values from `1024` through `268435456`. |
| `MxGateway:Worker:MaxMessageBytes` | `16842752` | Maximum worker IPC frame payload size in bytes. The validator allows values from `1024` through `268435456`, and additionally requires this to be at least `MxGateway:Protocol:MaxGrpcMessageBytes` plus a 65536-byte (64 KiB) envelope-overhead reserve. The default is the 16 MiB public gRPC cap plus that reserve. The gateway conveys the negotiated value to the worker in the handshake (`GatewayHello.max_frame_bytes`), so the worker frames to the same limit rather than a hard-coded default. |
`StartupProbeRetryAttempts`, `StartupProbeRetryDelayMilliseconds`,
`PipeConnectAttemptTimeoutMilliseconds`, timeout values, heartbeat values, and
`MaxMessageBytes` must be positive. `MaxMessageBytes` is intentionally bounded
to avoid accidental large allocations from malformed or oversized frames.
to avoid accidental large allocations from malformed or oversized frames. It
must also stay above `MaxGrpcMessageBytes` by the envelope-overhead reserve so a
maximally-sized accepted gRPC payload always fits one worker frame once wrapped
in a `WorkerEnvelope`; otherwise the oversized outbound write would fault the
whole session. Startup validation fails fast when the headroom is violated.
## Session Options
@@ -177,7 +184,7 @@ events (a "gap") and must re-snapshot; whatever is still retained is replayed.
| `MxGateway:Dashboard:RecentSessionLimit` | `200` | Maximum number of session summaries projected into each dashboard snapshot. |
| `MxGateway:Dashboard:ShowTagValues` | `false` | Reserved display control for tag values. The dashboard does not show full tag values by default. |
| `MxGateway:Dashboard:GroupToRole` | _(empty)_ | LDAP group → dashboard role mapping. Keys are LDAP group names (short CN or full DN — leading-RDN match). Values must be `Admin` (read/write, API-key CRUD) or `Viewer` (read-only). A user whose LDAP groups don't intersect this map cannot sign in; with no mapping at all, only the loopback bypass admits anyone. |
| `MxGateway:Dashboard:DisableLogin` | `false` | Dev/test only. When `true`, replaces the cookie authentication handler with `DashboardAutoLoginAuthenticationHandler`, which auto-authenticates every dashboard request — including requests from remote browsers, not just loopback — as `AutoLoginUser` holding both `Administrator` and `Viewer` roles. No login form, LDAP bind, or cookie is involved. A loud one-time startup warning is logged. Differs from `AllowAnonymousLocalhost`: `DisableLogin` mints a real authenticated principal (so role-gated write affordances appear), whereas `AllowAnonymousLocalhost` satisfies the authorization requirement on loopback only without minting a principal (write affordances stay hidden). Never enable in production. |
| `MxGateway:Dashboard:DisableLogin` | `false` | Dev/test only. When `true`, replaces the cookie authentication handler with `DashboardAutoLoginAuthenticationHandler`, which auto-authenticates every dashboard request — including requests from remote browsers, not just loopback — as `AutoLoginUser` holding both `Administrator` and `Viewer` roles. No login form, LDAP bind, or cookie is involved. A loud one-time startup warning is logged. Differs from `AllowAnonymousLocalhost`: `DisableLogin` mints a real authenticated principal (so role-gated write affordances appear), whereas `AllowAnonymousLocalhost` satisfies the authorization requirement on loopback only without minting a principal (write affordances stay hidden). Never enable in production. **Production hard-stop (SEC-04):** when the host runs in the `Production` environment and `DisableLogin` is `true`, startup validation fails and the process aborts — the flag is only accepted outside Production, where the one-time startup warning still fires. |
| `MxGateway:Dashboard:AutoLoginUser` | `(null)` | Username stamped on the synthetic principal when `DisableLogin` is `true`. Default `(null)` — a null or blank value falls back to `multi-role`. Has no effect when `DisableLogin` is `false`. |
`SnapshotIntervalMilliseconds` must be greater than zero. `RecentFaultLimit`
@@ -215,21 +222,67 @@ When the dashboard is enabled, three hubs are mapped under `/hubs/*`:
side-effect; the dashboard only sees events while a gRPC client is also
subscribed to that session.
`GET /hubs/token` (cookie-only) mints a 30-minute data-protected bearer
`GET /hubs/token` (cookie-only) mints a 5-minute data-protected bearer
token for the calling user; the Blazor pages use it via
`DashboardHubConnectionFactory` to authenticate the SignalR connection.
The factory refreshes the token on every (re)connect, so the short lifetime
(SEC-05) is transparent to clients. The token is not server-side revocable;
its short lifetime bounds exposure of a captured token (see
[GatewayDashboardDesign](./GatewayDashboardDesign.md)).
## LDAP Options (Dashboard Login)
The `MxGateway:Ldap` section configures the LDAP bind used by dashboard `/login`
(the gRPC API-key model is separate). The shipped defaults are the shared
dev/test GLAuth posture (`glauth.md`), not a production posture.
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Ldap:Enabled` | `true` | Enables LDAP-backed dashboard login. |
| `MxGateway:Ldap:Server` | `localhost` | LDAP server host. Dev points at the shared GLAuth instance (`10.100.0.35`). |
| `MxGateway:Ldap:Port` | `3893` | LDAP server port. Must be between `1` and `65535`. |
| `MxGateway:Ldap:Transport` | `None` | Connection transport: `None` (plaintext), `StartTls` (upgrade), or `Ldaps` (implicit TLS). The dev default is `None` because the shared GLAuth instance has LDAPS disabled and binds send cleartext. **Production hard-stop (SEC-06):** when the host runs in the `Production` environment, `Transport` must not be `None` — startup validation fails otherwise. Deployed hosts must set `Ldaps` or `StartTls`. |
| `MxGateway:Ldap:AllowInsecure` | `true` | Permits a plaintext bind. Must be `true` when `Transport` is `None`; set `false` (with `Ldaps`/`StartTls`) in production. |
| `MxGateway:Ldap:SearchBase` | `dc=zb,dc=local` | Search base DN. |
| `MxGateway:Ldap:ServiceAccountDn` | `cn=serviceaccount,dc=zb,dc=local` | Bind DN for the search account. |
| `MxGateway:Ldap:ServiceAccountPassword` | `serviceaccount123` | Search-account password. **Dev-only committed value.** On the deployed hosts, do not ship this in `appsettings.json`; supply it out-of-band via the env-var override `MxGateway__Ldap__ServiceAccountPassword` (double-underscore form, as set on the NSSM-wrapped services). The committed `serviceaccount123` is a shared dev credential for the local GLAuth instance and should be rotated; production must use a secret it does not share with dev. |
| `MxGateway:Ldap:UserNameAttribute` | `cn` | LDAP attribute holding the login user name. |
| `MxGateway:Ldap:DisplayNameAttribute` | `cn` | LDAP attribute holding the display name. |
| `MxGateway:Ldap:GroupAttribute` | `memberOf` | LDAP attribute enumerating group membership (mapped to dashboard roles via `MxGateway:Dashboard:GroupToRole`). |
When LDAP is enabled, `Server`, `SearchBase`, `ServiceAccountDn`,
`ServiceAccountPassword`, and the attribute names must be non-blank, and `Port`
must be in range. See `glauth.md` for the shared dev instance and the
dev→production hardening posture.
## Protocol Options
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Protocol:WorkerProtocolVersion` | `1` | Worker IPC protocol version expected by the gateway and worker. This must match `GatewayContractInfo.WorkerProtocolVersion`. |
| `MxGateway:Protocol:MaxGrpcMessageBytes` | `16777216` | Public gRPC max send and receive message size in bytes. The same default is used by official clients. The validator allows values from `1024` through `268435456`. |
| `MxGateway:Protocol:MaxGrpcMessageBytes` | `16777216` | Public gRPC max send and receive message size in bytes. The same default is used by official clients. The validator allows values from `1024` through `268435456`, and requires `MxGateway:Worker:MaxMessageBytes` to exceed this by at least the 64 KiB worker-frame envelope reserve (see the Worker options). |
The protocol option is exposed for diagnostics and explicit deployment
configuration, not for compatibility negotiation. A mismatch fails validation
at startup.
## Security Options
Bound from `MxGateway:Security`. These knobs tune the authentication hot path
(SEC-08 verification cache / last-used coalescing) and the auth-surface rate
limiting (SEC-11). Defaults are safe; leave them unless profiling or a threat
model requires otherwise.
| Option | Default | Description |
|--------|---------|-------------|
| `MxGateway:Security:ApiKeyVerificationCacheSeconds` | `15` | TTL, in seconds, of a cached successful API-key verification. A cache hit within this window skips the per-call SQLite read (and the coupled `last_used` write). Gateway-initiated revoke/rotate/delete invalidate the entry immediately; this TTL is the backstop for out-of-band mutations (a direct DB edit, or a revoke run by the separate `apikey` CLI process). `0` disables the cache. Must be zero or greater. |
| `MxGateway:Security:ApiKeyLastUsedCoalesceSeconds` | `60` | Coalescing window, in seconds, for the `last_used_utc` write. The library verifier writes `last_used` on every successful verification; this bounds the write to at most one per key per window, so a hammered key does not churn the WAL. `0` forwards every write. Must be zero or greater. |
| `MxGateway:Security:LoginRateLimitPermitLimit` | `10` | Maximum `POST /auth/login` attempts permitted per remote IP within `LoginRateLimitWindowSeconds` before requests are rejected with HTTP 429. Throttles LDAP credential stuffing before the bind is relayed to the directory. Must be greater than zero. |
| `MxGateway:Security:LoginRateLimitWindowSeconds` | `60` | Fixed-window length, in seconds, for the per-IP login rate limit. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureLimit` | `10` | Failed API-key verifications, per peer, within `ApiKeyFailureWindowSeconds` that trip the in-process short-circuit. Once tripped, the gRPC auth path rejects further attempts with `ResourceExhausted` **before** the store read; a successful verification resets the peer's counter. The peer is keyed on the presented key id (falling back to the transport address) so a single abusive credential behind a shared NAT is throttled without locking out co-located clients. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureWindowSeconds` | `60` | Sliding-window length, in seconds, over which API-key verification failures are counted per peer. Must be greater than zero. |
| `MxGateway:Security:ApiKeyFailureTrackedPeers` | `4096` | Maximum distinct peers tracked by the failure counter (a bounded LRU) so a spray of unique peer keys cannot grow memory without limit. Must be greater than zero. |
## Galaxy Options
| Option | Default | Description |
@@ -238,7 +291,7 @@ at startup.
| `MxGateway:Galaxy:CommandTimeoutSeconds` | `60` | Per-command SQL timeout for all Galaxy browse RPCs. |
| `MxGateway:Galaxy:DashboardRefreshIntervalSeconds` | `30` | Interval between background refreshes of the dashboard Galaxy summary cache. SQL is hit at most once per interval regardless of dashboard render rate. |
| `MxGateway:Galaxy:PersistSnapshot` | `true` | Persists the latest successful Galaxy browse dataset to disk. When `true`, the cache reloads that snapshot at startup so clients can still browse last-known data while the Galaxy database is unreachable. The restored data is served with `Stale` status until a live query confirms it. |
| `MxGateway:Galaxy:SnapshotCachePath` | `C:\ProgramData\MxGateway\galaxy-snapshot.json` | File path for the persisted Galaxy browse snapshot. Ignored when `PersistSnapshot` is `false`. The snapshot is written atomically (temp file plus rename). |
| `MxGateway:Galaxy:SnapshotCachePath` | `C:\ProgramData\MxGateway\galaxy-snapshot.json` | File path for the persisted Galaxy browse snapshot. Ignored when `PersistSnapshot` is `false`. The snapshot is written atomically (temp file plus rename). Set an **absolute** path — this option is bound by the shared `ZB.MOM.WW.GalaxyRepository` package (not by `GatewayOptions`), so the gateway validator does not enforce rooting on it; a relative value would resolve against the launch working directory (SEC-01). |
See [Galaxy Repository Browse](./GalaxyRepository.md) for the RPC surface and
behavior.
@@ -442,7 +495,7 @@ them.
| Option | Default | Purpose |
|---|---|---|
| `Tls:SelfSignedCertPath` | `C:\ProgramData\MxGateway\certs\gateway-selfsigned.pfx` | Where the generated certificate is persisted |
| `Tls:SelfSignedCertPath` | derived from `CommonApplicationData` (`C:\ProgramData\MxGateway\certs\gateway-selfsigned.pfx` on Windows, `/usr/share/MxGateway/certs/gateway-selfsigned.pfx` or the container equivalent elsewhere) | Where the generated certificate is persisted. The code default is built from `Environment.GetFolderPath(SpecialFolder.CommonApplicationData)` so a generated private key never lands in the launch working directory on a non-Windows host (SEC-01). Must be **rooted** (absolute); the validator rejects a non-rooted override. |
| `Tls:ValidityYears` | `10` | Lifetime of the generated certificate (validated 1100) |
| `Tls:AdditionalDnsNames` | `[]` | Extra DNS SANs (e.g. a load-balancer name) |
| `Tls:RegenerateIfExpired` | `true` | Replace an expired persisted certificate instead of failing |
+36 -8
View File
@@ -257,6 +257,15 @@ gated by a confirmation dialog before it reaches `ISessionManager`.
is killed immediately with no graceful-shutdown attempt. The session is
removed from the registry and the open-session slot is released either way.
Both actions write a canonical `AuditEvent` through `IAuditWriter` into the
`audit_event` store (category `SessionAdmin`, actions `dashboard-close-session`
and `dashboard-kill-worker`) in addition to the operational `ILogger` line — so a
worker killed mid-production leaves a durable, queryable row rather than only a
rotatable log entry. The event records the LDAP actor, the session id (`Target`),
the remote address, and an outcome of `Success`, `Failure`, or `Denied`
(an unauthorized attempt is still audited as `Denied`). This mirrors the API-key
management audit path.
### Workers page
Show:
@@ -436,11 +445,16 @@ Three authorization policies are registered:
cookie OR a `MxGateway.Dashboard.HubToken` bearer (used by WebSocket upgrades
where the cookie can't be forwarded).
Two environmental bypasses still apply: `MxGateway:Authentication:Mode = Disabled`
authorizes every request, and `MxGateway:Dashboard:AllowAnonymousLocalhost`
(default `true`) authorizes any loopback request without a role check. Remote
requests always require an authenticated principal carrying at least the
Viewer role.
Two environmental bypasses still apply, both scoped to **read-only** access:
`MxGateway:Authentication:Mode = Disabled` and `MxGateway:Dashboard:AllowAnonymousLocalhost`
(default `true`, loopback only) each satisfy a requirement that includes the Viewer
role (`AnyDashboardRole`) but **never** the `AdminOnly` requirement. Destructive/admin
surfaces (API-key CRUD, session Close, worker Kill) still demand a real `Administrator`
role claim, so the "anonymous localhost is read-only" contract holds at the policy layer
rather than only in downstream service re-checks. Remote requests (outside the `Disabled`
bypass) always require an authenticated principal carrying at least the Viewer role. The
loopback test trusts `Connection.RemoteIpAddress`; if forwarded-headers middleware is ever
added upstream, `DashboardAuthorizationHandler.IsLoopbackRequest()` must be revisited.
### DisableLogin dev bypass
@@ -486,9 +500,13 @@ dashboard mints short-lived bearer tokens for the connection:
and role claims to JSON, encrypts with the ASP.NET Core data-protection
time-limited protector under purpose
`ZB.MOM.WW.MxGateway.Dashboard.HubToken.v1`, and returns the protected
string. Lifetime is 30 minutes.
string. Lifetime is **5 minutes**.
3. The SignalR client passes the token as either `Authorization: Bearer …` or
`?access_token=…` (WebSocket upgrade query string).
`?access_token=…` (WebSocket upgrade query string). The query-string form is
the standard SignalR carriage for WebSocket upgrades, which cannot attach a
custom header; because it is easy to capture (proxy logs, browser history),
it must never be request-logged (see the remarks on
`HubTokenAuthenticationHandler`).
4. `HubTokenAuthenticationHandler` validates the protected payload and
rebuilds the `ClaimsPrincipal` with the carried roles.
5. The hubs' `[Authorize(Policy = HubClientsPolicy)]` accepts the resulting
@@ -496,7 +514,17 @@ dashboard mints short-lived bearer tokens for the connection:
`DashboardHubConnectionFactory` (scoped to the Blazor circuit) wraps the
HubConnectionBuilder and supplies a fresh token via `AccessTokenProvider` on
every (re)connect.
every (re)connect, so the short 5-minute lifetime is transparent to clients.
Caveat — logout does not revoke outstanding tokens. Logout clears the dashboard
cookie, but hub bearer tokens are self-contained, data-protection-encrypted, and
carry no server-side revocation state (no jti denylist). A token captured before
logout remains valid until it expires, and a role change or key revocation does
not take effect on an already-issued token until then. The 5-minute lifetime is
the deliberate mitigation: it bounds that exposure window without the cost of a
revocation store. Server-side revocation is deferred until per-session hub ACLs
land (see the per-session-ACL note), at which point tokens gain session/role
binding and a denylist becomes worthwhile.
## Configuration
+18 -2
View File
@@ -481,7 +481,9 @@ Internally it owns:
- pipe stream,
- read loop,
- write loop,
- event write loop,
- outbound command/control channel serialized by the write loop,
- unbounded event staging channel drained by the event write loop,
- bounded inbound event channel,
- pending command dictionary keyed by correlation id,
- heartbeat monitor,
@@ -499,15 +501,29 @@ The read loop:
1. Reads one frame.
2. Parses `WorkerEnvelope`.
3. Validates protocol fields.
4. Dispatches by body type:
4. Dispatches by body type, **synchronously and without blocking**:
- `WorkerCommandReply`: completes pending command.
- `WorkerEvent`: enqueues event.
- `WorkerEvent`: hands the event to the event write loop via a non-blocking staging write.
- `WorkerHeartbeat`: updates heartbeat timestamp.
- `WorkerFault`: faults session.
5. Stops when pipe closes or cancellation is requested.
If the pipe closes while the session is not closing, fault the session.
The read loop never awaits event enqueue. Events are staged to the event write
loop with a non-blocking write, so a full inbound event channel (a slow or
absent `StreamEvents` consumer) cannot stall the read loop behind an event and
delay a command reply or heartbeat (GWC-04). The bounded-channel backpressure
window (`EventChannelFullModeTimeout`) and the sustained-overflow fault are
applied by the event write loop, not the read loop.
### Event write loop
The event write loop drains the staging channel and performs the timed write
into the bounded inbound event channel. When the inbound channel stays full past
`EventChannelFullModeTimeout` it faults the session (`ProtocolViolation`) — the
same overflow contract as before, moved off the read loop.
### Write loop
The write loop serializes all writes to the pipe. No other code should write to
+28
View File
@@ -408,6 +408,34 @@ Run the gateway test project after shared gateway test infrastructure changes:
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj
```
## Continuous Integration
CI runs on Gitea Actions (`.gitea/workflows/ci.yml`; origin is Gitea at
`gitea.dohertylan.com`) on every push and pull request. The pipeline is split by
runtime because the x86 Worker cannot build on Linux:
- **`portable`** (Linux runner) — builds `src/ZB.MOM.WW.MxGateway.NonWindows.slnx`,
runs the codegen/descriptor freshness guard (`scripts/check-codegen.ps1`), runs the
gateway fake-worker tests, and builds/tests the clients that run on Linux: .NET client
build, Go (`gofmt` + `go build` + `go test`), Rust (`cargo fmt --check` + `cargo test`
+ `cargo clippy -D warnings`), and Python (`pytest`).
- **`java`** (Linux, JDK 17) — `gradle test`. The protobuf gradle plugin rewrites
`MxaccessGateway.java` with spurious protobuf-runtime-version churn on every build, so
when no `.proto` changed the job reverts that one file (`git checkout`) before asserting
the generated tree is clean. The dev Mac has no JRE, so Java verification is CI-only.
- **`windows`** (self-hosted windev runner) — the only host that builds the **x86 /
net48 Worker and Worker.Tests**; these are Windows-only and out of scope for the Linux
jobs. This job also proves the net48 build against the committed `Generated/` (the
definitive guard for the "regenerate and commit `Generated/`" rule).
- **`live-mxaccess`** (self-hosted, MXAccess-installed) — scheduled only
(`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`); never gates a push.
The freshness guard `scripts/check-codegen.ps1` fails the build when the committed
client descriptor set or the C# `Generated/` no longer matches the current `.proto`
sources — the codegen drift class this repo has hit repeatedly (stale client
descriptors, net48 `CS0246` on unregenerated protos). See
[Client Proto Generation](./ClientProtoGeneration.md) and [Contracts](./Contracts.md).
## Related Documentation
- [Cross-Language Smoke Matrix](./CrossLanguageSmokeMatrix.md)
+13 -1
View File
@@ -44,7 +44,7 @@ All counters are `Counter<long>`. Tag values come from the call sites listed und
| `mxgateway.faults` | `category` | Faults reported by session, event, or worker code paths. The category is a `SessionManagerErrorCode` or `WorkerClientErrorCode` name. |
| `mxgateway.workers.killed` | `reason` | Forced terminations of worker processes. |
| `mxgateway.workers.exited` | `reason` | Clean or fault-driven worker exits. |
| `mxgateway.heartbeats.failed` | `session_id` | Worker heartbeat misses tracked per session. |
| `mxgateway.heartbeats.failed` | *(none)* | Aggregate worker heartbeat misses. The exported counter carries **no** `session_id` tag — per-session attribution is unbounded cardinality (every session mints a new time series), so it is kept out of the exporter; use the dashboard snapshot or the log scope for per-session detail. |
| `mxgateway.grpc.streams.disconnected` | `reason` | Detachments of the dashboard or client gRPC event stream. |
| `mxgateway.retries.attempted` | `area` | Resilience retries executed by gateway components. |
@@ -203,6 +203,18 @@ metrics.RecordEventStreamSend(publicEvent.Family.ToString(), stopwatch.Elapsed);
`Dashboard/DashboardSnapshotService.cs` calls `_metrics.GetSnapshot()` once per `GetSnapshot` invocation and projects it into the dashboard transport types together with the session registry view. The dashboard receives a single, internally consistent snapshot per tick rather than reading individual counters at separate times. See [Gateway Dashboard Design](./GatewayDashboardDesign.md) and [Dashboard Interface Design](./DashboardInterfaceDesign.md) for the projection rules and wire format.
## Auth Store Write Churn
The per-RPC authentication path is a hidden write source: the shared verifier
refreshes `last_used_utc` on every authenticated call. To keep that off the
auth-store throughput ceiling, the gateway caches successful verifications for a
short TTL and coalesces the `last_used_utc` write to at most one per key per
window (`MxGateway:Security:ApiKeyVerificationCacheSeconds` /
`ApiKeyLastUsedCoalesceSeconds`, defaults 15 s / 60 s). This bounds WAL churn on
the bulk-read workload without dropping any metric — auth activity is still
observable via command-throughput counters and audit rows. See
[Authentication](./Authentication.md#hot-path-caching-and-last-used-coalescing).
## Related Documentation
- [Gateway Dashboard Design](./GatewayDashboardDesign.md)
+11 -2
View File
@@ -17,8 +17,17 @@ payload_length bytes protobuf WorkerEnvelope
```
The reader rejects zero-length payloads and payloads larger than the configured
maximum before allocating the payload buffer. The default maximum is 16 MiB,
matching the gateway process design.
maximum before allocating the payload buffer. The default maximum is the 16 MiB
public gRPC cap plus a 64 KiB envelope-overhead reserve (16842752 bytes) so a
maximally-sized accepted gRPC payload always fits one worker frame once wrapped
in a `WorkerEnvelope`.
The gateway is the source of truth for this maximum: it conveys the negotiated
value in the handshake as `GatewayHello.max_frame_bytes`, and the worker adopts
it as its `WorkerFrameProtocolOptions.MaxMessageBytes` instead of a hard-coded
default. A `max_frame_bytes` of 0 (an older gateway that never set the field)
means "use the worker's built-in default". This keeps both ends framing to the
same limit rather than depending on matched compile-time constants.
## Envelope Validation
+4 -1
View File
@@ -424,7 +424,10 @@ Optional diagnostics:
- `Ping`
- `GetSessionState`
- `GetWorkerInfo`
- `DrainEvents`
- `DrainEvents` — diagnostic; `max_events` is bounded (the gateway rejects requests
above a public ceiling, and the worker caps each reply at its own per-reply limit,
treating `max_events = 0` as "the default cap") so one drain cannot pack an
unbounded, session-killing reply frame.
- `ShutdownWorker`
Do not compress MXAccess semantics into generic verbs too early. A command enum
+96
View File
@@ -0,0 +1,96 @@
#!/usr/bin/env pwsh
# Codegen freshness guard for CI (IPC-01, IPC-19, IPC-20, CLI-02).
#
# Three checks, all Linux/macOS-runnable (no Server build, no x86 worker):
# 1. Published client descriptor set matches the current .proto sources (delegates to
# publish-client-proto-inputs.ps1 -Check, which normalizes source_code_info so it is
# protoc-version tolerant).
# 2. The committed C# under Contracts/Generated matches a fresh regeneration. Grpc.Tools is
# pinned in the contracts csproj, so a clean checkout regenerates byte-identical output; a
# non-empty git diff means a .proto was edited without regenerating and committing Generated/
# (which breaks the net48 worker build with CS0246 — see docs/Contracts.md).
# 3. The Rust crate's vendored protos (clients/rust/protos/*.proto — build inputs that make the
# crate buildable outside the repo, CLI-02) are byte-identical to the canonical Contracts
# protos. A drift means a .proto was edited without refreshing the vendored copies, which would
# publish a stale wire contract to crate consumers while the in-repo build stays correct.
#
# The x86 Worker + Worker.Tests are Windows-only and are guarded by the Windows CI job, not here.
[CmdletBinding()]
param()
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$repoRoot = Resolve-Path (Join-Path $PSScriptRoot '..')
$generatedDir = Join-Path $repoRoot 'src/ZB.MOM.WW.MxGateway.Contracts/Generated'
$contractsProject = Join-Path $repoRoot 'src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj'
$failures = New-Object System.Collections.Generic.List[string]
Write-Host '== Check 1/2: client descriptor set freshness =='
try {
& (Join-Path $PSScriptRoot 'publish-client-proto-inputs.ps1') -Check
if ($LASTEXITCODE -ne 0) {
$failures.Add('Client descriptor set is stale (publish-client-proto-inputs.ps1 -Check failed).')
}
}
catch {
$failures.Add("Descriptor freshness check failed: $($_.Exception.Message)")
}
Write-Host ''
Write-Host '== Check 2/2: Contracts/Generated matches a fresh regeneration =='
try {
# Force a full regeneration: Grpc.Tools skips regen when the committed .cs look up to date, so
# remove them first (the documented "del Generated/*.cs to force regen" trick).
if (Test-Path $generatedDir) {
Get-ChildItem -Path $generatedDir -Filter '*.cs' -Recurse -File | Remove-Item -Force
}
& dotnet build $contractsProject -c Release --nologo | Out-Host
if ($LASTEXITCODE -ne 0) {
$failures.Add('Contracts project failed to build during codegen check.')
}
$diff = (& git -C $repoRoot status --porcelain -- 'src/ZB.MOM.WW.MxGateway.Contracts/Generated').Trim()
if (-not [string]::IsNullOrEmpty($diff)) {
Write-Host $diff
$failures.Add('Contracts/Generated differs from a fresh regeneration. Run `dotnet build` on the contracts project and commit Generated/ (required for the net48 worker build).')
}
}
catch {
$failures.Add("Generated codegen check failed: $($_.Exception.Message)")
}
Write-Host ''
Write-Host '== Check 3/3: Rust vendored protos match canonical Contracts protos =='
try {
$canonicalProtoDir = Join-Path $repoRoot 'src/ZB.MOM.WW.MxGateway.Contracts/Protos'
$vendoredProtoDir = Join-Path $repoRoot 'clients/rust/protos'
foreach ($vendored in Get-ChildItem -Path $vendoredProtoDir -Filter '*.proto' -File) {
$canonical = Join-Path $canonicalProtoDir $vendored.Name
if (-not (Test-Path $canonical)) {
$failures.Add("Vendored Rust proto has no canonical counterpart: $($vendored.Name).")
continue
}
$vendoredHash = (Get-FileHash -Algorithm SHA256 $vendored.FullName).Hash
$canonicalHash = (Get-FileHash -Algorithm SHA256 $canonical).Hash
if ($vendoredHash -ne $canonicalHash) {
$failures.Add("Rust vendored proto drifted from canonical: clients/rust/protos/$($vendored.Name). Refresh it from src/ZB.MOM.WW.MxGateway.Contracts/Protos/$($vendored.Name).")
}
}
}
catch {
$failures.Add("Rust vendored proto check failed: $($_.Exception.Message)")
}
Write-Host ''
if ($failures.Count -gt 0) {
Write-Host 'Codegen freshness check FAILED:' -ForegroundColor Red
foreach ($failure in $failures) {
Write-Host " - $failure" -ForegroundColor Red
}
exit 1
}
Write-Host 'Codegen freshness check passed.' -ForegroundColor Green
+4 -2
View File
@@ -187,7 +187,9 @@ function Invoke-PackRust {
}
Write-Host 'Running cargo package...'
& cargo package --no-verify
# No --no-verify: the crate vendors its protos (clients/rust/protos) so
# the verify build proves it compiles standalone from the tarball (CLI-02).
& cargo package
if ($LASTEXITCODE -ne 0) { throw 'cargo package failed.' }
$packageDir = Join-Path $rustDir 'target/package'
@@ -207,7 +209,7 @@ function Invoke-PackRust {
Write-Host 'Publishing Rust crate to Gitea...' -ForegroundColor Yellow
Push-Location (Join-Path $RepoRoot 'clients/rust')
try {
& cargo publish --no-verify --registry dohertj2-gitea
& cargo publish --registry dohertj2-gitea
if ($LASTEXITCODE -ne 0) { throw 'cargo publish failed.' }
} finally { Pop-Location }
}
+99 -18
View File
@@ -6,23 +6,46 @@ param(
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
# Pinned protoc toolchain version used to generate the committed client descriptor set.
# See docs/ToolchainLinks.md. Generating (committing) the canonical artifact must use this
# exact version so the checked-in bytes are reproducible. The -Check comparison is made
# tolerant of protoc-version encoding drift (chiefly source_code_info, see IPC-20) by
# normalizing both the committed and the freshly built descriptor through the *same* protoc
# with source info stripped, so it does not false-fail across protoc releases.
$PinnedProtocVersion = "34.1"
$repoRoot = Resolve-Path (Join-Path $PSScriptRoot "..")
$protoRoot = Join-Path $repoRoot "src/ZB.MOM.WW.MxGateway.Contracts/Protos"
$manifestPath = Join-Path $repoRoot "clients/proto/proto-inputs.json"
$descriptorPath = Join-Path $repoRoot "clients/proto/descriptors/mxaccessgw-client-v1.protoset"
$protoFiles = @("mxaccess_gateway.proto", "mxaccess_worker.proto", "galaxy_repository.proto")
function Resolve-Protoc {
$pathCommand = Get-Command "protoc.exe" -ErrorAction SilentlyContinue
# Prefer protoc on PATH (protoc on Linux/macOS, protoc.exe on Windows), then fall back to
# the documented winget install location on Windows.
foreach ($name in @("protoc", "protoc.exe")) {
$pathCommand = Get-Command $name -ErrorAction SilentlyContinue
if ($null -ne $pathCommand) {
return $pathCommand.Source
}
}
if ($env:LOCALAPPDATA) {
$documentedPath = Join-Path $env:LOCALAPPDATA "Microsoft/WinGet/Packages/Google.Protobuf_Microsoft.Winget.Source_8wekyb3d8bbwe/bin/protoc.exe"
if (Test-Path $documentedPath) {
return $documentedPath
}
}
throw "Could not find protoc.exe. See docs/ToolchainLinks.md for the documented protobuf toolchain path."
throw "Could not find protoc on PATH. See docs/ToolchainLinks.md for the documented protobuf toolchain (protoc $PinnedProtocVersion)."
}
function Get-ProtocVersion {
param([string]$Protoc)
# `protoc --version` prints e.g. "libprotoc 34.1".
$raw = (& $Protoc --version) | Select-Object -First 1
return ($raw -replace '^libprotoc\s+', '').Trim()
}
function Ensure-Directory {
@@ -33,7 +56,29 @@ function Ensure-Directory {
}
}
function Compare-FileBytes {
function New-SourceInfoFreeDescriptor {
# Re-emits an existing descriptor set with source_code_info stripped, by round-tripping it
# through protoc with --descriptor_set_in and *without* --include_source_info. Because both
# the committed and freshly built descriptors are normalized through the same protoc binary,
# any protoc-version-specific encoding cancels out and the byte comparison is stable.
param(
[string]$Protoc,
[string]$InputDescriptor,
[string]$OutputDescriptor
)
& $Protoc `
"--descriptor_set_in=$InputDescriptor" `
"--include_imports" `
"--descriptor_set_out=$OutputDescriptor" `
@protoFiles
if ($LASTEXITCODE -ne 0) {
throw "protoc normalization (--descriptor_set_in) failed with exit code $LASTEXITCODE."
}
}
function Test-FileBytesEqual {
param(
[string]$ExpectedPath,
[string]$ActualPath
@@ -66,31 +111,67 @@ foreach ($output in $manifest.generatedOutputs.PSObject.Properties.Value) {
}
$protoc = Resolve-Protoc
$outputPath = $descriptorPath
if ($Check) {
$outputPath = Join-Path ([System.IO.Path]::GetTempPath()) ("mxaccessgw-client-v1-" + [System.Guid]::NewGuid().ToString("N") + ".protoset")
}
$protocVersion = Get-ProtocVersion $protoc
try {
if ($Check) {
# Version drift no longer produces a false "stale" failure: the comparison is normalized
# below. A mismatch is surfaced as a warning so it is visible without blocking the gate.
if ($protocVersion -ne $PinnedProtocVersion) {
Write-Warning "protoc version '$protocVersion' differs from the pinned '$PinnedProtocVersion'. The freshness comparison is normalized and tolerant of this, but regeneration must use the pinned version (see docs/ClientProtoGeneration.md)."
}
$freshDescriptor = Join-Path ([System.IO.Path]::GetTempPath()) ("mxaccessgw-client-v1-fresh-" + [System.Guid]::NewGuid().ToString("N") + ".protoset")
$committedNormalized = Join-Path ([System.IO.Path]::GetTempPath()) ("mxaccessgw-client-v1-committed-" + [System.Guid]::NewGuid().ToString("N") + ".protoset")
try {
# Fresh descriptor from the current .proto sources, source info omitted (already normalized).
& $protoc `
"--proto_path=$protoRoot" `
"--include_imports" `
"--include_source_info" `
"--descriptor_set_out=$outputPath" `
"mxaccess_gateway.proto" `
"mxaccess_worker.proto" `
"galaxy_repository.proto"
"--descriptor_set_out=$freshDescriptor" `
@protoFiles
if ($LASTEXITCODE -ne 0) {
throw "protoc failed with exit code $LASTEXITCODE."
}
if ($Check -and -not (Compare-FileBytes -ExpectedPath $descriptorPath -ActualPath $outputPath)) {
throw "Client proto descriptor is stale. Run scripts/publish-client-proto-inputs.ps1 and commit the updated descriptor."
if (-not (Test-Path $descriptorPath)) {
throw "Committed descriptor '$descriptorPath' does not exist. Run scripts/publish-client-proto-inputs.ps1 to create it."
}
# Normalize the committed descriptor (which carries source info) through the same protoc.
New-SourceInfoFreeDescriptor -Protoc $protoc -InputDescriptor $descriptorPath -OutputDescriptor $committedNormalized
if (-not (Test-FileBytesEqual -ExpectedPath $committedNormalized -ActualPath $freshDescriptor)) {
throw "Client proto descriptor is stale. Run scripts/publish-client-proto-inputs.ps1 with the pinned protoc ($PinnedProtocVersion) and commit the updated descriptor."
}
Write-Host "Client proto descriptor is up to date."
}
finally {
foreach ($temp in @($freshDescriptor, $committedNormalized)) {
if (Test-Path $temp) {
Remove-Item -LiteralPath $temp
}
}
}
}
finally {
if ($Check -and (Test-Path $outputPath)) {
Remove-Item -LiteralPath $outputPath
else {
# Generating the canonical committed artifact must use the pinned protoc so the bytes are reproducible.
if ($protocVersion -ne $PinnedProtocVersion) {
throw "protoc version '$protocVersion' does not match the pinned toolchain version '$PinnedProtocVersion'. Install the pinned protoc (see docs/ToolchainLinks.md) before regenerating the committed descriptor, or run with -Check to only verify freshness."
}
& $protoc `
"--proto_path=$protoRoot" `
"--include_imports" `
"--include_source_info" `
"--descriptor_set_out=$descriptorPath" `
@protoFiles
if ($LASTEXITCODE -ne 0) {
throw "protoc failed with exit code $LASTEXITCODE."
}
Write-Host "Wrote client proto descriptor to $descriptorPath (protoc $protocVersion)."
}
@@ -44,63 +44,64 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
"Y2Nlc3Nfd29ya2VyLnYxLldvcmtlckV2ZW50SAASPwoQd29ya2VyX2hlYXJ0",
"YmVhdBgTIAEoCzIjLm14YWNjZXNzX3dvcmtlci52MS5Xb3JrZXJIZWFydGJl",
"YXRIABI3Cgx3b3JrZXJfZmF1bHQYFCABKAsyHy5teGFjY2Vzc193b3JrZXIu",
"djEuV29ya2VyRmF1bHRIAEIGCgRib2R5IloKDEdhdGV3YXlIZWxsbxIiChpz",
"djEuV29ya2VyRmF1bHRIAEIGCgRib2R5InMKDEdhdGV3YXlIZWxsbxIiChpz",
"dXBwb3J0ZWRfcHJvdG9jb2xfdmVyc2lvbhgBIAEoDRINCgVub25jZRgCIAEo",
"CRIXCg9nYXRld2F5X3ZlcnNpb24YAyABKAkiaQoLV29ya2VySGVsbG8SGAoQ",
"cHJvdG9jb2xfdmVyc2lvbhgBIAEoDRINCgVub25jZRgCIAEoCRIZChF3b3Jr",
"ZXJfcHJvY2Vzc19pZBgDIAEoBRIWCg53b3JrZXJfdmVyc2lvbhgEIAEoCSKO",
"AQoLV29ya2VyUmVhZHkSGQoRd29ya2VyX3Byb2Nlc3NfaWQYASABKAUSFwoP",
"bXhhY2Nlc3NfcHJvZ2lkGAIgASgJEhYKDm14YWNjZXNzX2Nsc2lkGAMgASgJ",
"EjMKD3JlYWR5X3RpbWVzdGFtcBgEIAEoCzIaLmdvb2dsZS5wcm90b2J1Zi5U",
"aW1lc3RhbXAidwoNV29ya2VyQ29tbWFuZBIvCgdjb21tYW5kGAEgASgLMh4u",
"bXhhY2Nlc3NfZ2F0ZXdheS52MS5NeENvbW1hbmQSNQoRZW5xdWV1ZV90aW1l",
"c3RhbXAYAiABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wIoEBChJX",
"b3JrZXJDb21tYW5kUmVwbHkSMgoFcmVwbHkYASABKAsyIy5teGFjY2Vzc19n",
"YXRld2F5LnYxLk14Q29tbWFuZFJlcGx5EjcKE2NvbXBsZXRlZF90aW1lc3Rh",
"bXAYAiABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wIh4KDFdvcmtl",
"ckNhbmNlbBIOCgZyZWFzb24YASABKAkiUQoOV29ya2VyU2h1dGRvd24SLwoM",
"Z3JhY2VfcGVyaW9kGAEgASgLMhkuZ29vZ2xlLnByb3RvYnVmLkR1cmF0aW9u",
"Eg4KBnJlYXNvbhgCIAEoCSJIChFXb3JrZXJTaHV0ZG93bkFjaxIzCgZzdGF0",
"dXMYASABKAsyIy5teGFjY2Vzc19nYXRld2F5LnYxLlByb3RvY29sU3RhdHVz",
"IjoKC1dvcmtlckV2ZW50EisKBWV2ZW50GAEgASgLMhwubXhhY2Nlc3NfZ2F0",
"ZXdheS52MS5NeEV2ZW50IqUCCg9Xb3JrZXJIZWFydGJlYXQSGQoRd29ya2Vy",
"X3Byb2Nlc3NfaWQYASABKAUSLgoFc3RhdGUYAiABKA4yHy5teGFjY2Vzc193",
"b3JrZXIudjEuV29ya2VyU3RhdGUSPwobbGFzdF9zdGFfYWN0aXZpdHlfdGlt",
"ZXN0YW1wGAMgASgLMhouZ29vZ2xlLnByb3RvYnVmLlRpbWVzdGFtcBIdChVw",
"ZW5kaW5nX2NvbW1hbmRfY291bnQYBCABKA0SIgoab3V0Ym91bmRfZXZlbnRf",
"cXVldWVfZGVwdGgYBSABKA0SGwoTbGFzdF9ldmVudF9zZXF1ZW5jZRgGIAEo",
"BBImCh5jdXJyZW50X2NvbW1hbmRfY29ycmVsYXRpb25faWQYByABKAki9AEK",
"C1dvcmtlckZhdWx0EjkKCGNhdGVnb3J5GAEgASgOMicubXhhY2Nlc3Nfd29y",
"a2VyLnYxLldvcmtlckZhdWx0Q2F0ZWdvcnkSFgoOY29tbWFuZF9tZXRob2QY",
"AiABKAkSFAoHaHJlc3VsdBgDIAEoBUgAiAEBEhYKDmV4Y2VwdGlvbl90eXBl",
"GAQgASgJEhoKEmRpYWdub3N0aWNfbWVzc2FnZRgFIAEoCRI8Cg9wcm90b2Nv",
"bF9zdGF0dXMYBiABKAsyIy5teGFjY2Vzc19nYXRld2F5LnYxLlByb3RvY29s",
"U3RhdHVzQgoKCF9ocmVzdWx0KpcCCgtXb3JrZXJTdGF0ZRIcChhXT1JLRVJf",
"U1RBVEVfVU5TUEVDSUZJRUQQABIZChVXT1JLRVJfU1RBVEVfU1RBUlRJTkcQ",
"ARIcChhXT1JLRVJfU1RBVEVfSEFORFNIQUtJTkcQAhIhCh1XT1JLRVJfU1RB",
"VEVfSU5JVElBTElaSU5HX1NUQRADEhYKEldPUktFUl9TVEFURV9SRUFEWRAE",
"EiIKHldPUktFUl9TVEFURV9FWEVDVVRJTkdfQ09NTUFORBAFEh4KGldPUktF",
"Ul9TVEFURV9TSFVUVElOR19ET1dOEAYSGAoUV09SS0VSX1NUQVRFX1NUT1BQ",
"RUQQBxIYChRXT1JLRVJfU1RBVEVfRkFVTFRFRBAIKscEChNXb3JrZXJGYXVs",
"dENhdGVnb3J5EiUKIVdPUktFUl9GQVVMVF9DQVRFR09SWV9VTlNQRUNJRklF",
"RBAAEisKJ1dPUktFUl9GQVVMVF9DQVRFR09SWV9JTlZBTElEX0FSR1VNRU5U",
"UxABEjcKM1dPUktFUl9GQVVMVF9DQVRFR09SWV9HQVRFV0FZX0FVVEhFTlRJ",
"Q0FUSU9OX0ZBSUxFRBACEisKJ1dPUktFUl9GQVVMVF9DQVRFR09SWV9QUk9U",
"T0NPTF9NSVNNQVRDSBADEiwKKFdPUktFUl9GQVVMVF9DQVRFR09SWV9QUk9U",
"T0NPTF9WSU9MQVRJT04QBBIrCidXT1JLRVJfRkFVTFRfQ0FURUdPUllfUElQ",
"RV9ESVNDT05ORUNURUQQBRIyCi5XT1JLRVJfRkFVTFRfQ0FURUdPUllfTVhB",
"Q0NFU1NfQ1JFQVRJT05fRkFJTEVEEAYSMQotV09SS0VSX0ZBVUxUX0NBVEVH",
"T1JZX01YQUNDRVNTX0NPTU1BTkRfRkFJTEVEEAcSOgo2V09SS0VSX0ZBVUxU",
"X0NBVEVHT1JZX01YQUNDRVNTX0VWRU5UX0NPTlZFUlNJT05fRkFJTEVEEAgS",
"IgoeV09SS0VSX0ZBVUxUX0NBVEVHT1JZX1NUQV9IVU5HEAkSKAokV09SS0VS",
"X0ZBVUxUX0NBVEVHT1JZX1FVRVVFX09WRVJGTE9XEAoSKgomV09SS0VSX0ZB",
"VUxUX0NBVEVHT1JZX1NIVVRET1dOX1RJTUVPVVQQC0ImqgIjWkIuTU9NLldX",
"Lk14R2F0ZXdheS5Db250cmFjdHMuUHJvdG9iBnByb3RvMw=="));
"CRIXCg9nYXRld2F5X3ZlcnNpb24YAyABKAkSFwoPbWF4X2ZyYW1lX2J5dGVz",
"GAQgASgNImkKC1dvcmtlckhlbGxvEhgKEHByb3RvY29sX3ZlcnNpb24YASAB",
"KA0SDQoFbm9uY2UYAiABKAkSGQoRd29ya2VyX3Byb2Nlc3NfaWQYAyABKAUS",
"FgoOd29ya2VyX3ZlcnNpb24YBCABKAkijgEKC1dvcmtlclJlYWR5EhkKEXdv",
"cmtlcl9wcm9jZXNzX2lkGAEgASgFEhcKD214YWNjZXNzX3Byb2dpZBgCIAEo",
"CRIWCg5teGFjY2Vzc19jbHNpZBgDIAEoCRIzCg9yZWFkeV90aW1lc3RhbXAY",
"BCABKAsyGi5nb29nbGUucHJvdG9idWYuVGltZXN0YW1wIncKDVdvcmtlckNv",
"bW1hbmQSLwoHY29tbWFuZBgBIAEoCzIeLm14YWNjZXNzX2dhdGV3YXkudjEu",
"TXhDb21tYW5kEjUKEWVucXVldWVfdGltZXN0YW1wGAIgASgLMhouZ29vZ2xl",
"LnByb3RvYnVmLlRpbWVzdGFtcCKBAQoSV29ya2VyQ29tbWFuZFJlcGx5EjIK",
"BXJlcGx5GAEgASgLMiMubXhhY2Nlc3NfZ2F0ZXdheS52MS5NeENvbW1hbmRS",
"ZXBseRI3ChNjb21wbGV0ZWRfdGltZXN0YW1wGAIgASgLMhouZ29vZ2xlLnBy",
"b3RvYnVmLlRpbWVzdGFtcCIeCgxXb3JrZXJDYW5jZWwSDgoGcmVhc29uGAEg",
"ASgJIlEKDldvcmtlclNodXRkb3duEi8KDGdyYWNlX3BlcmlvZBgBIAEoCzIZ",
"Lmdvb2dsZS5wcm90b2J1Zi5EdXJhdGlvbhIOCgZyZWFzb24YAiABKAkiSAoR",
"V29ya2VyU2h1dGRvd25BY2sSMwoGc3RhdHVzGAEgASgLMiMubXhhY2Nlc3Nf",
"Z2F0ZXdheS52MS5Qcm90b2NvbFN0YXR1cyI6CgtXb3JrZXJFdmVudBIrCgVl",
"dmVudBgBIAEoCzIcLm14YWNjZXNzX2dhdGV3YXkudjEuTXhFdmVudCKlAgoP",
"V29ya2VySGVhcnRiZWF0EhkKEXdvcmtlcl9wcm9jZXNzX2lkGAEgASgFEi4K",
"BXN0YXRlGAIgASgOMh8ubXhhY2Nlc3Nfd29ya2VyLnYxLldvcmtlclN0YXRl",
"Ej8KG2xhc3Rfc3RhX2FjdGl2aXR5X3RpbWVzdGFtcBgDIAEoCzIaLmdvb2ds",
"ZS5wcm90b2J1Zi5UaW1lc3RhbXASHQoVcGVuZGluZ19jb21tYW5kX2NvdW50",
"GAQgASgNEiIKGm91dGJvdW5kX2V2ZW50X3F1ZXVlX2RlcHRoGAUgASgNEhsK",
"E2xhc3RfZXZlbnRfc2VxdWVuY2UYBiABKAQSJgoeY3VycmVudF9jb21tYW5k",
"X2NvcnJlbGF0aW9uX2lkGAcgASgJIvQBCgtXb3JrZXJGYXVsdBI5CghjYXRl",
"Z29yeRgBIAEoDjInLm14YWNjZXNzX3dvcmtlci52MS5Xb3JrZXJGYXVsdENh",
"dGVnb3J5EhYKDmNvbW1hbmRfbWV0aG9kGAIgASgJEhQKB2hyZXN1bHQYAyAB",
"KAVIAIgBARIWCg5leGNlcHRpb25fdHlwZRgEIAEoCRIaChJkaWFnbm9zdGlj",
"X21lc3NhZ2UYBSABKAkSPAoPcHJvdG9jb2xfc3RhdHVzGAYgASgLMiMubXhh",
"Y2Nlc3NfZ2F0ZXdheS52MS5Qcm90b2NvbFN0YXR1c0IKCghfaHJlc3VsdCqX",
"AgoLV29ya2VyU3RhdGUSHAoYV09SS0VSX1NUQVRFX1VOU1BFQ0lGSUVEEAAS",
"GQoVV09SS0VSX1NUQVRFX1NUQVJUSU5HEAESHAoYV09SS0VSX1NUQVRFX0hB",
"TkRTSEFLSU5HEAISIQodV09SS0VSX1NUQVRFX0lOSVRJQUxJWklOR19TVEEQ",
"AxIWChJXT1JLRVJfU1RBVEVfUkVBRFkQBBIiCh5XT1JLRVJfU1RBVEVfRVhF",
"Q1VUSU5HX0NPTU1BTkQQBRIeChpXT1JLRVJfU1RBVEVfU0hVVFRJTkdfRE9X",
"ThAGEhgKFFdPUktFUl9TVEFURV9TVE9QUEVEEAcSGAoUV09SS0VSX1NUQVRF",
"X0ZBVUxURUQQCCrHBAoTV29ya2VyRmF1bHRDYXRlZ29yeRIlCiFXT1JLRVJf",
"RkFVTFRfQ0FURUdPUllfVU5TUEVDSUZJRUQQABIrCidXT1JLRVJfRkFVTFRf",
"Q0FURUdPUllfSU5WQUxJRF9BUkdVTUVOVFMQARI3CjNXT1JLRVJfRkFVTFRf",
"Q0FURUdPUllfR0FURVdBWV9BVVRIRU5USUNBVElPTl9GQUlMRUQQAhIrCidX",
"T1JLRVJfRkFVTFRfQ0FURUdPUllfUFJPVE9DT0xfTUlTTUFUQ0gQAxIsCihX",
"T1JLRVJfRkFVTFRfQ0FURUdPUllfUFJPVE9DT0xfVklPTEFUSU9OEAQSKwon",
"V09SS0VSX0ZBVUxUX0NBVEVHT1JZX1BJUEVfRElTQ09OTkVDVEVEEAUSMgou",
"V09SS0VSX0ZBVUxUX0NBVEVHT1JZX01YQUNDRVNTX0NSRUFUSU9OX0ZBSUxF",
"RBAGEjEKLVdPUktFUl9GQVVMVF9DQVRFR09SWV9NWEFDQ0VTU19DT01NQU5E",
"X0ZBSUxFRBAHEjoKNldPUktFUl9GQVVMVF9DQVRFR09SWV9NWEFDQ0VTU19F",
"VkVOVF9DT05WRVJTSU9OX0ZBSUxFRBAIEiIKHldPUktFUl9GQVVMVF9DQVRF",
"R09SWV9TVEFfSFVORxAJEigKJFdPUktFUl9GQVVMVF9DQVRFR09SWV9RVUVV",
"RV9PVkVSRkxPVxAKEioKJldPUktFUl9GQVVMVF9DQVRFR09SWV9TSFVURE9X",
"Tl9USU1FT1VUEAtCJqoCI1pCLk1PTS5XVy5NeEdhdGV3YXkuQ29udHJhY3Rz",
"LlByb3RvYgZwcm90bzM="));
descriptor = pbr::FileDescriptor.FromGeneratedCode(descriptorData,
new pbr::FileDescriptor[] { global::Google.Protobuf.WellKnownTypes.DurationReflection.Descriptor, global::Google.Protobuf.WellKnownTypes.TimestampReflection.Descriptor, global::ZB.MOM.WW.MxGateway.Contracts.Proto.MxaccessGatewayReflection.Descriptor, },
new pbr::GeneratedClrTypeInfo(new[] {typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerState), typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerFaultCategory), }, null, new pbr::GeneratedClrTypeInfo[] {
new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerEnvelope), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerEnvelope.Parser, new[]{ "ProtocolVersion", "SessionId", "Sequence", "CorrelationId", "GatewayHello", "WorkerHello", "WorkerReady", "WorkerCommand", "WorkerCommandReply", "WorkerCancel", "WorkerShutdown", "WorkerShutdownAck", "WorkerEvent", "WorkerHeartbeat", "WorkerFault" }, new[]{ "Body" }, null, null, null),
new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello), global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello.Parser, new[]{ "SupportedProtocolVersion", "Nonce", "GatewayVersion" }, null, null, null, null),
new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello), global::ZB.MOM.WW.MxGateway.Contracts.Proto.GatewayHello.Parser, new[]{ "SupportedProtocolVersion", "Nonce", "GatewayVersion", "MaxFrameBytes" }, null, null, null, null),
new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerHello), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerHello.Parser, new[]{ "ProtocolVersion", "Nonce", "WorkerProcessId", "WorkerVersion" }, null, null, null, null),
new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerReady), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerReady.Parser, new[]{ "WorkerProcessId", "MxaccessProgid", "MxaccessClsid", "ReadyTimestamp" }, null, null, null, null),
new pbr::GeneratedClrTypeInfo(typeof(global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerCommand), global::ZB.MOM.WW.MxGateway.Contracts.Proto.WorkerCommand.Parser, new[]{ "Command", "EnqueueTimestamp" }, null, null, null, null),
@@ -1108,6 +1109,7 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
supportedProtocolVersion_ = other.supportedProtocolVersion_;
nonce_ = other.nonce_;
gatewayVersion_ = other.gatewayVersion_;
maxFrameBytes_ = other.maxFrameBytes_;
_unknownFields = pb::UnknownFieldSet.Clone(other._unknownFields);
}
@@ -1153,6 +1155,25 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
}
}
/// <summary>Field number for the "max_frame_bytes" field.</summary>
public const int MaxFrameBytesFieldNumber = 4;
private uint maxFrameBytes_;
/// <summary>
/// Maximum worker-frame payload size, in bytes, negotiated by the gateway from its
/// configured pipe limit. The worker adopts this as its frame-protocol MaxMessageBytes
/// instead of a hard-coded default; 0 (an older gateway that never set the field) means
/// "use the worker's built-in default". Sits above the public gRPC cap by an
/// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
/// </summary>
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
[global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
public uint MaxFrameBytes {
get { return maxFrameBytes_; }
set {
maxFrameBytes_ = value;
}
}
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
[global::System.CodeDom.Compiler.GeneratedCode("protoc", null)]
public override bool Equals(object other) {
@@ -1171,6 +1192,7 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
if (SupportedProtocolVersion != other.SupportedProtocolVersion) return false;
if (Nonce != other.Nonce) return false;
if (GatewayVersion != other.GatewayVersion) return false;
if (MaxFrameBytes != other.MaxFrameBytes) return false;
return Equals(_unknownFields, other._unknownFields);
}
@@ -1181,6 +1203,7 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
if (SupportedProtocolVersion != 0) hash ^= SupportedProtocolVersion.GetHashCode();
if (Nonce.Length != 0) hash ^= Nonce.GetHashCode();
if (GatewayVersion.Length != 0) hash ^= GatewayVersion.GetHashCode();
if (MaxFrameBytes != 0) hash ^= MaxFrameBytes.GetHashCode();
if (_unknownFields != null) {
hash ^= _unknownFields.GetHashCode();
}
@@ -1211,6 +1234,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
output.WriteRawTag(26);
output.WriteString(GatewayVersion);
}
if (MaxFrameBytes != 0) {
output.WriteRawTag(32);
output.WriteUInt32(MaxFrameBytes);
}
if (_unknownFields != null) {
_unknownFields.WriteTo(output);
}
@@ -1233,6 +1260,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
output.WriteRawTag(26);
output.WriteString(GatewayVersion);
}
if (MaxFrameBytes != 0) {
output.WriteRawTag(32);
output.WriteUInt32(MaxFrameBytes);
}
if (_unknownFields != null) {
_unknownFields.WriteTo(ref output);
}
@@ -1252,6 +1283,9 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
if (GatewayVersion.Length != 0) {
size += 1 + pb::CodedOutputStream.ComputeStringSize(GatewayVersion);
}
if (MaxFrameBytes != 0) {
size += 1 + pb::CodedOutputStream.ComputeUInt32Size(MaxFrameBytes);
}
if (_unknownFields != null) {
size += _unknownFields.CalculateSize();
}
@@ -1273,6 +1307,9 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
if (other.GatewayVersion.Length != 0) {
GatewayVersion = other.GatewayVersion;
}
if (other.MaxFrameBytes != 0) {
MaxFrameBytes = other.MaxFrameBytes;
}
_unknownFields = pb::UnknownFieldSet.MergeFrom(_unknownFields, other._unknownFields);
}
@@ -1304,6 +1341,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
GatewayVersion = input.ReadString();
break;
}
case 32: {
MaxFrameBytes = input.ReadUInt32();
break;
}
}
}
#endif
@@ -1335,6 +1376,10 @@ namespace ZB.MOM.WW.MxGateway.Contracts.Proto {
GatewayVersion = input.ReadString();
break;
}
case 32: {
MaxFrameBytes = input.ReadUInt32();
break;
}
}
}
}
@@ -42,6 +42,12 @@ message GatewayHello {
uint32 supported_protocol_version = 1;
string nonce = 2;
string gateway_version = 3;
// Maximum worker-frame payload size, in bytes, negotiated by the gateway from its
// configured pipe limit. The worker adopts this as its frame-protocol MaxMessageBytes
// instead of a hard-coded default; 0 (an older gateway that never set the field) means
// "use the worker's built-in default". Sits above the public gRPC cap by an
// envelope-overhead margin so an accepted gRPC payload always fits one worker frame.
uint32 max_frame_bytes = 4;
}
message WorkerHello {
@@ -23,6 +23,13 @@
</PropertyGroup>
<ItemGroup>
<!-- Generated\**\*.cs is Compile-Removed and regenerated by Grpc.Tools into the TRACKED Generated
folder. It MUST be committed after any .proto change: Grpc.Tools skips regeneration when the
committed output looks up to date, so on net10 the freshly regenerated code compiles (drift is
invisible) while the net48 worker, which consumes the COMMITTED Generated\*.cs, fails with
CS0246 on new types. Regenerate + commit after editing a .proto; if a build does not pick up
the change, delete Generated\*.cs to force a full regen. scripts/check-codegen.ps1 enforces
this in CI. See docs/Contracts.md. -->
<Compile Remove="Generated\**\*.cs" />
<Protobuf Include="Protos\mxaccess_gateway.proto" ProtoRoot="Protos" OutputDir="Generated" GrpcOutputDir="Generated" GrpcServices="Both" />
<Protobuf Include="Protos\mxaccess_worker.proto" ProtoRoot="Protos" OutputDir="Generated" GrpcServices="None" />
@@ -22,8 +22,8 @@
(IntegrationTests-028).
-->
<ItemGroup>
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.1.4" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.1.4" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="10.0.7" />
<PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="10.0.7" />
</ItemGroup>
@@ -5,8 +5,18 @@ public sealed class AuthenticationOptions
/// <summary>Gets the authentication mode.</summary>
public AuthenticationMode Mode { get; init; } = AuthenticationMode.ApiKey;
/// <summary>Gets the SQLite database path for authentication credentials.</summary>
public string SqlitePath { get; init; } = @"C:\ProgramData\MxGateway\gateway-auth.db";
/// <summary>
/// Gets the SQLite database path for authentication credentials. The default is
/// derived from <see cref="Environment.SpecialFolder.CommonApplicationData"/>
/// (<c>C:\ProgramData</c> on Windows, <c>/usr/share</c> or the container equivalent
/// elsewhere) so the credential store never lands in the launch working directory on a
/// non-Windows host. The production hosts override this with an explicit path in
/// <c>appsettings.json</c>; the validator rejects a non-rooted override.
/// </summary>
public string SqlitePath { get; init; } = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
"MxGateway",
"gateway-auth.db");
/// <summary>Gets the secret manager name for API key pepper.</summary>
public string PepperSecretName { get; init; } = "MxGateway:ApiKeyPepper";
@@ -1,4 +1,7 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.FileProviders;
using Microsoft.Extensions.Hosting;
using ZB.MOM.WW.Configuration;
namespace ZB.MOM.WW.MxGateway.Server.Configuration;
@@ -12,6 +15,14 @@ public static class GatewayConfigurationServiceCollectionExtensions
public static IServiceCollection AddGatewayConfiguration(
this IServiceCollection services, IConfiguration configuration)
{
// GatewayOptionsValidator depends on IHostEnvironment to gate the production-only rules
// (SEC-04/06). The real host and the apikey CLI both build through WebApplicationBuilder,
// which registers IHostEnvironment before this call — so TryAdd is a no-op there. Minimal
// containers (unit tests, tooling) have none; fall back to a non-production environment so
// the validator still activates and the production guards stay off outside a real
// deployment (the fail-fast guards only make sense against a real host environment).
services.TryAddSingleton<IHostEnvironment>(new FallbackHostEnvironment());
services.AddValidatedOptions<GatewayOptions, GatewayOptionsValidator>(
configuration, GatewayOptions.SectionName);
@@ -19,4 +30,19 @@ public static class GatewayConfigurationServiceCollectionExtensions
return services;
}
/// <summary>
/// Non-production <see cref="IHostEnvironment"/> used only when no real host registered one
/// (minimal test/tooling containers). The real host's registration always wins via TryAdd.
/// </summary>
private sealed class FallbackHostEnvironment : IHostEnvironment
{
public string EnvironmentName { get; set; } = Environments.Development;
public string ApplicationName { get; set; } = typeof(FallbackHostEnvironment).Assembly.GetName().Name ?? "MxGateway";
public string ContentRootPath { get; set; } = AppContext.BaseDirectory;
public IFileProvider ContentRootFileProvider { get; set; } = new NullFileProvider();
}
}
@@ -46,4 +46,10 @@ public sealed class GatewayOptions
/// <summary>Gets self-signed TLS certificate auto-generation options.</summary>
public TlsOptions Tls { get; init; } = new();
/// <summary>
/// Gets security hot-path options (API-key verification cache / last-used coalescing and
/// login / API-key rate limiting).
/// </summary>
public SecurityOptions Security { get; init; } = new();
}
@@ -1,6 +1,7 @@
using ZB.MOM.WW.Auth.Abstractions.Ldap;
using ZB.MOM.WW.Configuration;
using ZB.MOM.WW.MxGateway.Contracts;
using ZB.MOM.WW.MxGateway.Server.Workers;
namespace ZB.MOM.WW.MxGateway.Server.Configuration;
@@ -9,18 +10,83 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
private const int MinimumMaxMessageBytes = 1024;
private const int MaximumMaxMessageBytes = 256 * 1024 * 1024;
// Whether the host is running in the Production environment. Drives the production-only
// hard-stops (dashboard login disabled, plaintext LDAP transport) that must abort startup
// rather than merely warn. Non-production hosts keep the permissive dev posture.
private readonly bool _isProduction;
/// <summary>
/// Initializes a new instance of the <see cref="GatewayOptionsValidator"/> class for the
/// dependency-injection path, deriving the production posture from the host environment.
/// </summary>
/// <param name="environment">The host environment.</param>
public GatewayOptionsValidator(IHostEnvironment environment)
{
ArgumentNullException.ThrowIfNull(environment);
_isProduction = environment.IsProduction();
}
/// <summary>
/// Initializes a new instance of the <see cref="GatewayOptionsValidator"/> class for unit
/// tests and non-DI callers. Defaults to a non-production posture so the production-only
/// hard-stops do not fire; pass <see langword="true"/> to exercise them.
/// </summary>
/// <param name="isProduction">Whether to treat the host as running in Production.</param>
internal GatewayOptionsValidator(bool isProduction = false)
{
_isProduction = isProduction;
}
/// <inheritdoc />
protected override void Validate(ValidationBuilder builder, GatewayOptions options)
{
ValidateAuthentication(options.Authentication, builder);
ValidateLdap(options.Ldap, builder);
ValidateLdap(options.Ldap, builder, _isProduction);
ValidateWorker(options.Worker, builder);
ValidateSessions(options.Sessions, builder);
ValidateEvents(options.Events, builder);
ValidateDashboard(options.Dashboard, builder);
ValidateDashboard(options.Dashboard, builder, _isProduction);
ValidateProtocol(options.Protocol, builder);
ValidateFrameSizeHeadroom(options.Worker, options.Protocol, builder);
ValidateAlarms(options.Alarms, builder);
ValidateTls(options.Tls, builder);
ValidateSecurity(options.Security, builder);
}
private static void ValidateSecurity(SecurityOptions options, ValidationBuilder builder)
{
// Cache/coalesce windows may be 0 (disables that mechanism); negatives are invalid.
AddIfNegative(
options.ApiKeyVerificationCacheSeconds,
"MxGateway:Security:ApiKeyVerificationCacheSeconds must be greater than or equal to zero (0 disables the verification cache).",
builder);
AddIfNegative(
options.ApiKeyLastUsedCoalesceSeconds,
"MxGateway:Security:ApiKeyLastUsedCoalesceSeconds must be greater than or equal to zero (0 forwards every last_used write).",
builder);
// Rate-limit knobs must be positive: a zero permit/window/limit is a misconfiguration that
// would either reject every request or divide by zero rather than express an intent.
AddIfNotPositive(
options.LoginRateLimitPermitLimit,
"MxGateway:Security:LoginRateLimitPermitLimit must be greater than zero.",
builder);
AddIfNotPositive(
options.LoginRateLimitWindowSeconds,
"MxGateway:Security:LoginRateLimitWindowSeconds must be greater than zero.",
builder);
AddIfNotPositive(
options.ApiKeyFailureLimit,
"MxGateway:Security:ApiKeyFailureLimit must be greater than zero.",
builder);
AddIfNotPositive(
options.ApiKeyFailureWindowSeconds,
"MxGateway:Security:ApiKeyFailureWindowSeconds must be greater than zero.",
builder);
AddIfNotPositive(
options.ApiKeyFailureTrackedPeers,
"MxGateway:Security:ApiKeyFailureTrackedPeers must be greater than zero.",
builder);
}
private static void ValidateAuthentication(AuthenticationOptions options, ValidationBuilder builder)
@@ -41,6 +107,10 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
options.SqlitePath,
"MxGateway:Authentication:SqlitePath must be a valid filesystem path.",
builder);
AddIfNotRooted(
options.SqlitePath,
"MxGateway:Authentication:SqlitePath must be an absolute (rooted) path so the credential store never lands in the launch working directory.",
builder);
AddIfBlank(
options.PepperSecretName,
"MxGateway:Authentication:PepperSecretName is required when API-key authentication is enabled.",
@@ -48,7 +118,7 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
}
}
private static void ValidateLdap(LdapOptions options, ValidationBuilder builder)
private static void ValidateLdap(LdapOptions options, ValidationBuilder builder, bool isProduction)
{
if (!options.Enabled)
{
@@ -83,6 +153,16 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
{
builder.Add("MxGateway:Ldap:AllowInsecure must be true when Transport is None (plaintext).");
}
// Production hard-stop: plaintext LDAP binds send the operator's password in the clear.
// The permissive dev default (Transport=None against the shared GLAuth instance) is
// acceptable off-production but must never ship to a Production host. Deployed hosts must
// set Transport=Ldaps/StartTls; see docs/GatewayConfiguration.md and glauth.md.
if (isProduction && options.Transport == LdapTransport.None)
{
builder.Add(
"MxGateway:Ldap:Transport must not be None (plaintext) in the Production environment; use Ldaps or StartTls.");
}
}
private static void ValidateWorker(WorkerOptions options, ValidationBuilder builder)
@@ -224,8 +304,19 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
$"MxGateway:Events:MaxSparseArrayLength must be between 1 and {Array.MaxLength}.");
}
private static void ValidateDashboard(DashboardOptions options, ValidationBuilder builder)
private static void ValidateDashboard(DashboardOptions options, ValidationBuilder builder, bool isProduction)
{
// Production hard-stop: DisableLogin swaps in an auto-login handler that authenticates
// EVERY request (remote included) as AutoLoginUser holding both roles, turning the whole
// dashboard — API-key CRUD and worker Kill included — into an unauthenticated admin
// surface on a 0.0.0.0-bound port. It is a dev/test-only convenience; abort startup if it
// is set in Production. Non-production hosts keep the existing runtime warning.
if (isProduction && options.DisableLogin)
{
builder.Add(
"MxGateway:Dashboard:DisableLogin must not be true in the Production environment; it disables all dashboard authentication.");
}
// GroupToRole shape is validated even when the dashboard is disabled so
// misconfiguration surfaces at startup; emptiness is allowed, with the
// consequence that no LDAP user can sign in (login returns "no roles
@@ -348,6 +439,10 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
options.SelfSignedCertPath,
"MxGateway:Tls:SelfSignedCertPath must be a valid filesystem path.",
builder);
AddIfNotRooted(
options.SelfSignedCertPath,
"MxGateway:Tls:SelfSignedCertPath must be an absolute (rooted) path so the generated private key never lands in the launch working directory.",
builder);
foreach (string dns in options.AdditionalDnsNames)
{
@@ -373,6 +468,35 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
}
}
// The worker-frame (pipe) maximum must stay above the public gRPC cap by the envelope-overhead
// reserve, otherwise a maximally-sized accepted gRPC payload does not fit one worker frame once
// wrapped in a WorkerEnvelope and the outbound write faults the whole session (IPC-03). Fail fast
// at startup rather than mid-traffic. Only checked when both knobs are themselves in range so the
// message is not doubled up with the individual range errors.
private static void ValidateFrameSizeHeadroom(
WorkerOptions worker,
ProtocolOptions protocol,
ValidationBuilder builder)
{
bool workerInRange = worker.MaxMessageBytes is >= MinimumMaxMessageBytes and <= MaximumMaxMessageBytes;
bool grpcInRange = protocol.MaxGrpcMessageBytes is >= MinimumMaxMessageBytes and <= MaximumMaxMessageBytes;
if (!workerInRange || !grpcInRange)
{
return;
}
long requiredWorkerMax =
(long)protocol.MaxGrpcMessageBytes + WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes;
if (worker.MaxMessageBytes < requiredWorkerMax)
{
builder.Add(
$"MxGateway:Worker:MaxMessageBytes ({worker.MaxMessageBytes}) must be at least "
+ $"MxGateway:Protocol:MaxGrpcMessageBytes ({protocol.MaxGrpcMessageBytes}) plus the "
+ $"{WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes}-byte worker-frame envelope reserve "
+ $"(>= {requiredWorkerMax}).");
}
}
private static void AddIfBlank(string? value, string message, ValidationBuilder builder)
{
builder.RequireThat(!string.IsNullOrWhiteSpace(value), message);
@@ -388,6 +512,53 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
builder.RequireThat(value >= 0, message);
}
private static void AddIfNotRooted(string? value, string message, ValidationBuilder builder)
{
// Security-sensitive paths (the auth DB, the self-signed private key) must be absolute:
// a non-rooted value silently resolves against the launch working directory, so the store
// moves with the CWD and can leak into the source tree. Reject rather than auto-root —
// silent relocation of a credential store is worse than a boot error. Blank is handled by
// AddIfBlank; an empty value is not treated as non-rooted here.
if (string.IsNullOrWhiteSpace(value))
{
return;
}
if (!IsRootedForAnyPlatform(value))
{
builder.Add(message);
}
}
/// <summary>
/// Determines whether <paramref name="value"/> is an absolute path for <em>any</em> platform,
/// not just the host running the validator. This matters on the macOS dev box, where the
/// production <c>appsettings.json</c> ships Windows-absolute paths (<c>C:\ProgramData\...</c>)
/// that <see cref="Path.IsPathRooted(string)"/> reports as non-rooted on Unix. The intent of the
/// rooting check is to reject bare filenames that resolve against the launch working directory,
/// so a valid Windows drive-qualified or UNC path must pass regardless of the current OS.
/// </summary>
private static bool IsRootedForAnyPlatform(string value)
{
// Rooted on the current OS (Unix "/...", or a Windows drive/UNC path when on Windows).
if (Path.IsPathRooted(value))
{
return true;
}
// Windows drive-qualified path ("C:\..." or "C:/...") checked on a non-Windows host.
if (value.Length >= 3
&& char.IsLetter(value[0])
&& value[1] == ':'
&& (value[2] == '\\' || value[2] == '/'))
{
return true;
}
// Windows UNC path ("\\server\share") checked on a non-Windows host.
return value.StartsWith(@"\\", StringComparison.Ordinal);
}
private static void AddIfInvalidPath(string? value, string message, ValidationBuilder builder)
{
if (string.IsNullOrWhiteSpace(value))
@@ -0,0 +1,65 @@
namespace ZB.MOM.WW.MxGateway.Server.Configuration;
/// <summary>
/// Security hot-path options bound from <c>MxGateway:Security</c>. Groups the API-key verification
/// cache and <c>last_used</c> write-coalescing knobs (SEC-08) with the login and API-key
/// rate-limiting knobs (SEC-11). Defaults preserve safe, low-overhead behaviour without requiring
/// operators to configure anything.
/// </summary>
public sealed class SecurityOptions
{
/// <summary>The configuration sub-section this binds from.</summary>
public const string SectionName = "MxGateway:Security";
/// <summary>
/// Gets the time-to-live, in seconds, of a cached successful API-key verification. A cache hit
/// within this window skips the per-call SQLite read (and the library verifier's coupled
/// <c>last_used</c> write). Explicit invalidation on gateway-initiated revoke/rotate is the
/// primary correctness mechanism; this short TTL is the backstop for out-of-band mutations.
/// Set to <c>0</c> to disable caching. Default is 15 seconds.
/// </summary>
public int ApiKeyVerificationCacheSeconds { get; init; } = 15;
/// <summary>
/// Gets the coalescing window, in seconds, for the <c>last_used_utc</c> write. The library
/// verifier couples the write into every verification; this decorator forwards at most one
/// <c>MarkUsed</c> per key per window, so a hammered key produces at most one write per window
/// rather than one per authenticated RPC. Set to <c>0</c> to forward every write. Default is
/// 60 seconds (at most one write per key per minute).
/// </summary>
public int ApiKeyLastUsedCoalesceSeconds { get; init; } = 60;
/// <summary>
/// Gets the maximum number of <c>POST /auth/login</c> attempts permitted per remote IP within
/// <see cref="LoginRateLimitWindowSeconds"/> before requests are rejected with HTTP 429. Default
/// is 10.
/// </summary>
public int LoginRateLimitPermitLimit { get; init; } = 10;
/// <summary>
/// Gets the fixed-window length, in seconds, for the <c>POST /auth/login</c> per-IP rate limit.
/// Default is 60 seconds.
/// </summary>
public int LoginRateLimitWindowSeconds { get; init; } = 60;
/// <summary>
/// Gets the number of consecutive failed API-key verifications, per peer, within
/// <see cref="ApiKeyFailureWindowSeconds"/> that trips the in-process short-circuit. Once tripped,
/// the gRPC auth path rejects further attempts before the store read; a successful verification
/// resets the peer's counter. Default is 10.
/// </summary>
public int ApiKeyFailureLimit { get; init; } = 10;
/// <summary>
/// Gets the sliding-window length, in seconds, over which API-key verification failures are
/// counted per peer. Default is 60 seconds.
/// </summary>
public int ApiKeyFailureWindowSeconds { get; init; } = 60;
/// <summary>
/// Gets the maximum number of distinct peers tracked by the API-key failure counter. The counter
/// is a bounded LRU so a spray of unique peer keys cannot grow memory without limit. Default is
/// 4096.
/// </summary>
public int ApiKeyFailureTrackedPeers { get; init; } = 4096;
}
@@ -7,9 +7,19 @@ namespace ZB.MOM.WW.MxGateway.Server.Configuration;
/// </summary>
public sealed class TlsOptions
{
/// <summary>Path to the persisted self-signed PFX. Reused across restarts.</summary>
public string SelfSignedCertPath { get; init; } =
@"C:\ProgramData\MxGateway\certs\gateway-selfsigned.pfx";
/// <summary>
/// Path to the persisted self-signed PFX. Reused across restarts. The default is derived
/// from <see cref="Environment.SpecialFolder.CommonApplicationData"/> (<c>C:\ProgramData</c>
/// on Windows, <c>/usr/share</c> or the container equivalent elsewhere) so a generated
/// private key never lands in the launch working directory on a non-Windows host. The
/// production hosts override this with an explicit path; the validator rejects a non-rooted
/// override.
/// </summary>
public string SelfSignedCertPath { get; init; } = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
"MxGateway",
"certs",
"gateway-selfsigned.pfx");
/// <summary>Lifetime in years of a freshly generated certificate.</summary>
public int ValidityYears { get; init; } = 10;
@@ -33,6 +33,14 @@ public sealed class WorkerOptions
/// <summary>The grace period in seconds after a heartbeat before considering the worker unresponsive.</summary>
public int HeartbeatGraceSeconds { get; init; } = 15;
/// <summary>The maximum message size in bytes for IPC communication.</summary>
public int MaxMessageBytes { get; init; } = 16 * 1024 * 1024;
/// <summary>
/// The maximum worker-frame (pipe) message size in bytes. Must stay at least
/// <see cref="Workers.WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes"/> above
/// <see cref="ProtocolOptions.MaxGrpcMessageBytes"/> so a maximally-sized accepted gRPC payload
/// still fits one worker frame (IPC-03); the gateway conveys this value to the worker in the
/// handshake (<c>GatewayHello.max_frame_bytes</c>, IPC-02). Default is the 16 MB public gRPC cap
/// plus that reserve.
/// </summary>
public int MaxMessageBytes { get; init; } =
(16 * 1024 * 1024) + Workers.WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes;
}
@@ -164,6 +164,7 @@ else
<th scope="col">Constraints</th>
<th scope="col">Created</th>
<th scope="col">Last Used</th>
<th scope="col">Expires</th>
@if (CanManageApiKeys)
{
<th scope="col">Actions</th>
@@ -175,12 +176,13 @@ else
{
<tr>
<td><code>@key.KeyId</code></td>
<td><StatusBadge Text="@(key.RevokedUtc is null ? "Active" : "Revoked")" /></td>
<td><StatusBadge Text="@KeyStatus(key)" /></td>
<td>@DashboardDisplay.Text(key.DisplayName)</td>
<td>@DashboardDisplay.Text(string.Join(", ", key.Scopes.Order(StringComparer.Ordinal)))</td>
<td>@DashboardDisplay.Text(ConstraintText(key.Constraints))</td>
<td>@DashboardDisplay.DateTime(key.CreatedUtc)</td>
<td>@DashboardDisplay.DateTime(key.LastUsedUtc)</td>
<td>@(key.ExpiresUtc is null ? "Never" : DashboardDisplay.DateTime(key.ExpiresUtc))</td>
@if (CanManageApiKeys)
{
<td>
@@ -468,6 +470,35 @@ else
}
}
// Window before an expiry within which a key is flagged as "Expiring" (warn) rather than "Active".
private static readonly TimeSpan ExpiringSoonWindow = TimeSpan.FromDays(7);
// Status vocabulary understood by StatusBadge: Revoked wins over expiry; a past expiry is Expired
// (bad), an expiry inside ExpiringSoonWindow is Expiring (warn), otherwise Active (SEC-10).
private static string KeyStatus(DashboardApiKeySummary key)
{
if (key.RevokedUtc is not null)
{
return "Revoked";
}
if (key.ExpiresUtc is { } expiresAt)
{
DateTimeOffset now = DateTimeOffset.UtcNow;
if (expiresAt <= now)
{
return "Expired";
}
if (expiresAt - now <= ExpiringSoonWindow)
{
return "Expiring";
}
}
return "Active";
}
private static string ConstraintText(ApiKeyConstraints constraints)
{
if (constraints.IsEmpty)
@@ -9,8 +9,8 @@
{
"Ready" or "Healthy" or "Active" => StatusState.Ok,
"Creating" or "StartingWorker" or "WaitingForPipe" or "InitializingWorker" or "Closing"
or "Stale" or "Degraded" => StatusState.Warn,
"Faulted" or "Unavailable" => StatusState.Bad,
or "Stale" or "Degraded" or "Expiring" => StatusState.Warn,
"Faulted" or "Unavailable" or "Expired" => StatusState.Bad,
_ => StatusState.Idle,
};
}
@@ -15,7 +15,8 @@ public sealed class DashboardApiKeyManagementService(
ApiKeyAdminCommands adminCommands,
IApiKeyAdminStore adminStore,
IAuditWriter auditWriter,
IHttpContextAccessor httpContextAccessor) : IDashboardApiKeyManagementService
IHttpContextAccessor httpContextAccessor,
IApiKeyCacheInvalidator? cacheInvalidator = null) : IDashboardApiKeyManagementService
{
private const string UnauthorizedMessage = "Sign in with an authorized LDAP account to manage API keys.";
private const string PepperUnavailableMarker = "pepper unavailable";
@@ -100,6 +101,10 @@ public sealed class DashboardApiKeyManagementService(
.RevokeKeyAsync(normalizedKeyId, RemoteAddress(), cancellationToken)
.ConfigureAwait(false);
// SEC-08: drop any cached verification for this key in THIS gateway process so the revoke
// takes effect immediately rather than after the verification-cache TTL elapses.
cacheInvalidator?.Invalidate(normalizedKeyId);
await WriteDashboardAuditAsync(
user,
normalizedKeyId,
@@ -138,6 +143,10 @@ public sealed class DashboardApiKeyManagementService(
.RotateKeyAsync(normalizedKeyId, RemoteAddress(), cancellationToken)
.ConfigureAwait(false);
// SEC-08: the old secret's cached verification must not outlive the rotate — evict it so
// the superseded secret stops authenticating within this process immediately.
cacheInvalidator?.Invalidate(normalizedKeyId);
bool succeeded = rotated.Token is not null;
await WriteDashboardAuditAsync(
@@ -182,6 +191,10 @@ public sealed class DashboardApiKeyManagementService(
.DeleteAsync(normalizedKeyId, cancellationToken)
.ConfigureAwait(false);
// SEC-08: a deleted key is only reachable after a prior revoke, but evict defensively so no
// cached verification survives the delete.
cacheInvalidator?.Invalidate(normalizedKeyId);
await WriteDashboardAuditAsync(
user,
normalizedKeyId,
@@ -9,4 +9,5 @@ public sealed record DashboardApiKeySummary(
ApiKeyConstraints Constraints,
DateTimeOffset CreatedUtc,
DateTimeOffset? LastUsedUtc,
DateTimeOffset? RevokedUtc);
DateTimeOffset? RevokedUtc,
DateTimeOffset? ExpiresUtc = null);
@@ -11,6 +11,18 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// (b) authentication is fully disabled, or (c) the request is from loopback
/// and <c>MxGateway:Dashboard:AllowAnonymousLocalhost</c> is on.
/// </summary>
/// <remarks>
/// The environment bypasses in (b) and (c) grant <em>read-only</em> access only: they
/// satisfy a requirement that includes <see cref="DashboardRoles.Viewer"/> (i.e.
/// <see cref="DashboardAuthorizationRequirement.AnyDashboardRole"/>) but never the
/// <see cref="DashboardAuthorizationRequirement.AdminOnly"/> requirement. Destructive/admin
/// surfaces still require a real Admin role claim, preserving the documented
/// "anonymous localhost is read-only" contract at the policy layer rather than by accident
/// in downstream service re-checks. The loopback test trusts
/// <c>Connection.RemoteIpAddress</c>; if forwarded-headers middleware is ever added upstream,
/// <see cref="IsLoopbackRequest"/> must be revisited so a spoofed <c>X-Forwarded-For</c>
/// cannot masquerade as loopback.
/// </remarks>
public sealed class DashboardAuthorizationHandler(
IHttpContextAccessor httpContextAccessor,
IOptions<GatewayOptions> options) : AuthorizationHandler<DashboardAuthorizationRequirement>
@@ -22,14 +34,18 @@ public sealed class DashboardAuthorizationHandler(
{
GatewayOptions gatewayOptions = options.Value;
if (gatewayOptions.Authentication.Mode == AuthenticationMode.Disabled)
// Environment bypasses are read-only: they satisfy the Viewer requirement
// (AnyDashboardRole) but never AdminOnly, which lacks Viewer in RequiredRoles.
bool grantsReadOnly = requirement.RequiredRoles.Contains(DashboardRoles.Viewer);
if (grantsReadOnly && gatewayOptions.Authentication.Mode == AuthenticationMode.Disabled)
{
context.Succeed(requirement);
return Task.CompletedTask;
}
if (gatewayOptions.Dashboard.AllowAnonymousLocalhost && IsLoopbackRequest())
if (grantsReadOnly && gatewayOptions.Dashboard.AllowAnonymousLocalhost && IsLoopbackRequest())
{
context.Succeed(requirement);
@@ -1,4 +1,5 @@
using System.Text.Encodings.Web;
using System.Threading.RateLimiting;
using Microsoft.AspNetCore.Antiforgery;
using Microsoft.AspNetCore.Authentication;
using ZB.MOM.WW.MxGateway.Server.Configuration;
@@ -10,6 +11,41 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// <summary>Endpoint extensions for registering the gateway dashboard routes.</summary>
public static class DashboardEndpointRouteBuilderExtensions
{
/// <summary>
/// The named rate-limiter policy applied to the <c>POST /auth/login</c> route (SEC-11). Registered
/// in <c>GatewayApplication</c> via <see cref="GetLoginRateLimitPartition"/>.
/// </summary>
internal const string LoginRateLimiterPolicy = "dashboard-login";
/// <summary>
/// Builds the fixed-window rate-limit partition for a login request, keyed on the remote IP.
/// Shared by the production policy registration and the tests so both exercise identical limits.
/// </summary>
/// <param name="httpContext">The inbound request.</param>
/// <param name="security">The bound security options carrying the login-limit knobs.</param>
/// <returns>The per-IP fixed-window partition.</returns>
internal static RateLimitPartition<string> GetLoginRateLimitPartition(
HttpContext httpContext,
SecurityOptions security)
{
string partitionKey = httpContext.Connection.RemoteIpAddress?.ToString() ?? "unknown";
return RateLimitPartition.GetFixedWindowLimiter(
partitionKey,
_ => new FixedWindowRateLimiterOptions
{
PermitLimit = security.LoginRateLimitPermitLimit,
Window = TimeSpan.FromSeconds(security.LoginRateLimitWindowSeconds),
QueueLimit = 0,
AutoReplenishment = true,
});
}
// Test seam: a standalone partitioned limiter over the production partition logic, so a test can
// acquire leases and assert the (permit+1)th is rejected without an HTTP server.
internal static PartitionedRateLimiter<HttpContext> CreateLoginRateLimiter(SecurityOptions security)
=> PartitionedRateLimiter.Create<HttpContext, string>(
ctx => GetLoginRateLimitPartition(ctx, security));
/// <summary>Maps all gateway dashboard routes including login, logout, and Razor components.</summary>
/// <param name="endpoints">The endpoint route builder.</param>
/// <returns>The route builder for chaining.</returns>
@@ -40,6 +76,8 @@ public static class DashboardEndpointRouteBuilderExtensions
(HttpContext httpContext, IAntiforgery antiforgery, IDashboardAuthenticator authenticator) =>
PostLoginAsync(httpContext, antiforgery, authenticator))
.AllowAnonymous()
// SEC-11: throttle credential stuffing before the LDAP bind is relayed to the directory.
.RequireRateLimiting(LoginRateLimiterPolicy)
.WithName("DashboardLoginPost");
endpoints.MapPost(
@@ -1,6 +1,8 @@
using System.Security.Claims;
using System.Text.Json;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.Audit;
using ZB.MOM.WW.MxGateway.Server.Sessions;
namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
@@ -8,21 +10,35 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// <summary>
/// Default implementation of <see cref="IDashboardSessionAdminService"/>: gates
/// destructive session actions on the <see cref="DashboardRoles.Admin"/> role,
/// audit-logs successful operations, and converts <see cref="SessionManagerException"/>
/// (and any other unexpected exceptions) into <see cref="DashboardSessionAdminResult.Fail(string)"/>
/// so the Blazor pages never see a raw exception.
/// records each attempt as a canonical <see cref="AuditEvent"/> through <see cref="IAuditWriter"/>
/// (in addition to the operational <see cref="ILogger"/> line), and converts
/// <see cref="SessionManagerException"/> (and any other unexpected exceptions) into
/// <see cref="DashboardSessionAdminResult.Fail(string)"/> so the Blazor pages never see a raw exception.
/// </summary>
/// <remarks>
/// The constant <c>dashboard-admin-kill</c> is the reason passed to
/// <see cref="ISessionManager.KillWorkerAsync"/> and forwarded as the
/// <c>reason</c> tag on the <c>mxgateway.workers.killed</c> counter and in
/// the worker-kill audit log entries.
/// the worker-kill audit log entries. Destructive dashboard actions write durable,
/// queryable audit rows (actions <c>dashboard-close-session</c> / <c>dashboard-kill-worker</c>)
/// to the canonical <c>audit_event</c> store, mirroring the API-key management path so a
/// worker killed mid-production is not visible only in a rotatable <see cref="ILogger"/> line.
/// </remarks>
public sealed class DashboardSessionAdminService(
ISessionManager sessionManager,
IHttpContextAccessor httpContextAccessor,
IAuditWriter auditWriter,
ILogger<DashboardSessionAdminService>? logger = null) : IDashboardSessionAdminService
{
/// <summary>Canonical <see cref="AuditEvent.Category"/> for dashboard session-admin actions.</summary>
internal const string SessionAdminCategory = "SessionAdmin";
/// <summary>Canonical <see cref="AuditEvent.Action"/> for a dashboard session close.</summary>
internal const string CloseSessionAction = "dashboard-close-session";
/// <summary>Canonical <see cref="AuditEvent.Action"/> for a dashboard worker kill.</summary>
internal const string KillWorkerAction = "dashboard-kill-worker";
private const string UnauthorizedMessage = "Sign in as an Admin to close sessions or kill workers.";
private const string KillReason = "dashboard-admin-kill";
@@ -46,6 +62,8 @@ public sealed class DashboardSessionAdminService(
{
if (!CanManage(user))
{
await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Denied, "unauthorized", cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail(UnauthorizedMessage);
}
@@ -68,6 +86,15 @@ public sealed class DashboardSessionAdminService(
ResolveRemoteAddress(),
result.AlreadyClosed);
await WriteAuditAsync(
user,
CloseSessionAction,
sessionId,
AuditOutcome.Success,
result.AlreadyClosed ? "alreadyClosed" : "closed",
cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Success(
result.AlreadyClosed
? $"Session {sessionId} was already closed."
@@ -75,6 +102,8 @@ public sealed class DashboardSessionAdminService(
}
catch (SessionManagerException exception) when (exception.ErrorCode == SessionManagerErrorCode.SessionNotFound)
{
await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Failure, "not-found", cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail($"Session {sessionId} was not found.");
}
catch (SessionManagerException exception)
@@ -84,6 +113,8 @@ public sealed class DashboardSessionAdminService(
"Dashboard admin {Actor} close failed for session {SessionId}.",
actor,
sessionId);
await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Failure, exception.Message, cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail(
$"Close failed: {exception.Message}");
}
@@ -98,6 +129,8 @@ public sealed class DashboardSessionAdminService(
"Dashboard admin {Actor} close failed unexpectedly for session {SessionId}.",
actor,
sessionId);
await WriteAuditAsync(user, CloseSessionAction, sessionId, AuditOutcome.Failure, "unexpected", cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail(
$"Close failed unexpectedly for session {sessionId}. See the gateway log for details.");
}
@@ -111,6 +144,8 @@ public sealed class DashboardSessionAdminService(
{
if (!CanManage(user))
{
await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Denied, "unauthorized", cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail(UnauthorizedMessage);
}
@@ -133,6 +168,15 @@ public sealed class DashboardSessionAdminService(
ResolveRemoteAddress(),
result.AlreadyClosed);
await WriteAuditAsync(
user,
KillWorkerAction,
sessionId,
AuditOutcome.Success,
result.AlreadyClosed ? "alreadyClosed" : "killed",
cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Success(
result.AlreadyClosed
? $"Session {sessionId} was already closed."
@@ -140,6 +184,8 @@ public sealed class DashboardSessionAdminService(
}
catch (SessionManagerException exception) when (exception.ErrorCode == SessionManagerErrorCode.SessionNotFound)
{
await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Failure, "not-found", cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail($"Session {sessionId} was not found.");
}
catch (SessionManagerException exception)
@@ -149,6 +195,8 @@ public sealed class DashboardSessionAdminService(
"Dashboard admin {Actor} kill failed for session {SessionId}.",
actor,
sessionId);
await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Failure, exception.Message, cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail(
$"Kill failed: {exception.Message}");
}
@@ -163,6 +211,8 @@ public sealed class DashboardSessionAdminService(
"Dashboard admin {Actor} kill failed unexpectedly for session {SessionId}.",
actor,
sessionId);
await WriteAuditAsync(user, KillWorkerAction, sessionId, AuditOutcome.Failure, "unexpected", cancellationToken)
.ConfigureAwait(false);
return DashboardSessionAdminResult.Fail(
$"Kill failed unexpectedly for session {sessionId}. See the gateway log for details.");
}
@@ -177,4 +227,51 @@ public sealed class DashboardSessionAdminService(
{
return httpContextAccessor.HttpContext?.Connection.RemoteIpAddress?.ToString();
}
/// <summary>
/// Writes a canonical <see cref="AuditEvent"/> for a dashboard session-admin action through the
/// best-effort <see cref="IAuditWriter"/> (failures are swallowed/logged by the writer, so this
/// never throws and never masks the operation result). <see cref="AuditEvent.Actor"/> is the LDAP
/// operator, <see cref="AuditEvent.Target"/> the session id, and <paramref name="detail"/> is wrapped
/// as the <c>detail</c> field of the JSON extension bag.
/// </summary>
private async Task WriteAuditAsync(
ClaimsPrincipal user,
string action,
string sessionId,
AuditOutcome outcome,
string? detail,
CancellationToken cancellationToken)
{
AuditEvent auditEvent = new()
{
EventId = Guid.NewGuid(),
OccurredAtUtc = DateTimeOffset.UtcNow,
Actor = ResolveActor(user),
Action = action,
Outcome = outcome,
Category = SessionAdminCategory,
Target = sessionId,
SourceNode = ResolveRemoteAddress(),
CorrelationId = ParseCorrelationId(),
DetailsJson = WrapDetail(detail),
};
await auditWriter.WriteAsync(auditEvent, cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Derives a correlation id from the request trace identifier when it is a well-formed GUID;
/// otherwise null (the default <c>HttpContext.TraceIdentifier</c> is the connection:request form,
/// not a GUID, so it correlates to null rather than fabricating one).
/// </summary>
private Guid? ParseCorrelationId() =>
Guid.TryParse(httpContextAccessor.HttpContext?.TraceIdentifier, out Guid correlationId)
? correlationId
: null;
private static string? WrapDetail(string? detail) =>
detail is null
? null
: JsonSerializer.Serialize(new Dictionary<string, string> { ["detail"] = detail });
}
@@ -273,7 +273,8 @@ public sealed class DashboardSnapshotService : IDashboardSnapshotService
Constraints: ApiKeyConstraintSerializer.Deserialize(key.ConstraintsJson),
CreatedUtc: key.CreatedUtc,
LastUsedUtc: key.LastUsedUtc,
RevokedUtc: key.RevokedUtc))
RevokedUtc: key.RevokedUtc,
ExpiresUtc: key.ExpiresUtc))
.ToArray();
Volatile.Write(ref _apiKeySummaries, summaries);
@@ -10,6 +10,16 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// string (used by WebSocket upgrade requests that can't carry custom headers)
/// and validating it via <see cref="HubTokenService"/>.
/// </summary>
/// <remarks>
/// Carrying the token in the <c>?access_token=</c> query string is the standard SignalR pattern:
/// the WebSocket upgrade handshake cannot attach a custom <c>Authorization</c> header, so the JS
/// client appends the token to the URL. Because a query string is far easier to capture than a
/// header (proxy access logs, browser history, <c>Referer</c>), this value must NEVER be
/// request-logged. Serilog HTTP request logging is intentionally not enabled; if it is ever added,
/// scrub the <c>access_token</c> query parameter before the URL reaches a sink. The short token
/// lifetime (<see cref="HubTokenService.TokenLifetime"/>) is the backstop that bounds exposure of a
/// leaked token.
/// </remarks>
public sealed class HubTokenAuthenticationHandler : AuthenticationHandler<AuthenticationSchemeOptions>
{
private readonly HubTokenService _tokens;
@@ -26,7 +26,15 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
public sealed class HubTokenService
{
private const string ProtectorPurpose = "ZB.MOM.WW.MxGateway.Dashboard.HubToken.v1";
private static readonly TimeSpan TokenLifetime = TimeSpan.FromMinutes(30);
// Hub bearer tokens are single-purpose, data-protection-encrypted, and NOT server-side
// revocable. A short lifetime bounds the exposure window of a token captured from a proxy
// or log after logout (the cookie is cleared on logout, but outstanding tokens are not), and
// bounds how long a stale role set survives a role change. Five minutes is transparent to
// clients because DashboardHubConnectionFactory mints a fresh token on every (re)connect;
// see docs/GatewayDashboardDesign.md. Heavier jti-denylist revocation is deliberately
// deferred until per-session hub ACLs land (SEC-25), when tokens gain session binding.
internal static readonly TimeSpan TokenLifetime = TimeSpan.FromMinutes(5);
private readonly ITimeLimitedDataProtector _protector;
@@ -41,14 +49,24 @@ public sealed class HubTokenService
/// <summary>Issues a bearer token carrying the user's identity and roles.</summary>
/// <param name="user">The claims principal representing the user.</param>
/// <returns>The data-protected bearer token string.</returns>
public string Issue(ClaimsPrincipal user)
public string Issue(ClaimsPrincipal user) => Issue(user, TokenLifetime);
/// <summary>
/// Issues a bearer token that expires after the supplied lifetime. Test seam so a caller can
/// mint an already-expired token deterministically without wall-clock delay; production callers
/// use <see cref="Issue(ClaimsPrincipal)"/> and get <see cref="TokenLifetime"/>.
/// </summary>
/// <param name="user">The claims principal representing the user.</param>
/// <param name="lifetime">The lifetime applied to the token; a non-positive value yields an already-expired token.</param>
/// <returns>The data-protected bearer token string.</returns>
internal string Issue(ClaimsPrincipal user, TimeSpan lifetime)
{
ArgumentNullException.ThrowIfNull(user);
HubTokenPayload payload = new(
user.Identity?.Name,
user.FindFirstValue(ClaimTypes.NameIdentifier),
[.. user.FindAll(ClaimTypes.Role).Select(c => c.Value)]);
return _protector.Protect(JsonSerializer.Serialize(payload), TokenLifetime);
return _protector.Protect(JsonSerializer.Serialize(payload), lifetime);
}
/// <summary>Validates a token and returns the equivalent claims principal; null when invalid or expired.</summary>
@@ -41,6 +41,9 @@ public static class GatewayApplication
app.UseStaticFiles();
app.UseAuthentication();
app.UseAuthorization();
// SEC-11: the rate limiter must run after routing has selected the endpoint (so its
// RequireRateLimiting policy metadata is visible) and before the endpoint executes.
app.UseRateLimiter();
app.UseAntiforgery();
app.MapGatewayEndpoints();
@@ -68,6 +71,7 @@ public static class GatewayApplication
builder.Services.AddGatewayConfiguration(builder.Configuration);
builder.Services.AddSqliteAuthStore(builder.Configuration);
builder.Services.AddGatewayGrpcAuthorization();
AddLoginRateLimiter(builder);
builder.Services.AddHealthChecks()
.AddTypeActivatedCheck<AuthStoreHealthCheck>(
"auth-store",
@@ -101,6 +105,26 @@ public static class GatewayApplication
return builder;
}
// SEC-11: register the named fixed-window rate-limiter policy applied to POST /auth/login. The
// limit knobs are read from MxGateway:Security at startup; the per-request partition is keyed on
// the remote IP (see DashboardEndpointRouteBuilderExtensions.GetLoginRateLimitPartition).
private static void AddLoginRateLimiter(WebApplicationBuilder builder)
{
SecurityOptions security =
builder.Configuration.GetSection(SecurityOptions.SectionName).Get<SecurityOptions>()
?? new SecurityOptions();
builder.Services.AddRateLimiter(options =>
{
options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
options.AddPolicy(
DashboardEndpointRouteBuilderExtensions.LoginRateLimiterPolicy,
httpContext => DashboardEndpointRouteBuilderExtensions.GetLoginRateLimitPartition(
httpContext,
security));
});
}
private static void ConfigureSelfSignedTls(WebApplicationBuilder builder)
{
if (!Security.Tls.KestrelTlsInspector.RequiresGeneratedCertificate(builder.Configuration))
@@ -946,6 +946,7 @@ public sealed class MxAccessGatewayService(
WorkerClientErrorCode.GatewayShutdown => StatusCode.Cancelled,
WorkerClientErrorCode.InvalidState => StatusCode.FailedPrecondition,
WorkerClientErrorCode.ProtocolViolation => StatusCode.Internal,
WorkerClientErrorCode.CommandTooLarge => StatusCode.ResourceExhausted,
_ => StatusCode.Unavailable,
};
@@ -5,6 +5,13 @@ namespace ZB.MOM.WW.MxGateway.Server.Grpc;
public sealed class MxAccessGrpcRequestValidator
{
// Upper bound on a single DrainEvents request. DrainEvents is a diagnostics RPC that returns
// buffered events in one non-streaming reply, so an unbounded max_events could pack the whole
// queue into a session-killing frame (IPC-04). The worker independently caps each reply at its
// own MaxDrainEventsPerReply; this public bound rejects an obviously-abusive request loudly at
// the boundary. max_events = 0 is allowed and means "the worker's default batch cap".
private const uint MaxDrainEventsPerRequest = 10_000;
/// <summary>Validates an open session request.</summary>
/// <param name="request">The request to validate.</param>
public void ValidateOpenSession(OpenSessionRequest request)
@@ -69,6 +76,14 @@ public sealed class MxAccessGrpcRequestValidator
throw InvalidArgument(
$"Command kind {command.Kind} requires payload {expectedPayload} but received {command.PayloadCase}.");
}
// The payload case now matches the kind, so command.DrainEvents is non-null here.
if (command.Kind is MxCommandKind.DrainEvents && command.DrainEvents.MaxEvents > MaxDrainEventsPerRequest)
{
throw InvalidArgument(
$"DrainEvents max_events ({command.DrainEvents.MaxEvents}) must not exceed {MaxDrainEventsPerRequest}; "
+ "use 0 to request the worker default batch cap.");
}
}
private static MxCommand.PayloadOneofCase ExpectedPayload(MxCommandKind kind)
@@ -351,7 +351,11 @@ public sealed class GatewayMetrics : IDisposable
_heartbeatFailures++;
}
_heartbeatFailuresCounter.Add(1, new KeyValuePair<string, object?>("session_id", sessionId));
// Exported without a session_id tag: per-session attribution is unbounded cardinality
// (every session ever opened mints a new exporter time series, bloating Prometheus/OTLP
// storage on long-running gateways). Per-session detail stays available via the dashboard
// snapshot and the log scope; the exported counter tracks only the aggregate.
_heartbeatFailuresCounter.Add(1);
}
/// <summary>
@@ -67,6 +67,7 @@ public sealed class ApiKeyAdminCliRunner(ApiKeyAdminCommands commands)
Required(command.DisplayName),
command.Scopes,
ApiKeyConstraintSerializer.Serialize(command.Constraints),
command.ExpiresUtc,
remoteAddress: null,
cancellationToken)
.ConfigureAwait(false);
@@ -134,8 +135,16 @@ public sealed class ApiKeyAdminCliRunner(ApiKeyAdminCommands commands)
foreach (ApiKeyAdminListedKey key in result.Keys)
{
string revoked = key.RevokedUtc is null ? "active" : "revoked";
await output.WriteLineAsync($"{key.KeyId}\t{key.DisplayName}\t{revoked}\t{string.Join(',', key.Scopes)}")
string status = key.RevokedUtc is not null
? "revoked"
: key.ExpiresUtc is { } expiresAt && expiresAt <= DateTimeOffset.UtcNow
? "expired"
: "active";
string expiry = key.ExpiresUtc is { } expires
? expires.ToUniversalTime().ToString("u", System.Globalization.CultureInfo.InvariantCulture)
: "-";
await output.WriteLineAsync(
$"{key.KeyId}\t{key.DisplayName}\t{status}\t{expiry}\t{string.Join(',', key.Scopes)}")
.ConfigureAwait(false);
}
}
@@ -150,7 +159,8 @@ public sealed class ApiKeyAdminCliRunner(ApiKeyAdminCommands commands)
Constraints: ApiKeyConstraintSerializer.Deserialize(key.ConstraintsJson),
CreatedUtc: key.CreatedUtc,
LastUsedUtc: key.LastUsedUtc,
RevokedUtc: key.RevokedUtc);
RevokedUtc: key.RevokedUtc,
ExpiresUtc: key.ExpiresUtc);
}
private static string Required(string? value)
@@ -8,4 +8,5 @@ public sealed record ApiKeyAdminCommand(
string? KeyId,
string? DisplayName,
IReadOnlySet<string> Scopes,
ApiKeyConstraints Constraints);
ApiKeyConstraints Constraints,
DateTimeOffset? ExpiresUtc = null);
@@ -82,9 +82,11 @@ public static class ApiKeyAdminCommandLineParser
string? displayName = GetOption(options, "display-name");
IReadOnlySet<string> scopes = ParseScopes(GetOption(options, "scopes"));
ApiKeyConstraints constraints;
DateTimeOffset? expiresUtc;
try
{
constraints = ParseConstraints(options);
expiresUtc = ParseExpiry(GetOption(options, "expires"));
}
catch (FormatException exception)
{
@@ -111,7 +113,8 @@ public static class ApiKeyAdminCommandLineParser
KeyId: keyId,
DisplayName: displayName,
Scopes: scopes,
Constraints: constraints));
Constraints: constraints,
ExpiresUtc: expiresUtc));
}
private static bool TryParseKind(string value, out ApiKeyAdminCommandKind kind)
@@ -233,6 +236,42 @@ public static class ApiKeyAdminCommandLineParser
ReadHistorizedOnly: HasFlag(options, "read-historized-only"));
}
// Parses the optional --expires value into an absolute UTC expiry (SEC-10). Accepts a relative
// "<N>d"/"<N>h" duration from now (operator-friendly) or an absolute ISO-8601 instant/date
// (assumed UTC). Null/blank means no expiry — expiry stays opt-in, preserving prior behaviour.
private static DateTimeOffset? ParseExpiry(string? value)
{
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
string trimmed = value.Trim();
char unit = char.ToLowerInvariant(trimmed[^1]);
if (unit is 'd' or 'h'
&& int.TryParse(
trimmed[..^1],
System.Globalization.NumberStyles.None,
System.Globalization.CultureInfo.InvariantCulture,
out int amount))
{
TimeSpan offset = unit == 'd' ? TimeSpan.FromDays(amount) : TimeSpan.FromHours(amount);
return DateTimeOffset.UtcNow + offset;
}
if (DateTimeOffset.TryParse(
trimmed,
System.Globalization.CultureInfo.InvariantCulture,
System.Globalization.DateTimeStyles.AssumeUniversal | System.Globalization.DateTimeStyles.AdjustToUniversal,
out DateTimeOffset parsed))
{
return parsed;
}
throw new FormatException(
"--expires must be a relative duration ('<N>d' or '<N>h') or an absolute ISO-8601 UTC timestamp.");
}
private static int? ParseNullableInt(string? value)
{
if (string.IsNullOrWhiteSpace(value))
@@ -8,4 +8,5 @@ public sealed record ApiKeyAdminListedKey(
ApiKeyConstraints Constraints,
DateTimeOffset CreatedUtc,
DateTimeOffset? LastUsedUtc,
DateTimeOffset? RevokedUtc);
DateTimeOffset? RevokedUtc,
DateTimeOffset? ExpiresUtc = null);
@@ -1,4 +1,6 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.Audit;
using ZB.MOM.WW.Auth.Abstractions.ApiKeys;
@@ -6,6 +8,7 @@ using ZB.MOM.WW.Auth.ApiKeys;
using ZB.MOM.WW.Auth.ApiKeys.Admin;
using ZB.MOM.WW.Auth.ApiKeys.DependencyInjection;
using ZB.MOM.WW.Auth.ApiKeys.Sqlite;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Security.Audit;
namespace ZB.MOM.WW.MxGateway.Server.Security.Authentication;
@@ -66,6 +69,34 @@ public static class AuthStoreServiceCollectionExtensions
// migrator and the migration hosted service.
services.AddZbApiKeyAuth(effectiveConfig, AuthenticationSectionPath);
// SEC-08 hot-path decorators. Every gRPC call previously did a SQLite read plus a
// last_used_utc WRITE via IApiKeyVerifier.VerifyAsync (the library verifier couples the
// mark into verification). Two gateway-side decorators cut that cost without editing the
// external library:
// * CoalescingMarkApiKeyStore wraps the library's IApiKeyStore and coalesces the
// last_used write to at most one per key per MxGateway:Security window (default 60s),
// so the library verifier's per-call mark no longer churns the WAL.
// * CachingApiKeyVerifier wraps the library's IApiKeyVerifier with a short-TTL memory
// cache (MxGateway:Security:ApiKeyVerificationCacheSeconds, default 15s) keyed on a
// SHA-256 hash of the presented token, so a cache hit skips the store read entirely.
// Invalidation of a cached verification on a gateway-initiated revoke/rotate/delete is
// wired at the dashboard admin call site (DashboardApiKeyManagementService) via
// IApiKeyCacheInvalidator; the short TTL is the backstop for out-of-band mutations.
// Read the security knobs straight off IConfiguration rather than IOptions<GatewayOptions>:
// touching IOptions<GatewayOptions>.Value would force the whole-options validation pipeline
// (which needs IHostEnvironment) at auth-store resolution, breaking the bare-DI auth tests.
// GatewayOptions is still validated at startup, which covers these same values.
SecurityOptions security = configuration.GetSection(SecurityOptions.SectionName).Get<SecurityOptions>()
?? new SecurityOptions();
services.AddMemoryCache();
DecorateSingleton<IApiKeyStore>(
services,
(sp, inner) => new CoalescingMarkApiKeyStore(
inner,
security,
sp.GetService<TimeProvider>() ?? TimeProvider.System));
DecorateVerifierWithCache(services, security);
services.AddSingleton(sp =>
new SqliteCanonicalAuditStore(sp.GetRequiredService<AuthSqliteConnectionFactory>()));
// Resolve the logger defensively: the production host always registers ILogger<T>, but the
@@ -103,4 +134,58 @@ public static class AuthStoreServiceCollectionExtensions
return services;
}
/// <summary>
/// Replaces the last registration of <typeparamref name="TService"/> with a singleton that wraps
/// it. The wrapped (inner) service is created once, preserving singleton semantics. Used to layer
/// the SEC-08 store decorator over the external library's registration without editing it.
/// </summary>
private static void DecorateSingleton<TService>(
IServiceCollection services,
Func<IServiceProvider, TService, TService> decorator)
where TService : class
{
Func<IServiceProvider, TService> innerFactory = CaptureInnerFactory<TService>(services);
services.AddSingleton(sp => decorator(sp, innerFactory(sp)));
}
// Wraps the library's IApiKeyVerifier with CachingApiKeyVerifier and exposes the same instance
// as IApiKeyCacheInvalidator so admin call sites can drop cached verifications on revoke/rotate.
private static void DecorateVerifierWithCache(IServiceCollection services, SecurityOptions security)
{
Func<IServiceProvider, IApiKeyVerifier> innerFactory = CaptureInnerFactory<IApiKeyVerifier>(services);
services.AddSingleton(sp => new CachingApiKeyVerifier(
innerFactory(sp),
sp.GetRequiredService<IMemoryCache>(),
security));
services.AddSingleton<IApiKeyVerifier>(sp => sp.GetRequiredService<CachingApiKeyVerifier>());
services.AddSingleton<IApiKeyCacheInvalidator>(sp => sp.GetRequiredService<CachingApiKeyVerifier>());
}
// Removes the current last registration of TService and returns a factory that materialises that
// original implementation exactly once when first resolved. The removed descriptor may be a
// type, factory or instance registration.
private static Func<IServiceProvider, TService> CaptureInnerFactory<TService>(IServiceCollection services)
where TService : class
{
ServiceDescriptor descriptor = services.LastOrDefault(d => d.ServiceType == typeof(TService))
?? throw new InvalidOperationException(
$"No existing registration for {typeof(TService)} to decorate; register the library provider first.");
services.Remove(descriptor);
if (descriptor.ImplementationInstance is TService instance)
{
return _ => instance;
}
if (descriptor.ImplementationFactory is not null)
{
return sp => (TService)descriptor.ImplementationFactory(sp);
}
Type implementationType = descriptor.ImplementationType
?? throw new InvalidOperationException(
$"Registration for {typeof(TService)} has no implementation type, factory or instance to decorate.");
return sp => (TService)ActivatorUtilities.CreateInstance(sp, implementationType);
}
}
@@ -0,0 +1,159 @@
using System.Collections.Concurrent;
using System.Security.Cryptography;
using System.Text;
using Microsoft.Extensions.Caching.Memory;
using ZB.MOM.WW.Auth.Abstractions.ApiKeys;
using ZB.MOM.WW.MxGateway.Server.Configuration;
namespace ZB.MOM.WW.MxGateway.Server.Security.Authentication;
/// <summary>
/// Lets gateway-side admin call sites drop a cached API-key verification when they revoke, rotate
/// or delete a key, so a mutation applied through the running gateway process takes effect
/// immediately rather than after the cache TTL elapses.
/// </summary>
public interface IApiKeyCacheInvalidator
{
/// <summary>Removes every cached verification for the given key id.</summary>
/// <param name="keyId">The key id whose cached verifications should be dropped.</param>
void Invalidate(string keyId);
}
/// <summary>
/// An <see cref="IApiKeyVerifier"/> decorator that caches successful verifications for a short,
/// configurable TTL (<see cref="SecurityOptions.ApiKeyVerificationCacheSeconds"/>), keyed on a
/// SHA-256 hash of the presented token. A cache hit returns the cached result without calling the
/// inner (library) verifier, so it avoids both the per-call SQLite read and the <c>last_used</c>
/// write the library verifier couples into <see cref="IApiKeyVerifier.VerifyAsync"/>.
/// </summary>
/// <remarks>
/// <para>
/// Only successful verifications are cached. Failures and unparseable headers always fall through
/// to the inner verifier — caching a failure risks pinning a transiently-wrong negative, and the
/// SEC-11 per-peer failure counter (not this cache) is what bounds brute-force cost.
/// </para>
/// <para>
/// The cache key is the hex SHA-256 of the presented token (which embeds both the key id and the
/// secret). Hashing means the plaintext secret is never stored in memory. The hash is a cache index
/// only; the authentic constant-time secret comparison remains inside the inner verifier, reached
/// on every cache miss.
/// </para>
/// <para>
/// Correctness on mutation is provided by two mechanisms: gateway-initiated revoke/rotate/delete
/// call <see cref="Invalidate"/> directly (see <c>DashboardApiKeyManagementService</c>), and the
/// short TTL is the backstop for out-of-band mutations (a direct DB edit, or a revoke issued by the
/// separate <c>apikey</c> CLI process, whose in-memory cache is not this process's cache).
/// </para>
/// </remarks>
public sealed class CachingApiKeyVerifier : IApiKeyVerifier, IApiKeyCacheInvalidator
{
private const string CacheKeyPrefix = "mxgw:apikeyverif:";
private readonly IApiKeyVerifier _inner;
private readonly IMemoryCache _cache;
private readonly TimeSpan _ttl;
// keyId -> set of cache keys currently held for that id, so a revoke/rotate can evict every
// secret variant (e.g. an old secret still within TTL after a rotate).
private readonly ConcurrentDictionary<string, ConcurrentDictionary<string, byte>> _keyIdIndex =
new(StringComparer.Ordinal);
/// <summary>Initializes a new instance of the <see cref="CachingApiKeyVerifier"/> class.</summary>
/// <param name="inner">The wrapped verifier (the library verifier) reached on a cache miss.</param>
/// <param name="cache">The shared memory cache.</param>
/// <param name="security">Security options carrying the verification-cache TTL.</param>
public CachingApiKeyVerifier(IApiKeyVerifier inner, IMemoryCache cache, SecurityOptions security)
: this(
inner,
cache,
TimeSpan.FromSeconds((security ?? throw new ArgumentNullException(nameof(security)))
.ApiKeyVerificationCacheSeconds))
{
}
// Test/explicit-TTL seam.
internal CachingApiKeyVerifier(IApiKeyVerifier inner, IMemoryCache cache, TimeSpan ttl)
{
ArgumentNullException.ThrowIfNull(inner);
ArgumentNullException.ThrowIfNull(cache);
_inner = inner;
_cache = cache;
_ttl = ttl;
}
/// <inheritdoc />
public async Task<ApiKeyVerification> VerifyAsync(string authorizationHeader, CancellationToken ct)
{
if (_ttl <= TimeSpan.Zero || !TryComputeCacheKey(authorizationHeader, out string cacheKey))
{
return await _inner.VerifyAsync(authorizationHeader, ct).ConfigureAwait(false);
}
if (_cache.TryGetValue(cacheKey, out ApiKeyVerification? cached) && cached is not null)
{
return cached;
}
ApiKeyVerification result = await _inner.VerifyAsync(authorizationHeader, ct).ConfigureAwait(false);
if (result.Succeeded && result.Identity is not null)
{
_cache.Set(cacheKey, result, new MemoryCacheEntryOptions
{
AbsoluteExpirationRelativeToNow = _ttl,
});
IndexCacheKey(result.Identity.KeyId, cacheKey);
}
return result;
}
/// <inheritdoc />
public void Invalidate(string keyId)
{
if (string.IsNullOrEmpty(keyId))
{
return;
}
if (_keyIdIndex.TryRemove(keyId, out ConcurrentDictionary<string, byte>? cacheKeys))
{
foreach (string cacheKey in cacheKeys.Keys)
{
_cache.Remove(cacheKey);
}
}
}
private void IndexCacheKey(string keyId, string cacheKey)
{
ConcurrentDictionary<string, byte> set = _keyIdIndex.GetOrAdd(
keyId,
static _ => new ConcurrentDictionary<string, byte>(StringComparer.Ordinal));
set.TryAdd(cacheKey, 0);
}
private static bool TryComputeCacheKey(string? authorizationHeader, out string cacheKey)
{
cacheKey = string.Empty;
if (string.IsNullOrEmpty(authorizationHeader))
{
return false;
}
ReadOnlySpan<char> header = authorizationHeader.AsSpan().Trim();
const string bearer = "Bearer ";
ReadOnlySpan<char> token = header.StartsWith(bearer, StringComparison.OrdinalIgnoreCase)
? header[bearer.Length..].Trim()
: header;
if (token.IsEmpty)
{
return false;
}
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(token.ToString()));
cacheKey = CacheKeyPrefix + Convert.ToHexString(hash);
return true;
}
}
@@ -0,0 +1,101 @@
using System.Collections.Concurrent;
using ZB.MOM.WW.Auth.Abstractions.ApiKeys;
using ZB.MOM.WW.MxGateway.Server.Configuration;
namespace ZB.MOM.WW.MxGateway.Server.Security.Authentication;
/// <summary>
/// An <see cref="IApiKeyStore"/> decorator that coalesces <see cref="MarkUsedAsync"/> writes to at
/// most one per key per <see cref="SecurityOptions.ApiKeyLastUsedCoalesceSeconds"/> window. Lookups
/// pass through unchanged.
/// </summary>
/// <remarks>
/// The library verifier couples the <c>last_used_utc</c> write into every successful verification
/// by calling <see cref="IApiKeyStore.MarkUsedAsync"/>. On the bulk-read hot path that turns a
/// per-RPC database write into the throughput ceiling. This decorator sits under the library
/// verifier: it forwards the first mark for a key, then drops subsequent marks until the window
/// elapses, so <c>last_used_utc</c> is refreshed at most once per key per window while the read
/// path stays correct. <c>last_used_utc</c> is a coarse staleness indicator, not an audit record
/// (audit rows are written separately), so bounded staleness of up to one window is acceptable.
/// </remarks>
public sealed class CoalescingMarkApiKeyStore : IApiKeyStore
{
private readonly IApiKeyStore _inner;
private readonly TimeSpan _window;
private readonly TimeProvider _clock;
// keyId -> UtcTicks of the last forwarded mark. Bounded by the number of API keys, which is
// small; no eviction needed.
private readonly ConcurrentDictionary<string, long> _lastMarkedTicks = new(StringComparer.Ordinal);
/// <summary>Initializes a new instance of the <see cref="CoalescingMarkApiKeyStore"/> class.</summary>
/// <param name="inner">The wrapped store.</param>
/// <param name="security">Security options carrying the coalescing window.</param>
/// <param name="clock">The time provider.</param>
public CoalescingMarkApiKeyStore(IApiKeyStore inner, SecurityOptions security, TimeProvider clock)
: this(
inner,
TimeSpan.FromSeconds((security ?? throw new ArgumentNullException(nameof(security)))
.ApiKeyLastUsedCoalesceSeconds),
clock)
{
}
// Test/explicit-window seam.
internal CoalescingMarkApiKeyStore(IApiKeyStore inner, TimeSpan window, TimeProvider clock)
{
ArgumentNullException.ThrowIfNull(inner);
ArgumentNullException.ThrowIfNull(clock);
_inner = inner;
_window = window;
_clock = clock;
}
/// <inheritdoc />
public Task<ApiKeyRecord?> FindByKeyIdAsync(string keyId, CancellationToken ct)
=> _inner.FindByKeyIdAsync(keyId, ct);
/// <inheritdoc />
public Task<ApiKeyRecord?> FindActiveByKeyIdAsync(string keyId, CancellationToken ct)
=> _inner.FindActiveByKeyIdAsync(keyId, ct);
/// <inheritdoc />
public Task MarkUsedAsync(string keyId, DateTimeOffset whenUtc, CancellationToken ct)
{
ArgumentNullException.ThrowIfNull(keyId);
if (_window <= TimeSpan.Zero)
{
return _inner.MarkUsedAsync(keyId, whenUtc, ct);
}
long now = _clock.GetUtcNow().UtcTicks;
while (true)
{
if (_lastMarkedTicks.TryGetValue(keyId, out long last))
{
if (now - last < _window.Ticks)
{
// Within the coalescing window — drop the write.
return Task.CompletedTask;
}
if (_lastMarkedTicks.TryUpdate(keyId, now, last))
{
return _inner.MarkUsedAsync(keyId, whenUtc, ct);
}
// Lost the race to another writer; re-evaluate.
continue;
}
if (_lastMarkedTicks.TryAdd(keyId, now))
{
return _inner.MarkUsedAsync(keyId, whenUtc, ct);
}
// Lost the race to another writer; re-evaluate.
}
}
}
@@ -1,3 +1,5 @@
using System.Collections.Concurrent;
using LibApiKeyIdentity = ZB.MOM.WW.Auth.Abstractions.ApiKeys.ApiKeyIdentity;
namespace ZB.MOM.WW.MxGateway.Server.Security.Authentication;
@@ -17,6 +19,36 @@ namespace ZB.MOM.WW.MxGateway.Server.Security.Authentication;
/// </remarks>
public static class GatewayApiKeyIdentityMapper
{
// SEC-08: memoize the constraints deserialization keyed by the raw JSON blob so the per-call
// JSON parse on the auth hot path collapses to a dictionary lookup. Distinct blobs are bounded
// by the number of API keys (small); ApiKeyConstraints is an immutable record, so a parsed
// instance is safe to share across callers. The cap is a defensive backstop against a pathological
// spread of distinct blobs — past it, we simply parse without caching rather than grow unbounded.
private const int MaxCachedConstraintBlobs = 1024;
private static readonly ConcurrentDictionary<string, ApiKeyConstraints> ConstraintCache =
new(StringComparer.Ordinal);
private static ApiKeyConstraints DeserializeConstraints(string? constraintsJson)
{
if (string.IsNullOrWhiteSpace(constraintsJson))
{
return ApiKeyConstraints.Empty;
}
if (ConstraintCache.TryGetValue(constraintsJson, out ApiKeyConstraints? cached))
{
return cached;
}
ApiKeyConstraints parsed = ApiKeyConstraintSerializer.Deserialize(constraintsJson);
if (ConstraintCache.Count < MaxCachedConstraintBlobs)
{
ConstraintCache.TryAdd(constraintsJson, parsed);
}
return parsed;
}
/// <summary>
/// Converts a shared API-key identity into the gateway identity, deserializing the opaque
/// constraints JSON into <see cref="ApiKeyConstraints"/>.
@@ -38,6 +70,6 @@ public static class GatewayApiKeyIdentityMapper
KeyPrefix: "mxgw",
DisplayName: identity.DisplayName,
Scopes: identity.Scopes,
Constraints: ApiKeyConstraintSerializer.Deserialize(constraintsJson));
Constraints: DeserializeConstraints(constraintsJson));
}
}
@@ -0,0 +1,152 @@
using System.Collections.Concurrent;
using ZB.MOM.WW.MxGateway.Server.Configuration;
namespace ZB.MOM.WW.MxGateway.Server.Security.Authorization;
/// <summary>
/// Cheap, in-process per-peer sliding-window failure counter for the gRPC auth path (SEC-11). It is
/// checked BEFORE the API-key verification store read and short-circuits a peer that has exceeded
/// <see cref="SecurityOptions.ApiKeyFailureLimit"/> failed attempts within
/// <see cref="SecurityOptions.ApiKeyFailureWindowSeconds"/>; a successful verification resets the
/// peer's counter.
/// </summary>
/// <remarks>
/// <para>
/// The peer key is the API key id when the presented token parses, falling back to the transport
/// peer address otherwise. Keying on key id (per the NAT caveat in <c>glauth.md</c>) means a single
/// abusive credential behind a shared NAT is throttled without locking out unrelated clients on the
/// same address.
/// </para>
/// <para>
/// The tracked-peer set is a bounded LRU (<see cref="SecurityOptions.ApiKeyFailureTrackedPeers"/>) so
/// a spray of unique peer keys cannot grow memory without limit. Successful peers are removed on
/// reset, so in steady state the map holds only peers with recent failures — the common success path
/// is a lock-free dictionary miss.
/// </para>
/// </remarks>
public sealed class ApiKeyFailureLimiter
{
private readonly int _limit;
private readonly long _windowTicks;
private readonly int _maxPeers;
private readonly TimeProvider _clock;
private readonly ConcurrentDictionary<string, PeerState> _peers = new(StringComparer.Ordinal);
/// <summary>Initializes a new instance of the <see cref="ApiKeyFailureLimiter"/> class.</summary>
/// <param name="security">Security options carrying the failure-limit knobs.</param>
/// <param name="clock">The time provider.</param>
public ApiKeyFailureLimiter(SecurityOptions security, TimeProvider clock)
: this(
(security ?? throw new ArgumentNullException(nameof(security))).ApiKeyFailureLimit,
TimeSpan.FromSeconds(security.ApiKeyFailureWindowSeconds),
security.ApiKeyFailureTrackedPeers,
clock)
{
}
// Test/explicit seam.
internal ApiKeyFailureLimiter(int limit, TimeSpan window, int maxPeers, TimeProvider clock)
{
ArgumentNullException.ThrowIfNull(clock);
_limit = limit;
_windowTicks = window.Ticks;
_maxPeers = maxPeers;
_clock = clock;
}
/// <summary>Returns whether the peer has reached the failure limit within the current window.</summary>
/// <param name="peer">The peer key (key id or peer address).</param>
/// <returns><see langword="true"/> when the peer should be short-circuited.</returns>
public bool IsBlocked(string peer)
{
ArgumentNullException.ThrowIfNull(peer);
if (_limit <= 0)
{
return false;
}
if (!_peers.TryGetValue(peer, out PeerState? state))
{
return false;
}
long now = _clock.GetUtcNow().UtcTicks;
lock (state)
{
Prune(state, now);
return state.FailureTicks.Count >= _limit;
}
}
/// <summary>Records a failed verification attempt for the peer.</summary>
/// <param name="peer">The peer key (key id or peer address).</param>
public void RecordFailure(string peer)
{
ArgumentNullException.ThrowIfNull(peer);
if (_limit <= 0)
{
return;
}
long now = _clock.GetUtcNow().UtcTicks;
PeerState state = _peers.GetOrAdd(peer, static _ => new PeerState());
lock (state)
{
Prune(state, now);
state.FailureTicks.Enqueue(now);
state.LastActivityTicks = now;
}
EvictIfOverCapacity();
}
/// <summary>Clears the peer's failure count after a successful verification.</summary>
/// <param name="peer">The peer key (key id or peer address).</param>
public void Reset(string peer)
{
ArgumentNullException.ThrowIfNull(peer);
_peers.TryRemove(peer, out _);
}
private void Prune(PeerState state, long now)
{
while (state.FailureTicks.Count > 0 && now - state.FailureTicks.Peek() >= _windowTicks)
{
state.FailureTicks.Dequeue();
}
}
private void EvictIfOverCapacity()
{
// Best-effort eviction: only runs when the map exceeds the cap (rare, since only peers with
// recent failures are tracked). Removes the least-recently-active peer. Racy under
// concurrency, which is acceptable for a bound rather than an exact policy.
while (_peers.Count > _maxPeers)
{
string? oldest = null;
long oldestTicks = long.MaxValue;
foreach (KeyValuePair<string, PeerState> entry in _peers)
{
long activity = Volatile.Read(ref entry.Value.LastActivityTicks);
if (activity < oldestTicks)
{
oldestTicks = activity;
oldest = entry.Key;
}
}
if (oldest is null || !_peers.TryRemove(oldest, out _))
{
break;
}
}
}
private sealed class PeerState
{
public Queue<long> FailureTicks { get; } = new();
public long LastActivityTicks;
}
}
@@ -15,7 +15,8 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
IApiKeyVerifier apiKeyVerifier,
GatewayGrpcScopeResolver scopeResolver,
IGatewayRequestIdentityAccessor identityAccessor,
IOptions<GatewayOptions> options) : Interceptor
IOptions<GatewayOptions> options,
ApiKeyFailureLimiter failureLimiter) : Interceptor
{
/// <inheritdoc />
public override async Task<TResponse> UnaryServerHandler<TRequest, TResponse>(
@@ -63,6 +64,19 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
string? authorizationHeader = context.RequestHeaders.GetValue("authorization");
// SEC-11: short-circuit a peer that has already failed too many times inside the sliding
// window BEFORE the verification store read, so online guessing cannot spend a SQLite read
// per attempt. The peer key prefers the presented key id over the transport address (NAT
// caveat). ResourceExhausted signals throttling without revealing whether any particular
// secret was valid.
string peerKey = ResolvePeerKey(authorizationHeader, context);
if (failureLimiter.IsBlocked(peerKey))
{
throw new RpcException(new Status(
StatusCode.ResourceExhausted,
"Too many authentication attempts. Try again later."));
}
// The shared verifier owns parse + pepper + lookup + revocation + constant-time compare,
// returning a discriminated failure rather than throwing. Every authentication failure maps
// to Unauthenticated with an opaque message; the client never learns which stage failed.
@@ -72,11 +86,16 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
if (!verification.Succeeded || verification.Identity is null)
{
failureLimiter.RecordFailure(peerKey);
throw new RpcException(new Status(
StatusCode.Unauthenticated,
"Missing or invalid API key."));
}
// Successful authentication clears the peer's failure count so a legitimate client that
// fat-fingered a few attempts is not penalised once it recovers.
failureLimiter.Reset(peerKey);
ApiKeyIdentity identity = GatewayApiKeyIdentityMapper.ToGatewayIdentity(verification.Identity);
string requiredScope = scopeResolver.ResolveRequiredScope(request);
@@ -89,4 +108,30 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
return identity;
}
// Resolves the failure-limiter partition key: the presented key id (token shape
// mxgw_<keyId>_<secret>) when the header parses, otherwise the transport peer address. Keying on
// key id throttles a single abusive credential without locking out co-located clients behind NAT.
private static string ResolvePeerKey(string? authorizationHeader, ServerCallContext context)
{
if (!string.IsNullOrWhiteSpace(authorizationHeader))
{
ReadOnlySpan<char> header = authorizationHeader.AsSpan().Trim();
const string bearer = "Bearer ";
ReadOnlySpan<char> token = header.StartsWith(bearer, StringComparison.OrdinalIgnoreCase)
? header[bearer.Length..].Trim()
: header;
// mxgw_<keyId>_<secret>: the key id is the second underscore-delimited segment. The
// secret may itself contain underscores, but the key id is unaffected.
string tokenText = token.ToString();
string[] parts = tokenText.Split('_');
if (parts.Length >= 3 && parts[0].Length > 0 && parts[1].Length > 0)
{
return "key:" + parts[1];
}
}
return "peer:" + context.Peer;
}
}
@@ -20,6 +20,7 @@ public sealed class GatewayGrpcScopeResolver
MxCommandRequest commandRequest => ResolveCommandScope(commandRequest.Command?.Kind ?? MxCommandKind.Unspecified),
AcknowledgeAlarmRequest => GatewayScopes.InvokeWrite,
StreamAlarmsRequest => GatewayScopes.EventsRead,
QueryActiveAlarmsRequest => GatewayScopes.EventsRead,
TestConnectionRequest or
GetLastDeployTimeRequest or
DiscoverHierarchyRequest or
@@ -19,6 +19,13 @@ public static class GrpcAuthorizationServiceCollectionExtensions
services.AddSingleton<GatewayGrpcScopeResolver>();
services.AddSingleton<IGatewayRequestIdentityAccessor, GatewayRequestIdentityAccessor>();
services.AddSingleton<IConstraintEnforcer, ConstraintEnforcer>();
// SEC-11 per-peer failure counter, checked before the verification store read. Bind the knobs
// from IConfiguration directly (not IOptions<GatewayOptions>) to avoid coupling this
// registration to the whole-options validation pipeline.
services.AddSingleton(sp => new ApiKeyFailureLimiter(
sp.GetRequiredService<IConfiguration>().GetSection(SecurityOptions.SectionName).Get<SecurityOptions>()
?? new SecurityOptions(),
sp.GetService<TimeProvider>() ?? TimeProvider.System));
services.AddSingleton<GatewayGrpcAuthorizationInterceptor>();
services
.AddOptions<global::Grpc.AspNetCore.Server.GrpcServiceOptions>()
@@ -163,7 +163,15 @@ public sealed class SelfSignedCertificateProvider
// temp file empty, harden its permissions, and only then write the PFX into
// the already-protected file. The temp path is in the same directory as the
// target so the Move is atomic and preserves the hardened DACL/mode.
string temp = path + ".tmp";
//
// The temp name carries a unique suffix rather than a fixed "<path>.tmp": two
// processes (or two parallel callers) generating to the same target must not
// collide on one temp file. On Windows a fixed name makes the second writer's
// File.Create/Move fail with "the process cannot access the file ... because it
// is being used by another process"; a unique name lets each generation stage
// independently, and the final atomic Move (last-writer-wins) still yields a
// valid, equivalent certificate at the shared path.
string temp = $"{path}.{Guid.NewGuid():N}.tmp";
using (File.Create(temp)) { }
HardenPermissions(temp);
@@ -24,6 +24,13 @@ public sealed class WorkerClient : IWorkerClient
private readonly WorkerFrameWriter _writer;
private readonly Channel<WorkerEnvelope> _outboundEnvelopes;
private readonly Channel<WorkerEvent> _events;
// Staging hand-off between the read loop and the dedicated event writer. The read loop writes
// here with a non-blocking TryWrite so a full consumer channel (_events) can never stall the read
// loop behind an event — replies and heartbeats keep flowing (GWC-04). Unbounded, but only fills
// during the bounded EventChannelFullModeTimeout window before EventWriteLoopAsync faults on a
// sustained backlog, after which the read loop stops.
private readonly Channel<WorkerEvent> _eventStaging;
private readonly ConcurrentDictionary<string, PendingCommand> _pendingCommands = new(StringComparer.Ordinal);
private readonly SemaphoreSlim _pendingCommandSlots;
private readonly CancellationTokenSource _stopCts = new();
@@ -35,6 +42,7 @@ public sealed class WorkerClient : IWorkerClient
private int _eventsReaderClaimed;
private Task? _readLoopTask;
private Task? _writeLoopTask;
private Task? _eventWriteLoopTask;
private Task? _heartbeatLoopTask;
private bool _workerStartRecorded;
private bool _disposed;
@@ -82,6 +90,14 @@ public sealed class WorkerClient : IWorkerClient
FullMode = BoundedChannelFullMode.Wait,
AllowSynchronousContinuations = false,
});
_eventStaging = Channel.CreateUnbounded<WorkerEvent>(
new UnboundedChannelOptions
{
// The read loop is the only writer; EventWriteLoopAsync is the only reader.
SingleReader = true,
SingleWriter = true,
AllowSynchronousContinuations = false,
});
_lastHeartbeatAt = _timeProvider.GetUtcNow();
}
@@ -144,6 +160,7 @@ public sealed class WorkerClient : IWorkerClient
MarkReady(readyEnvelope.WorkerReady);
_readLoopTask = Task.Run(ReadLoopAsync);
_eventWriteLoopTask = Task.Run(EventWriteLoopAsync);
_heartbeatLoopTask = Task.Run(HeartbeatLoopAsync);
}
@@ -187,7 +204,23 @@ public sealed class WorkerClient : IWorkerClient
try
{
await EnqueueAsync(CreateCommandEnvelope(correlationId, command), cancellationToken).ConfigureAwait(false);
WorkerEnvelope commandEnvelope = CreateCommandEnvelope(correlationId, command);
// Reject an oversized command at the enqueue boundary so only this correlation fails
// (ResourceExhausted) rather than the frame reaching the write loop and faulting the whole
// session (IPC-03). Command envelopes are the only gateway-authored outbound payload whose
// size the caller controls; checking here keeps a MessageTooLarge in the write loop a
// genuine desync signal.
int envelopeSize = commandEnvelope.CalculateSize();
if (envelopeSize > _connection.FrameOptions.MaxMessageBytes)
{
throw new WorkerClientException(
WorkerClientErrorCode.CommandTooLarge,
$"Worker command {method} serializes to {envelopeSize} bytes, exceeding the negotiated "
+ $"worker-frame maximum of {_connection.FrameOptions.MaxMessageBytes} bytes.");
}
await EnqueueAsync(commandEnvelope, cancellationToken).ConfigureAwait(false);
using CancellationTokenSource timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
Task timeoutTask = Task.Delay(timeout, timeoutCts.Token);
Task<WorkerCommandReply> replyTask = pendingCommand.Task;
@@ -328,6 +361,7 @@ public sealed class WorkerClient : IWorkerClient
KillOwnedProcess("Dispose");
_stopCts.Cancel();
_outboundEnvelopes.Writer.TryComplete();
_eventStaging.Writer.TryComplete();
_events.Writer.TryComplete();
CompletePendingCommands(
new WorkerClientException(
@@ -382,7 +416,7 @@ public sealed class WorkerClient : IWorkerClient
while (!_stopCts.IsCancellationRequested)
{
WorkerEnvelope envelope = await _reader.ReadAsync(_stopCts.Token).ConfigureAwait(false);
await DispatchEnvelopeAsync(envelope, _stopCts.Token).ConfigureAwait(false);
DispatchEnvelope(envelope);
}
}
catch (OperationCanceledException) when (_stopCts.IsCancellationRequested || IsTerminalState())
@@ -481,12 +515,14 @@ public sealed class WorkerClient : IWorkerClient
return true;
}
/// <summary>Routes received envelope to appropriate handler.</summary>
/// <summary>
/// Routes a received envelope to its handler. Every branch dispatches synchronously and
/// immediately — the event branch only stages the event for the dedicated writer — so a full
/// event channel can never delay a command reply, heartbeat, fault, or shutdown ack behind an
/// event backlog (GWC-04).
/// </summary>
/// <param name="envelope">The envelope to dispatch.</param>
/// <param name="cancellationToken">Cancellation token.</param>
private async Task DispatchEnvelopeAsync(
WorkerEnvelope envelope,
CancellationToken cancellationToken)
private void DispatchEnvelope(WorkerEnvelope envelope)
{
switch (envelope.BodyCase)
{
@@ -494,7 +530,7 @@ public sealed class WorkerClient : IWorkerClient
CompleteCommand(envelope);
break;
case WorkerEnvelope.BodyOneofCase.WorkerEvent:
await EnqueueWorkerEventAsync(envelope.WorkerEvent, cancellationToken).ConfigureAwait(false);
StageWorkerEvent(envelope.WorkerEvent);
break;
case WorkerEnvelope.BodyOneofCase.WorkerHeartbeat:
MarkHeartbeat(envelope.WorkerHeartbeat);
@@ -517,6 +553,45 @@ public sealed class WorkerClient : IWorkerClient
}
}
/// <summary>
/// Hands a received worker event to the dedicated event writer without blocking the read loop.
/// The staging channel is unbounded and this is the only writer, so <c>TryWrite</c> always
/// succeeds unless the channel has been completed during shutdown — in which case the event is
/// safely dropped because the client is closing. Backpressure and the sustained-overflow fault
/// are applied by <see cref="EventWriteLoopAsync"/> against the bounded consumer channel,
/// off the read loop (GWC-04).
/// </summary>
/// <param name="workerEvent">The event received from the worker.</param>
private void StageWorkerEvent(WorkerEvent workerEvent)
{
if (workerEvent.Event is not null)
{
_metrics?.EventReceived(SessionId, workerEvent.Event.Family.ToString());
}
_eventStaging.Writer.TryWrite(workerEvent);
}
/// <summary>
/// Drains staged worker events and applies the bounded-channel backpressure (and
/// sustained-overflow fault) on a dedicated task, so the timed <see cref="Channel"/> write
/// never runs on the read loop (GWC-04). Mirrors <see cref="WriteLoopAsync"/> for events.
/// </summary>
private async Task EventWriteLoopAsync()
{
try
{
await foreach (WorkerEvent workerEvent in
_eventStaging.Reader.ReadAllAsync(_stopCts.Token).ConfigureAwait(false))
{
await EnqueueWorkerEventAsync(workerEvent, _stopCts.Token).ConfigureAwait(false);
}
}
catch (OperationCanceledException) when (_stopCts.IsCancellationRequested || IsTerminalState())
{
}
}
/// <summary>
/// Enqueues a worker event for client consumption. The channel is
/// configured with <see cref="BoundedChannelFullMode.Wait"/>
@@ -536,11 +611,6 @@ public sealed class WorkerClient : IWorkerClient
WorkerEvent workerEvent,
CancellationToken cancellationToken)
{
if (workerEvent.Event is not null)
{
_metrics?.EventReceived(SessionId, workerEvent.Event.Family.ToString());
}
if (_events.Writer.TryWrite(workerEvent))
{
int queueDepth = Interlocked.Increment(ref _eventQueueDepth);
@@ -731,6 +801,7 @@ public sealed class WorkerClient : IWorkerClient
_stopCts.Cancel();
_outboundEnvelopes.Writer.TryComplete();
_eventStaging.Writer.TryComplete();
_events.Writer.TryComplete();
CompletePendingCommands(
new WorkerClientException(
@@ -764,6 +835,7 @@ public sealed class WorkerClient : IWorkerClient
_stopCts.Cancel();
_outboundEnvelopes.Writer.TryComplete(fault);
_eventStaging.Writer.TryComplete(fault);
_events.Writer.TryComplete(fault);
KillOwnedProcess(errorCode.ToString());
CompletePendingCommands(fault);
@@ -910,6 +982,10 @@ public sealed class WorkerClient : IWorkerClient
SupportedProtocolVersion = _connection.FrameOptions.ProtocolVersion,
Nonce = _connection.Nonce,
GatewayVersion = typeof(GatewayContractInfo).Assembly.GetName().Version?.ToString() ?? GatewayVersionFallback,
// Convey the negotiated worker-frame maximum so the worker adopts it instead of a
// hard-coded default (IPC-02). Sits above the public gRPC cap by the envelope reserve.
MaxFrameBytes = (uint)_connection.FrameOptions.MaxMessageBytes,
});
}
@@ -985,7 +1061,7 @@ public sealed class WorkerClient : IWorkerClient
/// <param name="cancellationToken">Cancellation token.</param>
private async Task WaitForBackgroundTasksAsync(CancellationToken cancellationToken)
{
Task[] tasks = new[] { _readLoopTask, _writeLoopTask, _heartbeatLoopTask }
Task[] tasks = new[] { _readLoopTask, _writeLoopTask, _eventWriteLoopTask, _heartbeatLoopTask }
.Where(task => task is not null)
.Cast<Task>()
.ToArray();
@@ -12,4 +12,9 @@ public enum WorkerClientErrorCode
GatewayShutdown,
WriteFailed,
PendingCommandLimitExceeded,
// The serialized command envelope exceeds the negotiated worker-frame maximum. Rejected at the
// enqueue boundary so only the offending command fails (mapped to ResourceExhausted) instead of
// the oversized frame reaching the write loop and faulting the whole session (IPC-03).
CommandTooLarge,
}
@@ -10,6 +10,15 @@ public sealed class WorkerFrameProtocolOptions
/// <summary>Default maximum message size in bytes (16 MB).</summary>
public const int DefaultMaxMessageBytes = 16 * 1024 * 1024;
/// <summary>
/// Byte margin the worker-frame (pipe) maximum must keep above the public gRPC message cap so a
/// gRPC payload accepted at the public boundary always fits inside one worker frame once wrapped
/// in a <c>WorkerEnvelope</c> (correlation id, timestamps, oneof framing). Without this headroom
/// the pipe max equals the gRPC max and a maximally-sized accepted request faults the whole
/// session on the outbound write (IPC-03). 64 KiB is far larger than the fixed envelope overhead.
/// </summary>
public const int EnvelopeOverheadReserveBytes = 64 * 1024;
/// <summary>
/// Initializes worker frame protocol options with a session ID.
/// </summary>
@@ -6,10 +6,10 @@
<ItemGroup>
<PackageReference Include="Grpc.AspNetCore" Version="2.76.0" />
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Auth.ApiKeys" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Auth.AspNetCore" Version="0.1.2" />
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.1.4" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.1.4" />
<PackageReference Include="ZB.MOM.WW.Auth.ApiKeys" Version="0.1.4" />
<PackageReference Include="ZB.MOM.WW.Auth.AspNetCore" Version="0.1.4" />
<PackageReference Include="ZB.MOM.WW.Audit" Version="0.1.0" />
<PackageReference Include="ZB.MOM.WW.Theme" Version="0.3.1" />
<PackageReference Include="ZB.MOM.WW.Configuration" Version="0.1.0" />
@@ -38,7 +38,7 @@
"ShutdownTimeoutSeconds": 10,
"HeartbeatIntervalSeconds": 5,
"HeartbeatGraceSeconds": 15,
"MaxMessageBytes": 16777216
"MaxMessageBytes": 16842752
},
"Sessions": {
"DefaultCommandTimeoutSeconds": 30,
@@ -14,7 +14,15 @@ public sealed class GatewayOptionsTests
GatewayOptions options = BindOptions(new Dictionary<string, string?>());
Assert.Equal(AuthenticationMode.ApiKey, options.Authentication.Mode);
Assert.Equal(@"C:\ProgramData\MxGateway\gateway-auth.db", options.Authentication.SqlitePath);
// SEC-01: the default is derived from CommonApplicationData (C:\ProgramData on Windows,
// /usr/share on Unix) rather than a Windows literal, so assert against the same derivation
// to stay platform-correct.
Assert.Equal(
Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData),
"MxGateway",
"gateway-auth.db"),
options.Authentication.SqlitePath);
Assert.Equal("MxGateway:ApiKeyPepper", options.Authentication.PepperSecretName);
Assert.True(options.Authentication.RunMigrationsOnStartup);
@@ -27,7 +35,8 @@ public sealed class GatewayOptionsTests
Assert.Equal(10, options.Worker.ShutdownTimeoutSeconds);
Assert.Equal(5, options.Worker.HeartbeatIntervalSeconds);
Assert.Equal(15, options.Worker.HeartbeatGraceSeconds);
Assert.Equal(16 * 1024 * 1024, options.Worker.MaxMessageBytes);
// 16 MiB public gRPC cap plus the 64 KiB worker-frame envelope reserve (IPC-03 headroom).
Assert.Equal((16 * 1024 * 1024) + (64 * 1024), options.Worker.MaxMessageBytes);
Assert.Equal(30, options.Sessions.DefaultCommandTimeoutSeconds);
Assert.Equal(64, options.Sessions.MaxSessions);
@@ -1,5 +1,6 @@
using Microsoft.Extensions.Options;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using LdapTransport = ZB.MOM.WW.Auth.Abstractions.Ldap.LdapTransport;
namespace ZB.MOM.WW.MxGateway.Tests.Configuration;
@@ -548,4 +549,272 @@ public sealed class GatewayOptionsValidatorTests
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Succeeded);
}
// -------------------------------------------------------------------------
// SEC-01: security-sensitive paths must be rooted (absolute)
// -------------------------------------------------------------------------
private static GatewayOptions CloneWithAuthentication(GatewayOptions source, AuthenticationOptions authentication)
=> new()
{
Authentication = authentication,
Ldap = source.Ldap,
Worker = source.Worker,
Sessions = source.Sessions,
Events = source.Events,
Dashboard = source.Dashboard,
Protocol = source.Protocol,
Alarms = source.Alarms,
Tls = source.Tls,
};
/// <summary>Verifies the default (CommonApplicationData-derived) auth DB path is rooted and passes validation.</summary>
[Fact]
public void Validate_Succeeds_WithDefaultRootedSqlitePath()
{
// The default AuthenticationOptions.SqlitePath is derived from CommonApplicationData,
// which is rooted on every host OS.
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, ValidOptions());
Assert.True(result.Succeeded);
}
/// <summary>Verifies a non-rooted <see cref="AuthenticationOptions.SqlitePath"/> fails validation.</summary>
[Fact]
public void Validate_Fails_WhenSqlitePathNotRooted()
{
GatewayOptions options = CloneWithAuthentication(
ValidOptions(),
new AuthenticationOptions { SqlitePath = "gateway-auth.db" });
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(
result.Failures!,
f => f.Contains("MxGateway:Authentication:SqlitePath") && f.Contains("rooted"));
}
/// <summary>Verifies a non-rooted <see cref="TlsOptions.SelfSignedCertPath"/> fails validation.</summary>
[Fact]
public void Validate_Fails_WhenSelfSignedCertPathNotRooted()
{
GatewayOptions options = CloneWithTls(
ValidOptions(),
new TlsOptions { SelfSignedCertPath = "gateway-selfsigned.pfx" });
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(
result.Failures!,
f => f.Contains("MxGateway:Tls:SelfSignedCertPath") && f.Contains("rooted"));
}
// -------------------------------------------------------------------------
// SEC-04: DisableLogin production guard
// -------------------------------------------------------------------------
private static GatewayOptions CloneWithDashboard(GatewayOptions source, DashboardOptions dashboard)
=> new()
{
Authentication = source.Authentication,
Ldap = source.Ldap,
Worker = source.Worker,
Sessions = source.Sessions,
Events = source.Events,
Dashboard = dashboard,
Protocol = source.Protocol,
Alarms = source.Alarms,
Tls = source.Tls,
};
/// <summary>Verifies <see cref="DashboardOptions.DisableLogin"/> set to <see langword="true"/> aborts startup in Production.</summary>
[Fact]
public void Validate_Fails_WhenDisableLoginTrueInProduction()
{
GatewayOptions options = CloneWithDashboard(
ValidOptions(),
new DashboardOptions { DisableLogin = true });
ValidateOptionsResult result = new GatewayOptionsValidator(isProduction: true).Validate(null, options);
Assert.True(result.Failed);
Assert.Contains(
result.Failures!,
f => f.Contains("MxGateway:Dashboard:DisableLogin") && f.Contains("Production"));
}
/// <summary>Verifies <see cref="DashboardOptions.DisableLogin"/> set to <see langword="true"/> is accepted outside Production.</summary>
[Fact]
public void Validate_Succeeds_WhenDisableLoginTrueInDevelopment()
{
GatewayOptions options = CloneWithDashboard(
ValidOptions(),
new DashboardOptions { DisableLogin = true });
ValidateOptionsResult result = new GatewayOptionsValidator(isProduction: false).Validate(null, options);
Assert.True(result.Succeeded);
}
// -------------------------------------------------------------------------
// SEC-06: LDAP plaintext transport production guard
// -------------------------------------------------------------------------
/// <summary>Verifies plaintext LDAP transport (None) aborts startup in Production.</summary>
[Fact]
public void Validate_Fails_WhenLdapTransportNoneInProduction()
{
// The class default LDAP options ship Transport=None + AllowInsecure=true (dev posture).
ValidateOptionsResult result = new GatewayOptionsValidator(isProduction: true).Validate(null, ValidOptions());
Assert.True(result.Failed);
Assert.Contains(
result.Failures!,
f => f.Contains("MxGateway:Ldap:Transport") && f.Contains("Production"));
}
/// <summary>Verifies plaintext LDAP transport (None) is accepted outside Production.</summary>
[Fact]
public void Validate_Succeeds_WhenLdapTransportNoneInDevelopment()
{
ValidateOptionsResult result = new GatewayOptionsValidator(isProduction: false).Validate(null, ValidOptions());
Assert.True(result.Succeeded);
}
/// <summary>Verifies secure LDAP transport (Ldaps) passes validation in Production.</summary>
[Fact]
public void Validate_Succeeds_WhenLdapTransportLdapsInProduction()
{
GatewayOptions options = CloneWithLdap(
ValidOptions(),
new LdapOptions { Enabled = true, Transport = LdapTransport.Ldaps, AllowInsecure = false });
ValidateOptionsResult result = new GatewayOptionsValidator(isProduction: true).Validate(null, options);
Assert.True(result.Succeeded);
}
// ---- SEC-11: MxGateway:Security validation ----
private static GatewayOptions WithSecurity(SecurityOptions security) => new() { Security = security };
/// <summary>Verifies the default security options pass validation.</summary>
[Fact]
public void Validate_Succeeds_WithDefaultSecurityOptions()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, WithSecurity(new SecurityOptions()));
Assert.True(result.Succeeded);
}
/// <summary>Verifies a zero verification-cache TTL is allowed (disables caching).</summary>
[Fact]
public void Validate_Succeeds_WhenVerificationCacheSecondsZero()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithSecurity(new SecurityOptions { ApiKeyVerificationCacheSeconds = 0 }));
Assert.True(result.Succeeded);
}
/// <summary>Verifies a negative verification-cache TTL fails validation.</summary>
[Fact]
public void Validate_Fails_WhenVerificationCacheSecondsNegative()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithSecurity(new SecurityOptions { ApiKeyVerificationCacheSeconds = -1 }));
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("ApiKeyVerificationCacheSeconds"));
}
/// <summary>Verifies a negative last-used coalesce window fails validation.</summary>
[Fact]
public void Validate_Fails_WhenLastUsedCoalesceSecondsNegative()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithSecurity(new SecurityOptions { ApiKeyLastUsedCoalesceSeconds = -5 }));
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("ApiKeyLastUsedCoalesceSeconds"));
}
/// <summary>Verifies a zero login rate-limit permit fails validation.</summary>
[Fact]
public void Validate_Fails_WhenLoginRateLimitPermitLimitZero()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithSecurity(new SecurityOptions { LoginRateLimitPermitLimit = 0 }));
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("LoginRateLimitPermitLimit"));
}
/// <summary>Verifies a zero login rate-limit window fails validation.</summary>
[Fact]
public void Validate_Fails_WhenLoginRateLimitWindowSecondsZero()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithSecurity(new SecurityOptions { LoginRateLimitWindowSeconds = 0 }));
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("LoginRateLimitWindowSeconds"));
}
/// <summary>Verifies a zero API-key failure limit fails validation.</summary>
[Fact]
public void Validate_Fails_WhenApiKeyFailureLimitZero()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithSecurity(new SecurityOptions { ApiKeyFailureLimit = 0 }));
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("ApiKeyFailureLimit"));
}
/// <summary>Verifies a zero tracked-peer cap fails validation.</summary>
[Fact]
public void Validate_Fails_WhenApiKeyFailureTrackedPeersZero()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithSecurity(new SecurityOptions { ApiKeyFailureTrackedPeers = 0 }));
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("ApiKeyFailureTrackedPeers"));
}
private static GatewayOptions WithWorkerAndProtocol(WorkerOptions worker, ProtocolOptions protocol)
{
GatewayOptions source = ValidOptions();
return new GatewayOptions
{
Authentication = source.Authentication,
Ldap = source.Ldap,
Worker = worker,
Sessions = source.Sessions,
Events = source.Events,
Dashboard = source.Dashboard,
Protocol = protocol,
Alarms = source.Alarms,
Tls = source.Tls,
};
}
/// <summary>
/// Verifies the default worker-frame maximum keeps the required envelope-overhead reserve above
/// the default public gRPC cap, so a stock configuration passes the IPC-03 headroom check.
/// </summary>
[Fact]
public void Validate_Succeeds_WhenWorkerFrameMaxHasEnvelopeHeadroom()
{
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(null, ValidOptions());
Assert.True(result.Succeeded);
}
/// <summary>
/// Verifies that a worker-frame maximum equal to the gRPC cap (zero headroom) fails validation:
/// a maximally-sized accepted gRPC payload would not fit one worker frame once wrapped in a
/// WorkerEnvelope, faulting the whole session on the outbound write (IPC-03).
/// </summary>
[Fact]
public void Validate_Fails_WhenWorkerFrameMaxEqualsGrpcMaxWithoutHeadroom()
{
const int grpcMax = 16 * 1024 * 1024;
ValidateOptionsResult result = new GatewayOptionsValidator().Validate(
null,
WithWorkerAndProtocol(
new WorkerOptions { MaxMessageBytes = grpcMax },
new ProtocolOptions { MaxGrpcMessageBytes = grpcMax }));
Assert.True(result.Failed);
Assert.Contains(result.Failures!, f => f.Contains("MaxMessageBytes") && f.Contains("reserve"));
}
}
@@ -1,5 +1,6 @@
using System.Text.Json;
using Google.Protobuf;
using Google.Protobuf.Reflection;
using ZB.MOM.WW.MxGateway.Contracts;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
@@ -7,6 +8,102 @@ namespace ZB.MOM.WW.MxGateway.Tests.Contracts;
public sealed class ClientProtoInputTests
{
/// <summary>
/// Guards the published client descriptor set against silent staleness (IPC-01). Every message
/// and field compiled into the in-process contract (which the build regenerates from the current
/// <c>.proto</c> sources) must appear in the committed protoset. A missing symbol means the
/// descriptor was not regenerated after a proto change; run
/// <c>scripts/publish-client-proto-inputs.ps1</c> and commit the refreshed protoset.
/// The check is semantic (symbol presence) rather than byte-wise, so it is independent of protoc
/// version and does not require protoc on the test runner.
/// </summary>
[Fact]
public void Descriptor_ContainsEveryContractMessageAndField()
{
DirectoryInfo repositoryRoot = FindRepositoryRoot();
string descriptorPath = Path.Combine(
repositoryRoot.FullName,
"clients",
"proto",
"descriptors",
"mxaccessgw-client-v1.protoset");
Assert.True(File.Exists(descriptorPath), $"Expected descriptor set '{descriptorPath}' to exist.");
FileDescriptorSet descriptorSet = FileDescriptorSet.Parser.ParseFrom(File.ReadAllBytes(descriptorPath));
HashSet<string> publishedMessages = new(StringComparer.Ordinal);
HashSet<string> publishedFields = new(StringComparer.Ordinal);
foreach (FileDescriptorProto file in descriptorSet.File)
{
foreach (DescriptorProto message in file.MessageType)
{
CollectPublishedSymbols(file.Package, message, publishedMessages, publishedFields);
}
}
List<string> missing = [];
foreach (FileDescriptor file in new[] { MxaccessGatewayReflection.Descriptor, MxaccessWorkerReflection.Descriptor })
{
foreach (MessageDescriptor message in file.MessageTypes)
{
CollectMissingContractSymbols(message, publishedMessages, publishedFields, missing);
}
}
Assert.True(
missing.Count == 0,
"Published client descriptor is stale; regenerate with scripts/publish-client-proto-inputs.ps1 and commit. "
+ "Missing symbols: "
+ string.Join(", ", missing));
}
private static void CollectPublishedSymbols(
string package,
DescriptorProto message,
HashSet<string> messages,
HashSet<string> fields)
{
string fullName = string.IsNullOrEmpty(package) ? message.Name : package + "." + message.Name;
messages.Add(fullName);
foreach (FieldDescriptorProto field in message.Field)
{
fields.Add(fullName + "/" + field.Name);
}
foreach (DescriptorProto nested in message.NestedType)
{
CollectPublishedSymbols(fullName, nested, messages, fields);
}
}
private static void CollectMissingContractSymbols(
MessageDescriptor message,
HashSet<string> publishedMessages,
HashSet<string> publishedFields,
List<string> missing)
{
if (!publishedMessages.Contains(message.FullName))
{
missing.Add(message.FullName);
}
foreach (FieldDescriptor field in message.Fields.InDeclarationOrder())
{
string key = message.FullName + "/" + field.Name;
if (!publishedFields.Contains(key))
{
missing.Add(key);
}
}
foreach (MessageDescriptor nested in message.NestedTypes)
{
CollectMissingContractSymbols(nested, publishedMessages, publishedFields, missing);
}
}
/// <summary>Verifies that the proto inputs manifest declares current protocol versions and existing source files.</summary>
[Fact]
public void Manifest_DeclaresCurrentProtocolVersionsAndExistingInputs()
@@ -1,6 +1,7 @@
using Microsoft.Extensions.Diagnostics.HealthChecks;
using ZB.MOM.WW.Auth.ApiKeys.Sqlite;
using ZB.MOM.WW.MxGateway.Server.Diagnostics;
using ZB.MOM.WW.MxGateway.Tests.Security.Authentication;
namespace ZB.MOM.WW.MxGateway.Tests.Diagnostics;
@@ -17,15 +18,15 @@ public sealed class AuthStoreHealthCheckTests
[Fact]
public async Task Healthy_WhenStoreReachable()
{
var path = Path.Combine(Path.GetTempPath(), $"authcheck-{Guid.NewGuid():N}.db");
try
{
var check = new AuthStoreHealthCheck(FactoryFor(path));
// Open a real SQLite file via the health check. TempDatabaseDirectory clears the
// Microsoft.Data.Sqlite connection pool on dispose before deleting the file; without
// that the pool keeps the .db handle open and the delete throws "used by another
// process" on Windows (a latent full-suite flake — the file opens fine on macOS).
using TempDatabaseDirectory directory = TempDatabaseDirectory.Create("authcheck");
var check = new AuthStoreHealthCheck(FactoryFor(directory.DatabasePath()));
var result = await check.CheckHealthAsync(new HealthCheckContext());
Assert.Equal(HealthStatus.Healthy, result.Status);
}
finally { if (File.Exists(path)) File.Delete(path); }
}
/// <summary>The health check reports unhealthy when the database path cannot be opened.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
@@ -128,11 +128,58 @@ public sealed class DashboardAuthorizationHandlerTests
Assert.False(context.HasSucceeded);
}
/// <summary>
/// Verifies that the anonymous-localhost bypass is read-only: a loopback request with
/// <c>AllowAnonymousLocalhost</c> on is denied the <see cref="DashboardAuthorizationRequirement.AdminOnly"/>
/// requirement, so destructive/admin surfaces are never reachable without a real Admin role.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task HandleAsync_AnonymousLocalhostAllowed_DoesNotSatisfyAdminPolicy()
{
AuthorizationHandlerContext context = await AuthorizeAsync(
new ClaimsPrincipal(new ClaimsIdentity()),
IPAddress.Loopback,
allowAnonymousLocalhost: true,
DashboardAuthorizationRequirement.AdminOnly);
Assert.False(context.HasSucceeded);
}
/// <summary>
/// Verifies that with authentication fully disabled the environment bypass stays read-only:
/// a remote request satisfies <see cref="DashboardAuthorizationRequirement.AnyDashboardRole"/>
/// (Viewer) but is denied <see cref="DashboardAuthorizationRequirement.AdminOnly"/>.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task HandleAsync_AuthenticationDisabled_GrantsViewerButNotAdmin()
{
AuthorizationHandlerContext viewerContext = await AuthorizeAsync(
new ClaimsPrincipal(new ClaimsIdentity()),
IPAddress.Parse("10.0.0.5"),
allowAnonymousLocalhost: false,
DashboardAuthorizationRequirement.AnyDashboardRole,
authenticationMode: AuthenticationMode.Disabled);
Assert.True(viewerContext.HasSucceeded);
AuthorizationHandlerContext adminContext = await AuthorizeAsync(
new ClaimsPrincipal(new ClaimsIdentity()),
IPAddress.Parse("10.0.0.5"),
allowAnonymousLocalhost: false,
DashboardAuthorizationRequirement.AdminOnly,
authenticationMode: AuthenticationMode.Disabled);
Assert.False(adminContext.HasSucceeded);
}
private static async Task<AuthorizationHandlerContext> AuthorizeAsync(
ClaimsPrincipal principal,
IPAddress remoteAddress,
bool allowAnonymousLocalhost,
DashboardAuthorizationRequirement requirement)
DashboardAuthorizationRequirement requirement,
AuthenticationMode authenticationMode = AuthenticationMode.ApiKey)
{
DefaultHttpContext httpContext = new();
httpContext.Connection.RemoteIpAddress = remoteAddress;
@@ -140,6 +187,10 @@ public sealed class DashboardAuthorizationHandlerTests
new HttpContextAccessor { HttpContext = httpContext },
Options.Create(new GatewayOptions
{
Authentication = new AuthenticationOptions
{
Mode = authenticationMode,
},
Dashboard = new DashboardOptions
{
AllowAnonymousLocalhost = allowAnonymousLocalhost,
@@ -0,0 +1,66 @@
using System.Net;
using System.Threading.RateLimiting;
using Microsoft.AspNetCore.Http;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Dashboard;
namespace ZB.MOM.WW.MxGateway.Tests.Gateway.Dashboard;
/// <summary>
/// SEC-11: the <c>POST /auth/login</c> fixed-window per-IP rate limiter. Exercised through the same
/// partition factory the production policy registers, so the test pins the real permit limit and
/// per-IP partitioning without standing up an HTTP server.
/// </summary>
public sealed class DashboardLoginRateLimitTests
{
/// <summary>A burst beyond the permit limit from one IP has its excess attempts rejected (HTTP 429 in the pipeline).</summary>
[Fact]
public void LoginRateLimiter_BurstBeyondPermitLimit_RejectsExcessFromSameIp()
{
SecurityOptions security = new()
{
LoginRateLimitPermitLimit = 3,
LoginRateLimitWindowSeconds = 60,
};
using PartitionedRateLimiter<HttpContext> limiter =
DashboardEndpointRouteBuilderExtensions.CreateLoginRateLimiter(security);
HttpContext context = ContextWithIp("10.0.0.7");
for (int i = 0; i < 3; i++)
{
using RateLimitLease lease = limiter.AttemptAcquire(context);
Assert.True(lease.IsAcquired);
}
using RateLimitLease rejected = limiter.AttemptAcquire(context);
Assert.False(rejected.IsAcquired);
}
/// <summary>Distinct client IPs are partitioned independently — one IP's burst does not starve another.</summary>
[Fact]
public void LoginRateLimiter_DistinctIps_PartitionedIndependently()
{
SecurityOptions security = new()
{
LoginRateLimitPermitLimit = 1,
LoginRateLimitWindowSeconds = 60,
};
using PartitionedRateLimiter<HttpContext> limiter =
DashboardEndpointRouteBuilderExtensions.CreateLoginRateLimiter(security);
using RateLimitLease first = limiter.AttemptAcquire(ContextWithIp("10.0.0.1"));
using RateLimitLease exhaustedForFirst = limiter.AttemptAcquire(ContextWithIp("10.0.0.1"));
using RateLimitLease second = limiter.AttemptAcquire(ContextWithIp("10.0.0.2"));
Assert.True(first.IsAcquired);
Assert.False(exhaustedForFirst.IsAcquired);
Assert.True(second.IsAcquired);
}
private static HttpContext ContextWithIp(string ip)
{
DefaultHttpContext context = new();
context.Connection.RemoteIpAddress = IPAddress.Parse(ip);
return context;
}
}
@@ -1,6 +1,8 @@
using System.Collections.Concurrent;
using System.Diagnostics.CodeAnalysis;
using System.Security.Claims;
using Microsoft.AspNetCore.Http;
using ZB.MOM.WW.Audit;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Dashboard;
using ZB.MOM.WW.MxGateway.Server.Sessions;
@@ -198,14 +200,91 @@ public sealed class DashboardSessionAdminServiceTests
Assert.False(string.IsNullOrWhiteSpace(result.Message));
}
private static DashboardSessionAdminService CreateService(ISessionManager sessionManager)
/// <summary>
/// Verifies that a successful close writes a canonical <c>dashboard-close-session</c>
/// <see cref="AuditEvent"/> — actor, session-id target, and Success outcome — to the
/// audit store, not only the operational log (SEC-12).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task CloseSessionAsync_AdminClose_WritesCanonicalAuditEvent()
{
FakeSessionManager sessionManager = new();
RecordingAuditWriter auditWriter = new();
DashboardSessionAdminService service = CreateService(sessionManager, auditWriter);
DashboardSessionAdminResult result = await service.CloseSessionAsync(
CreateUser(DashboardRoles.Admin),
"session-1",
CancellationToken.None);
Assert.True(result.Succeeded);
AuditEvent audit = Assert.Single(auditWriter.Events);
Assert.Equal(DashboardSessionAdminService.CloseSessionAction, audit.Action);
Assert.Equal(DashboardSessionAdminService.SessionAdminCategory, audit.Category);
Assert.Equal("session-1", audit.Target);
Assert.Equal("tester", audit.Actor);
Assert.Equal(AuditOutcome.Success, audit.Outcome);
}
/// <summary>
/// Verifies that a successful kill writes a canonical <c>dashboard-kill-worker</c>
/// <see cref="AuditEvent"/> to the audit store (SEC-12).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task KillWorkerAsync_AdminKill_WritesCanonicalAuditEvent()
{
FakeSessionManager sessionManager = new();
RecordingAuditWriter auditWriter = new();
DashboardSessionAdminService service = CreateService(sessionManager, auditWriter);
DashboardSessionAdminResult result = await service.KillWorkerAsync(
CreateUser(DashboardRoles.Admin),
"session-1",
CancellationToken.None);
Assert.True(result.Succeeded);
AuditEvent audit = Assert.Single(auditWriter.Events);
Assert.Equal(DashboardSessionAdminService.KillWorkerAction, audit.Action);
Assert.Equal("session-1", audit.Target);
Assert.Equal(AuditOutcome.Success, audit.Outcome);
}
/// <summary>
/// Verifies that an unauthorized (viewer) close attempt still writes a <c>Denied</c>
/// audit row, so rejected destructive attempts are durably recorded (SEC-12).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task CloseSessionAsync_ViewerDenied_WritesDeniedAuditEvent()
{
FakeSessionManager sessionManager = new();
RecordingAuditWriter auditWriter = new();
DashboardSessionAdminService service = CreateService(sessionManager, auditWriter);
DashboardSessionAdminResult result = await service.CloseSessionAsync(
CreateUser(DashboardRoles.Viewer),
"session-1",
CancellationToken.None);
Assert.False(result.Succeeded);
AuditEvent audit = Assert.Single(auditWriter.Events);
Assert.Equal(DashboardSessionAdminService.CloseSessionAction, audit.Action);
Assert.Equal(AuditOutcome.Denied, audit.Outcome);
}
private static DashboardSessionAdminService CreateService(
ISessionManager sessionManager,
IAuditWriter? auditWriter = null)
{
DefaultHttpContext httpContext = new();
httpContext.Connection.RemoteIpAddress = System.Net.IPAddress.Loopback;
return new DashboardSessionAdminService(
sessionManager,
new HttpContextAccessor { HttpContext = httpContext });
new HttpContextAccessor { HttpContext = httpContext },
auditWriter ?? new RecordingAuditWriter());
}
private static ClaimsPrincipal CreateUser(string role)
@@ -219,6 +298,21 @@ public sealed class DashboardSessionAdminServiceTests
return new ClaimsPrincipal(identity);
}
private sealed class RecordingAuditWriter : IAuditWriter
{
private readonly ConcurrentQueue<AuditEvent> _events = new();
/// <summary>Gets the audit events written through this writer, in order.</summary>
public IReadOnlyList<AuditEvent> Events => _events.ToArray();
/// <inheritdoc />
public Task WriteAsync(AuditEvent evt, CancellationToken ct = default)
{
_events.Enqueue(evt);
return Task.CompletedTask;
}
}
private sealed class FakeSessionManager : ISessionManager
{
/// <summary>Gets the number of times CloseSessionAsync was invoked.</summary>
@@ -623,6 +623,27 @@ public sealed class DashboardSnapshotServiceTests
{
return Task.FromResult(false);
}
/// <summary>Does nothing; the fake never changes scopes (added to IApiKeyAdminStore in Auth 0.1.3).</summary>
/// <param name="keyId">Key identifier.</param>
/// <param name="scopes">Replacement scope set.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns><see langword="false"/>, always.</returns>
public Task<bool> SetScopesAsync(string keyId, IReadOnlySet<string> scopes, CancellationToken ct)
{
return Task.FromResult(false);
}
/// <summary>Does nothing; the fake never toggles key state (added to IApiKeyAdminStore in Auth 0.1.3).</summary>
/// <param name="keyId">Key identifier.</param>
/// <param name="enabled">Desired enabled state.</param>
/// <param name="whenUtc">Timestamp for the state change.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns><see langword="false"/>, always.</returns>
public Task<bool> SetEnabledAsync(string keyId, bool enabled, DateTimeOffset whenUtc, CancellationToken ct)
{
return Task.FromResult(false);
}
}
private class CountingApiKeyAdminStore(params ApiKeyListItem[] records) : FakeApiKeyAdminStore
@@ -115,4 +115,63 @@ public sealed class HubTokenServiceTests
Assert.Null(service.Validate("this-is-not-a-protected-payload"));
}
/// <summary>
/// Issue/validate round-trip: a freshly minted token (default <see cref="HubTokenService.TokenLifetime"/>)
/// validates and reconstructs the caller's identity and roles.
/// </summary>
[Fact]
public void IssueThenValidate_FreshToken_RoundTripsIdentityAndRoles()
{
HubTokenService service = new(new EphemeralDataProtectionProvider());
ClaimsIdentity identity = new(
[
new Claim(ClaimTypes.Name, "bob"),
new Claim(ClaimTypes.NameIdentifier, "bob-id"),
new Claim(ClaimTypes.Role, DashboardRoles.Viewer),
new Claim(ClaimTypes.Role, DashboardRoles.Admin),
],
authenticationType: "test",
nameType: ClaimTypes.Name,
roleType: ClaimTypes.Role);
string token = service.Issue(new ClaimsPrincipal(identity));
ClaimsPrincipal? result = service.Validate(token);
Assert.NotNull(result);
Assert.Equal("bob", result.Identity?.Name);
Assert.True(result.IsInRole(DashboardRoles.Viewer));
Assert.True(result.IsInRole(DashboardRoles.Admin));
}
/// <summary>
/// The default token lifetime is the short (5-minute) window mandated by SEC-05, not the
/// former 30-minute window. Pins the value so a regression that widens the exposure window
/// of an irrevocable, query-string-carried token is caught in CI.
/// </summary>
[Fact]
public void TokenLifetime_IsFiveMinutes()
{
Assert.Equal(TimeSpan.FromMinutes(5), HubTokenService.TokenLifetime);
}
/// <summary>
/// A token whose lifetime has elapsed is rejected by <see cref="HubTokenService.Validate"/>.
/// Uses the internal lifetime-issuing seam with a negative lifetime so the token is already
/// expired at mint time — deterministic, no wall-clock delay.
/// </summary>
[Fact]
public void Validate_ExpiredToken_ReturnsNull()
{
HubTokenService service = new(new EphemeralDataProtectionProvider());
ClaimsIdentity identity = new(
[new Claim(ClaimTypes.Name, "carol")],
authenticationType: "test");
string expiredToken = service.Issue(
new ClaimsPrincipal(identity),
TimeSpan.FromMinutes(-1));
Assert.Null(service.Validate(expiredToken));
}
}
@@ -9,6 +9,10 @@ using ZB.MOM.WW.MxGateway.Server;
namespace ZB.MOM.WW.MxGateway.Tests.Gateway;
// Sets process-global Kestrel/TLS environment variables that GatewayApplication.Build reads;
// serialized against all other collections so a parallel host-building test cannot inherit them
// mid-run and race on the generated-certificate file. See GlobalEnvironmentCollection.
[Collection(TestSupport.GlobalEnvironmentCollection.Name)]
public sealed class GatewayTlsBootstrapTests
{
/// <summary>
@@ -0,0 +1,45 @@
using Grpc.Core;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Grpc;
namespace ZB.MOM.WW.MxGateway.Tests.Gateway.Grpc;
public sealed class MxAccessGrpcRequestValidatorTests
{
private static MxCommandRequest DrainRequest(uint maxEvents) => new()
{
SessionId = "session-1",
Command = new MxCommand
{
Kind = MxCommandKind.DrainEvents,
DrainEvents = new DrainEventsCommand { MaxEvents = maxEvents },
},
};
/// <summary>
/// Verifies a DrainEvents request within the per-request ceiling passes validation, including the
/// <c>max_events = 0</c> "worker default cap" sentinel (IPC-04).
/// </summary>
[Theory]
[InlineData(0u)]
[InlineData(1u)]
[InlineData(10_000u)]
public void ValidateInvoke_AllowsDrainEvents_WithinCeiling(uint maxEvents)
{
MxAccessGrpcRequestValidator validator = new();
validator.ValidateInvoke(DrainRequest(maxEvents));
}
/// <summary>
/// Verifies a DrainEvents request above the per-request ceiling is rejected with InvalidArgument
/// so one accepted request cannot pack an unbounded reply frame (IPC-04).
/// </summary>
[Fact]
public void ValidateInvoke_RejectsDrainEvents_AboveCeiling()
{
MxAccessGrpcRequestValidator validator = new();
RpcException exception = Assert.Throws<RpcException>(() => validator.ValidateInvoke(DrainRequest(10_001)));
Assert.Equal(StatusCode.InvalidArgument, exception.StatusCode);
Assert.Contains("max_events", exception.Status.Detail, StringComparison.Ordinal);
}
}
@@ -56,6 +56,52 @@ public sealed class WorkerClientTests
Assert.Equal(MxCommandKind.Ping, reply.Reply.Kind);
}
/// <summary>
/// Verifies that a command whose serialized envelope exceeds the negotiated worker-frame maximum
/// fails only that command with <see cref="WorkerClientErrorCode.CommandTooLarge"/> at the enqueue
/// boundary, leaving the client ready for subsequent commands (IPC-03). Without the pre-check the
/// oversized frame would reach the write loop and fault the whole session.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task InvokeAsync_WhenCommandExceedsFrameMax_FailsOnlyThatCommandAndStaysReady()
{
await using PipePair pipePair = await PipePair.CreateAsync();
await using WorkerClient client = CreateClient(pipePair, maxMessageBytes: 4096);
await CompleteHandshakeAsync(client, pipePair);
WorkerCommand oversized = new()
{
Command = new MxCommand
{
Kind = MxCommandKind.Write,
Write = new WriteCommand
{
ServerHandle = 1,
ItemHandle = 2,
Value = new MxValue { StringValue = new string('x', 8192) },
},
},
};
WorkerClientException exception = await Assert.ThrowsAsync<WorkerClientException>(
async () => await client.InvokeAsync(oversized, TestTimeout, CancellationToken.None));
Assert.Equal(WorkerClientErrorCode.CommandTooLarge, exception.ErrorCode);
Assert.Equal(WorkerClientState.Ready, client.State);
// A subsequent normally-sized command still round-trips: the session was not faulted.
Task<WorkerCommandReply> nextInvoke = client.InvokeAsync(
CreateCommand(MxCommandKind.Ping),
TestTimeout,
CancellationToken.None);
WorkerEnvelope commandEnvelope = await pipePair.WorkerReader.ReadAsync().AsTask().WaitAsync(TestTimeout);
await pipePair.WorkerWriter.WriteAsync(
CreateCommandReplyEnvelope(commandEnvelope.CorrelationId, MxCommandKind.Ping));
WorkerCommandReply reply = await nextInvoke.WaitAsync(TestTimeout);
Assert.Equal(MxCommandKind.Ping, reply.Reply.Kind);
Assert.Equal(WorkerClientState.Ready, client.State);
}
/// <summary>Verifies that InvokeAsync ignores late replies and keeps the client ready.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -172,6 +218,51 @@ public sealed class WorkerClientTests
Assert.Equal(WorkerClientState.Faulted, client.State);
}
/// <summary>
/// Verifies that a command reply arriving on the pipe after events is dispatched promptly even
/// when the event channel is full and has no consumer — event enqueue is decoupled from the read
/// loop, so a blocked event writer cannot delay a reply (GWC-04). The event full-mode timeout is
/// set far above the command timeout: without the decoupling the read loop would block behind the
/// full event channel and the in-flight InvokeAsync would hit CommandTimeout.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task ReadLoop_WhenEventChannelFull_DispatchesCommandReplyWithoutBlocking()
{
await using PipePair pipePair = await PipePair.CreateAsync();
await using WorkerClient client = CreateClient(
pipePair,
new WorkerClientOptions
{
EventChannelCapacity = 1,
// Much larger than TestTimeout: the read loop must not block for this long behind the
// full event channel while a reply waits.
EventChannelFullModeTimeout = TimeSpan.FromSeconds(30),
HeartbeatGrace = TimeSpan.FromSeconds(60),
HeartbeatCheckInterval = TimeSpan.FromSeconds(60),
});
await CompleteHandshakeAsync(client, pipePair);
// No StreamEvents consumer is attached, so the capacity-1 event channel fills and the event
// writer blocks on the second event's timed WriteAsync.
await pipePair.WorkerWriter.WriteAsync(
CreateEventEnvelope(sequence: 11, MxEventFamily.OnDataChange));
await pipePair.WorkerWriter.WriteAsync(
CreateEventEnvelope(sequence: 12, MxEventFamily.OnDataChange));
Task<WorkerCommandReply> invokeTask = client.InvokeAsync(
CreateCommand(MxCommandKind.Ping),
TestTimeout,
CancellationToken.None);
WorkerEnvelope commandEnvelope = await pipePair.WorkerReader.ReadAsync().AsTask().WaitAsync(TestTimeout);
await pipePair.WorkerWriter.WriteAsync(
CreateCommandReplyEnvelope(commandEnvelope.CorrelationId, MxCommandKind.Ping));
WorkerCommandReply reply = await invokeTask.WaitAsync(TestTimeout);
Assert.Equal(MxCommandKind.Ping, reply.Reply.Kind);
Assert.Equal(WorkerClientState.Ready, client.State);
}
/// <summary>
/// Verifies that when the client faults it kills the owned worker process.
/// The assertion waits on <see cref="FakeWorkerProcess.WaitForExitAsync"/>, which
@@ -564,9 +655,13 @@ public sealed class WorkerClientTests
WorkerClientOptions? options = null,
GatewayMetrics? metrics = null,
WorkerProcessHandle? processHandle = null,
TimeProvider? timeProvider = null)
TimeProvider? timeProvider = null,
int maxMessageBytes = WorkerFrameProtocolOptions.DefaultMaxMessageBytes)
{
WorkerFrameProtocolOptions frameOptions = new(SessionId);
WorkerFrameProtocolOptions frameOptions = new(
SessionId,
GatewayContractInfo.WorkerProtocolVersion,
maxMessageBytes);
WorkerClientConnection connection = new(
SessionId,
Nonce,
@@ -157,6 +157,56 @@ public sealed class GatewayMetricsTests
Assert.Equal(2, capturedMode);
}
/// <summary>
/// Verifies that <see cref="GatewayMetrics.HeartbeatFailed"/> increments
/// <c>mxgateway.heartbeats.failed</c> without emitting a <c>session_id</c> tag: the tag is
/// unbounded cardinality (every session mints a new exporter time series), so per-session
/// attribution is deliberately kept out of the exported counter (SEC-20).
/// </summary>
[Fact]
public void HeartbeatFailed_IncrementsCounterWithoutSessionIdTag()
{
using GatewayMetrics metrics = new();
using MeterListener listener = new();
long capturedValue = 0;
bool sawSessionIdTag = false;
listener.InstrumentPublished = (instrument, meterListener) =>
{
if (ReferenceEquals(instrument.Meter, metrics.Meter)
&& instrument.Name == "mxgateway.heartbeats.failed")
{
meterListener.EnableMeasurementEvents(instrument);
}
};
listener.SetMeasurementEventCallback<long>(
(instrument, measurement, tags, _) =>
{
if (!ReferenceEquals(instrument.Meter, metrics.Meter)
|| instrument.Name != "mxgateway.heartbeats.failed")
{
return;
}
capturedValue += measurement;
foreach (KeyValuePair<string, object?> tag in tags)
{
if (tag.Key == "session_id")
{
sawSessionIdTag = true;
}
}
});
listener.Start();
metrics.HeartbeatFailed("session-1");
Assert.Equal(1, capturedValue);
Assert.False(sawSessionIdTag);
Assert.Equal(1, metrics.GetSnapshot().HeartbeatFailures);
}
/// <summary>Verifies that removing session events only affects that session.</summary>
[Fact]
public void RemoveSessionEvents_RemovesOnlyThatSession()
@@ -0,0 +1,54 @@
namespace ZB.MOM.WW.MxGateway.Tests.ProjectStructure;
/// <summary>
/// Repo-hygiene guards over the source tree. See SEC-01: a Windows-absolute default path that
/// resolves relative to the launch working directory silently materializes a SQLite credential
/// store inside the tree. This test fails if any such <c>*.db</c> file appears under <c>src/</c>.
/// </summary>
public sealed class GatewayTreeHygieneTests
{
/// <summary>Verifies no SQLite database files are committed/materialized under the source tree (build output excluded).</summary>
[Fact]
public void SourceTree_ContainsNoSqliteDatabaseFiles()
{
DirectoryInfo sourceRoot = FindSourceRoot();
string[] databaseFiles = Directory
.EnumerateFiles(sourceRoot.FullName, "*.db", SearchOption.AllDirectories)
.Where(path => !IsBuildOutput(path))
.ToArray();
Assert.True(
databaseFiles.Length == 0,
"Unexpected *.db file(s) found under the source tree (build output excluded): "
+ string.Join(", ", databaseFiles)
+ ". A SQLite DB in the tree usually means a Windows-absolute default path resolved "
+ "relative to the working directory; see SEC-01.");
}
// Build output (bin/obj) is regenerated by test/compile runs and is gitignored, so exclude it.
private static bool IsBuildOutput(string path)
{
string normalized = path.Replace('\\', '/');
return normalized.Contains("/bin/", StringComparison.Ordinal)
|| normalized.Contains("/obj/", StringComparison.Ordinal);
}
// The directory holding ZB.MOM.WW.MxGateway.slnx is the src/ root of the working tree.
private static DirectoryInfo FindSourceRoot()
{
DirectoryInfo? current = new(AppContext.BaseDirectory);
while (current is not null)
{
if (File.Exists(Path.Combine(current.FullName, "ZB.MOM.WW.MxGateway.slnx")))
{
return current;
}
current = current.Parent;
}
throw new DirectoryNotFoundException("Could not locate src/ZB.MOM.WW.MxGateway.slnx from the test output directory.");
}
}
@@ -52,6 +52,40 @@ public sealed class ApiKeyAdminCliRunnerTests : IDisposable
Assert.Contains(auditRecords, record => record.EventType == "create-key" && record.KeyId == "operator01");
}
/// <summary>
/// Verifies that a key created with an already-past <c>--expires</c> is rejected by the verifier
/// — the CLI expiry wiring reaches the store and the library enforces it end-to-end (SEC-10).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task CreateKeyAsync_WithPastExpiry_KeyIsRejectedByVerifier()
{
await using ServiceProvider services = BuildServices(CreateTempDatabasePath());
ApiKeyAdminCliRunner runner = services.GetRequiredService<ApiKeyAdminCliRunner>();
StringWriter output = new();
await runner.RunAsync(
new ApiKeyAdminCommand(
Kind: ApiKeyAdminCommandKind.CreateKey,
Json: true,
SqlitePath: null,
Pepper: null,
KeyId: "operator01",
DisplayName: "Operator",
Scopes: new HashSet<string>(StringComparer.Ordinal) { "session:open" },
Constraints: ApiKeyConstraints.Empty,
ExpiresUtc: DateTimeOffset.UtcNow - TimeSpan.FromMinutes(1)),
output,
CancellationToken.None);
string apiKey = ReadApiKey(output.ToString());
IApiKeyVerifier verifier = services.GetRequiredService<IApiKeyVerifier>();
ApiKeyVerification verification = await verifier.VerifyAsync($"Bearer {apiKey}", CancellationToken.None);
Assert.False(verification.Succeeded);
}
/// <summary>Verifies that ListKeysAsync does not print the raw secret.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -52,6 +52,61 @@ public sealed class ApiKeyAdminCommandLineParserTests
Assert.Contains("events:read", result.Command.Scopes);
}
/// <summary>A create-key command without --expires leaves the key non-expiring (opt-in, SEC-10).</summary>
[Fact]
public void Parse_CreateKeyCommand_WithoutExpires_HasNoExpiry()
{
ApiKeyAdminParseResult result = ApiKeyAdminCommandLineParser.Parse(
["apikey", "create-key", "--key-id", "operator01", "--display-name", "Operator", "--scopes", "session:open"]);
Assert.NotNull(result.Command);
Assert.Null(result.Command.ExpiresUtc);
}
/// <summary>An absolute ISO-8601 --expires value is parsed to that exact UTC instant (SEC-10).</summary>
[Fact]
public void Parse_CreateKeyCommand_WithAbsoluteExpires_ParsesInstant()
{
ApiKeyAdminParseResult result = ApiKeyAdminCommandLineParser.Parse(
["apikey", "create-key", "--key-id", "operator01", "--display-name", "Operator", "--scopes", "session:open",
"--expires", "2030-01-02T03:04:05Z"]);
Assert.NotNull(result.Command);
Assert.Equal(
new DateTimeOffset(2030, 1, 2, 3, 4, 5, TimeSpan.Zero),
result.Command.ExpiresUtc);
}
/// <summary>A relative "&lt;N&gt;d" --expires value resolves to a future UTC instant (SEC-10).</summary>
[Fact]
public void Parse_CreateKeyCommand_WithRelativeExpires_ResolvesToFuture()
{
DateTimeOffset before = DateTimeOffset.UtcNow;
ApiKeyAdminParseResult result = ApiKeyAdminCommandLineParser.Parse(
["apikey", "create-key", "--key-id", "operator01", "--display-name", "Operator", "--scopes", "session:open",
"--expires", "30d"]);
Assert.NotNull(result.Command);
Assert.NotNull(result.Command.ExpiresUtc);
Assert.InRange(
result.Command.ExpiresUtc!.Value,
before + TimeSpan.FromDays(30) - TimeSpan.FromMinutes(1),
DateTimeOffset.UtcNow + TimeSpan.FromDays(30) + TimeSpan.FromMinutes(1));
}
/// <summary>An unparseable --expires value fails at parse time (SEC-10).</summary>
[Fact]
public void Parse_CreateKeyCommand_WithInvalidExpires_Fails()
{
ApiKeyAdminParseResult result = ApiKeyAdminCommandLineParser.Parse(
["apikey", "create-key", "--key-id", "operator01", "--display-name", "Operator", "--scopes", "session:open",
"--expires", "not-a-date"]);
Assert.Null(result.Command);
Assert.NotNull(result.Error);
Assert.Contains("--expires", result.Error, StringComparison.Ordinal);
}
/// <summary>
/// A create-key command with a non-canonical scope
/// string (e.g. CLAUDE.md's stale <c>invoke</c> instead of <c>invoke:read</c>)
@@ -0,0 +1,201 @@
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Time.Testing;
using ZB.MOM.WW.Auth.Abstractions.ApiKeys;
using ZB.MOM.WW.MxGateway.Server.Security.Authentication;
using LibApiKeyIdentity = ZB.MOM.WW.Auth.Abstractions.ApiKeys.ApiKeyIdentity;
namespace ZB.MOM.WW.MxGateway.Tests.Security.Authentication;
/// <summary>
/// SEC-08 hot-path decorators. Covers both mechanisms: <see cref="CachingApiKeyVerifier"/>
/// (read/verification coalescing plus revoke/rotate invalidation) and
/// <see cref="CoalescingMarkApiKeyStore"/> (the <c>last_used</c> write coalescing that keeps the
/// per-RPC database write off the throughput ceiling).
/// </summary>
public sealed class CachingApiKeyVerifierTests
{
private const string Header = "Bearer mxgw_operator01_super-secret";
/// <summary>A cache hit within the TTL returns the cached result and never calls the inner verifier.</summary>
[Fact]
public async Task VerifyAsync_RepeatedWithinTtl_CallsInnerOnce()
{
FakeVerifier inner = new(Success("operator01"));
using MemoryCache cache = NewCache();
CachingApiKeyVerifier verifier = new(inner, cache, TimeSpan.FromSeconds(15));
ApiKeyVerification first = await verifier.VerifyAsync(Header, CancellationToken.None);
ApiKeyVerification second = await verifier.VerifyAsync(Header, CancellationToken.None);
Assert.True(first.Succeeded);
Assert.True(second.Succeeded);
Assert.Equal(1, inner.CallCount);
}
/// <summary>Different presented secrets are cached under distinct keys (no cross-secret aliasing).</summary>
[Fact]
public async Task VerifyAsync_DifferentTokens_NotAliased()
{
FakeVerifier inner = new(Success("operator01"));
using MemoryCache cache = NewCache();
CachingApiKeyVerifier verifier = new(inner, cache, TimeSpan.FromSeconds(15));
await verifier.VerifyAsync("Bearer mxgw_operator01_secret-a", CancellationToken.None);
await verifier.VerifyAsync("Bearer mxgw_operator01_secret-b", CancellationToken.None);
Assert.Equal(2, inner.CallCount);
}
/// <summary>Failed verifications are never cached; every attempt reaches the inner verifier.</summary>
[Fact]
public async Task VerifyAsync_FailedVerification_NotCached()
{
FakeVerifier inner = new(Failure(ApiKeyFailure.SecretMismatch));
using MemoryCache cache = NewCache();
CachingApiKeyVerifier verifier = new(inner, cache, TimeSpan.FromSeconds(15));
await verifier.VerifyAsync(Header, CancellationToken.None);
await verifier.VerifyAsync(Header, CancellationToken.None);
Assert.Equal(2, inner.CallCount);
}
/// <summary>A TTL of zero disables caching: the inner verifier is called on every request.</summary>
[Fact]
public async Task VerifyAsync_ZeroTtl_DisablesCache()
{
FakeVerifier inner = new(Success("operator01"));
using MemoryCache cache = NewCache();
CachingApiKeyVerifier verifier = new(inner, cache, TimeSpan.Zero);
await verifier.VerifyAsync(Header, CancellationToken.None);
await verifier.VerifyAsync(Header, CancellationToken.None);
Assert.Equal(2, inner.CallCount);
}
/// <summary>Invalidating a key id (revoke/rotate) drops its cached verification, forcing a re-verify.</summary>
[Fact]
public async Task Invalidate_DropsCachedEntry_ForcesReverify()
{
FakeVerifier inner = new(Success("operator01"));
using MemoryCache cache = NewCache();
CachingApiKeyVerifier verifier = new(inner, cache, TimeSpan.FromSeconds(30));
await verifier.VerifyAsync(Header, CancellationToken.None);
Assert.Equal(1, inner.CallCount);
((IApiKeyCacheInvalidator)verifier).Invalidate("operator01");
await verifier.VerifyAsync(Header, CancellationToken.None);
Assert.Equal(2, inner.CallCount);
}
/// <summary>
/// The store decorator coalesces repeated <c>MarkUsed</c> writes for the same key inside the
/// window down to a single forwarded write — the ≤1/min guarantee for <c>last_used_utc</c>.
/// </summary>
[Fact]
public async Task CoalescingStore_RepeatedMarksWithinWindow_ForwardsOnce()
{
FakeStore inner = new();
FakeTimeProvider clock = new(DateTimeOffset.UtcNow);
CoalescingMarkApiKeyStore store = new(inner, TimeSpan.FromMinutes(1), clock);
// Ten rapid authenticated calls in the same minute.
for (int i = 0; i < 10; i++)
{
await store.MarkUsedAsync("operator01", clock.GetUtcNow(), CancellationToken.None);
clock.Advance(TimeSpan.FromSeconds(5));
}
Assert.Equal(1, inner.MarkUsedCount);
}
/// <summary>After the window elapses the next mark is forwarded again (staleness is bounded, not frozen).</summary>
[Fact]
public async Task CoalescingStore_AfterWindow_ForwardsAgain()
{
FakeStore inner = new();
FakeTimeProvider clock = new(DateTimeOffset.UtcNow);
CoalescingMarkApiKeyStore store = new(inner, TimeSpan.FromMinutes(1), clock);
await store.MarkUsedAsync("operator01", clock.GetUtcNow(), CancellationToken.None);
clock.Advance(TimeSpan.FromSeconds(61));
await store.MarkUsedAsync("operator01", clock.GetUtcNow(), CancellationToken.None);
Assert.Equal(2, inner.MarkUsedCount);
}
/// <summary>Distinct keys are coalesced independently.</summary>
[Fact]
public async Task CoalescingStore_DistinctKeys_TrackedSeparately()
{
FakeStore inner = new();
FakeTimeProvider clock = new(DateTimeOffset.UtcNow);
CoalescingMarkApiKeyStore store = new(inner, TimeSpan.FromMinutes(1), clock);
await store.MarkUsedAsync("key-a", clock.GetUtcNow(), CancellationToken.None);
await store.MarkUsedAsync("key-b", clock.GetUtcNow(), CancellationToken.None);
await store.MarkUsedAsync("key-a", clock.GetUtcNow(), CancellationToken.None);
Assert.Equal(2, inner.MarkUsedCount);
}
/// <summary>A zero window disables coalescing: every mark is forwarded.</summary>
[Fact]
public async Task CoalescingStore_ZeroWindow_ForwardsEveryMark()
{
FakeStore inner = new();
FakeTimeProvider clock = new(DateTimeOffset.UtcNow);
CoalescingMarkApiKeyStore store = new(inner, TimeSpan.Zero, clock);
await store.MarkUsedAsync("operator01", clock.GetUtcNow(), CancellationToken.None);
await store.MarkUsedAsync("operator01", clock.GetUtcNow(), CancellationToken.None);
Assert.Equal(2, inner.MarkUsedCount);
}
private static MemoryCache NewCache() => new(new MemoryCacheOptions());
private static ApiKeyVerification Success(string keyId) => new(
Succeeded: true,
Identity: new LibApiKeyIdentity(
KeyId: keyId,
DisplayName: "Operator Key",
Scopes: new HashSet<string>(StringComparer.Ordinal),
Constraints: null),
Failure: null);
private static ApiKeyVerification Failure(ApiKeyFailure failure) =>
new(Succeeded: false, Identity: null, Failure: failure);
private sealed class FakeVerifier(ApiKeyVerification result) : IApiKeyVerifier
{
public int CallCount { get; private set; }
public Task<ApiKeyVerification> VerifyAsync(string authorizationHeader, CancellationToken ct)
{
CallCount++;
return Task.FromResult(result);
}
}
private sealed class FakeStore : IApiKeyStore
{
public int MarkUsedCount { get; private set; }
public Task<ApiKeyRecord?> FindByKeyIdAsync(string keyId, CancellationToken ct)
=> Task.FromResult<ApiKeyRecord?>(null);
public Task<ApiKeyRecord?> FindActiveByKeyIdAsync(string keyId, CancellationToken ct)
=> Task.FromResult<ApiKeyRecord?>(null);
public Task MarkUsedAsync(string keyId, DateTimeOffset whenUtc, CancellationToken ct)
{
MarkUsedCount++;
return Task.CompletedTask;
}
}
}
@@ -327,7 +327,7 @@ public sealed class GatewayGrpcAuthorizationInterceptorTests
RpcException exception = await Assert.ThrowsAsync<RpcException>(
() => interceptor.ServerStreamingServerHandler(
new StreamAlarmsRequest(),
new QueryActiveAlarmsRequest(),
new RecordingServerStreamWriter<ActiveAlarmSnapshot>(),
ContextWithAuthorization("Bearer mxgw_operator01_secret"),
(_, _, _) => Task.CompletedTask));
@@ -347,7 +347,7 @@ public sealed class GatewayGrpcAuthorizationInterceptorTests
RecordingServerStreamWriter<ActiveAlarmSnapshot> streamWriter = new();
await interceptor.ServerStreamingServerHandler(
new StreamAlarmsRequest(),
new QueryActiveAlarmsRequest(),
streamWriter,
ContextWithAuthorization("Bearer mxgw_operator01_secret"),
async (_, writer, _) =>
@@ -358,6 +358,102 @@ public sealed class GatewayGrpcAuthorizationInterceptorTests
Assert.Single(streamWriter.Messages);
}
/// <summary>
/// SEC-11: once a peer has exceeded the failure limit, the interceptor short-circuits with
/// <see cref="StatusCode.ResourceExhausted"/> BEFORE calling the verifier, so an online guessing
/// loop stops spending a store read per attempt. A verifier that always fails is used; after the
/// limit is reached the verifier is no longer invoked.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task UnaryServerHandler_ExceedsFailureLimit_ShortCircuitsBeforeVerify()
{
CountingFailureVerifier verifier = new(Failure(ApiKeyFailure.SecretMismatch));
ApiKeyFailureLimiter limiter = new(
limit: 3,
window: TimeSpan.FromMinutes(1),
maxPeers: 16,
clock: TimeProvider.System);
GatewayGrpcAuthorizationInterceptor interceptor = CreateInterceptor(
verifier,
new GatewayRequestIdentityAccessor(),
failureLimiter: limiter);
// The first three attempts reach the verifier and fail (recording a failure each time).
for (int attempt = 0; attempt < 3; attempt++)
{
RpcException failure = await Assert.ThrowsAsync<RpcException>(
() => interceptor.UnaryServerHandler(
new OpenSessionRequest(),
ContextWithAuthorization("Bearer mxgw_operator01_bad-secret"),
(_, _) => Task.FromResult(new OpenSessionReply())));
Assert.Equal(StatusCode.Unauthenticated, failure.StatusCode);
}
Assert.Equal(3, verifier.CallCount);
// The fourth attempt is short-circuited: ResourceExhausted, and the verifier is NOT called.
RpcException throttled = await Assert.ThrowsAsync<RpcException>(
() => interceptor.UnaryServerHandler(
new OpenSessionRequest(),
ContextWithAuthorization("Bearer mxgw_operator01_bad-secret"),
(_, _) => Task.FromResult(new OpenSessionReply())));
Assert.Equal(StatusCode.ResourceExhausted, throttled.StatusCode);
Assert.Equal(3, verifier.CallCount);
}
/// <summary>
/// SEC-11: a successful verification resets the peer's failure counter, so accumulated failures
/// from a fat-fingered secret do not lock out a client that subsequently authenticates.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task UnaryServerHandler_SuccessResetsFailureCounter()
{
ApiKeyFailureLimiter limiter = new(
limit: 3,
window: TimeSpan.FromMinutes(1),
maxPeers: 16,
clock: TimeProvider.System);
// Two failures against the same key id, then a success (which resets), then two more
// failures — without the reset the fifth attempt would be blocked at the limit of 3.
GatewayGrpcAuthorizationInterceptor failing = CreateInterceptor(
new FakeApiKeyVerifier(Failure(ApiKeyFailure.SecretMismatch)),
new GatewayRequestIdentityAccessor(),
failureLimiter: limiter);
GatewayGrpcAuthorizationInterceptor succeeding = CreateInterceptor(
new FakeApiKeyVerifier(SuccessWithScopes(GatewayScopes.SessionOpen)),
new GatewayRequestIdentityAccessor(),
failureLimiter: limiter);
for (int i = 0; i < 2; i++)
{
await Assert.ThrowsAsync<RpcException>(
() => failing.UnaryServerHandler(
new OpenSessionRequest(),
ContextWithAuthorization("Bearer mxgw_operator01_bad"),
(_, _) => Task.FromResult(new OpenSessionReply())));
}
await succeeding.UnaryServerHandler(
new OpenSessionRequest(),
ContextWithAuthorization("Bearer mxgw_operator01_good"),
(_, _) => Task.FromResult(new OpenSessionReply { SessionId = "s" }));
// Post-reset: two more failures still map to Unauthenticated (not ResourceExhausted).
for (int i = 0; i < 2; i++)
{
RpcException ex = await Assert.ThrowsAsync<RpcException>(
() => failing.UnaryServerHandler(
new OpenSessionRequest(),
ContextWithAuthorization("Bearer mxgw_operator01_bad"),
(_, _) => Task.FromResult(new OpenSessionReply())));
Assert.Equal(StatusCode.Unauthenticated, ex.StatusCode);
}
}
private static MxAccessGatewayService CreateService(
ISessionManager sessionManager,
IGatewayRequestIdentityAccessor identityAccessor)
@@ -377,7 +473,8 @@ public sealed class GatewayGrpcAuthorizationInterceptorTests
private static GatewayGrpcAuthorizationInterceptor CreateInterceptor(
IApiKeyVerifier apiKeyVerifier,
IGatewayRequestIdentityAccessor identityAccessor,
AuthenticationMode authenticationMode = AuthenticationMode.ApiKey)
AuthenticationMode authenticationMode = AuthenticationMode.ApiKey,
ApiKeyFailureLimiter? failureLimiter = null)
{
return new GatewayGrpcAuthorizationInterceptor(
apiKeyVerifier,
@@ -389,7 +486,12 @@ public sealed class GatewayGrpcAuthorizationInterceptorTests
{
Mode = authenticationMode
}
}));
}),
failureLimiter ?? new ApiKeyFailureLimiter(
limit: 1000,
window: TimeSpan.FromMinutes(1),
maxPeers: 1000,
clock: TimeProvider.System));
}
private static ApiKeyVerification SuccessWithScopes(params string[] scopes)
@@ -531,6 +633,22 @@ public sealed class GatewayGrpcAuthorizationInterceptorTests
}
}
private sealed class CountingFailureVerifier(ApiKeyVerification result) : IApiKeyVerifier
{
/// <summary>Gets the number of times the verifier was invoked.</summary>
public int CallCount { get; private set; }
/// <summary>Returns the configured result and counts the invocation.</summary>
/// <param name="authorizationHeader">The authorization header to verify.</param>
/// <param name="ct">Cancellation token.</param>
/// <returns>The configured verification result.</returns>
public Task<ApiKeyVerification> VerifyAsync(string authorizationHeader, CancellationToken ct)
{
CallCount++;
return Task.FromResult(result);
}
}
private sealed class FakeApiKeyVerifier(ApiKeyVerification result) : IApiKeyVerifier
{
/// <summary>Gets whether the verifier was called.</summary>
@@ -0,0 +1,18 @@
namespace ZB.MOM.WW.MxGateway.Tests.TestSupport;
/// <summary>
/// xUnit collection for tests that mutate <em>process-global</em> state (environment variables
/// read by <c>GatewayApplication.Build</c>, e.g. <c>Kestrel__Endpoints__…</c> and
/// <c>MxGateway__Tls__SelfSignedCertPath</c>). <c>DisableParallelization</c> keeps such a test from
/// running concurrently with any other collection: otherwise a parallel host-building test inherits
/// the mutated variables mid-run and the two race on the same generated-certificate file (on Windows,
/// "the process cannot access the file … because it is being used by another process"). Membership is
/// deliberately narrow — only add classes that set/clear real environment variables, not ones that
/// pass configuration through <c>Build([...])</c> command-line args.
/// </summary>
[CollectionDefinition(Name, DisableParallelization = true)]
public sealed class GlobalEnvironmentCollection
{
/// <summary>The collection name applied via <c>[Collection(GlobalEnvironmentCollection.Name)]</c>.</summary>
public const string Name = "GlobalEnvironmentMutation";
}
@@ -0,0 +1,51 @@
using System.Runtime.CompilerServices;
namespace ZB.MOM.WW.MxGateway.Tests.TestSupport;
/// <summary>
/// Defaults the host environment to Development and isolates the test process's on-disk
/// gateway paths, for the whole test assembly.
/// </summary>
/// <remarks>
/// Many tests build the full gateway host through <c>GatewayApplication.Build</c> against the dev
/// <c>appsettings.json</c> (which ships <c>Ldap:Transport=None</c> and other dev-only defaults).
/// An unset environment resolves to Production, where the SEC-04/06 production guards fail startup
/// by design — so those host-building tests would trip the guards. Setting the environment to
/// Development once, before any test runs, keeps that suite exercising app wiring rather than
/// production-config validation. Tests that specifically assert production behavior construct
/// <c>GatewayOptionsValidator</c> with its <c>isProduction</c> constructor (no host environment) and
/// are unaffected; a test that needs Production can still pass <c>--environment=Production</c>, which
/// overrides this default.
/// <para>
/// The self-signed-cert path is also defaulted to a per-process temp file. Otherwise every
/// full-host test that triggers HTTPS-cert generation writes the shared
/// <c>CommonApplicationData/MxGateway/certs/gateway-selfsigned.pfx</c> default — parallel xUnit
/// test classes then collide on that path's temp file (Windows: "the process cannot access the
/// file ... because it is being used by another process"), and on a shared CI/dev box the suite
/// also fights the deployed gateway service for the same file. Pointing at a per-process path
/// isolates the suite: the first host-building test generates the cert, the rest load it.
/// </para>
/// </remarks>
internal static class TestHostEnvironmentInitializer
{
[ModuleInitializer]
internal static void SetDevelopmentEnvironmentDefault()
{
if (string.IsNullOrEmpty(Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")))
{
Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Development");
}
if (string.IsNullOrEmpty(Environment.GetEnvironmentVariable("MxGateway__Tls__SelfSignedCertPath")))
{
// ProcessId keeps the path stable within this test process (so a valid cert
// generated by the first host-building test is reused by the rest) yet unique
// across processes (so concurrent test runs / the deployed service never share it).
string certPath = Path.Combine(
Path.GetTempPath(),
$"mxgw-tests-{Environment.ProcessId}",
"gateway-selfsigned.pfx");
Environment.SetEnvironmentVariable("MxGateway__Tls__SelfSignedCertPath", certPath);
}
}
}
@@ -1,5 +1,7 @@
using System;
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using ZB.MOM.WW.MxGateway.Contracts;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
@@ -305,6 +307,101 @@ public sealed class WorkerFrameProtocolTests
Nonce);
}
/// <summary>
/// Verifies that under concurrent writers every frame receives a distinct, gap-free sequence in
/// strictly increasing on-wire order — the sequence is stamped by the writer at write time, so the
/// wire order and the stamped sequence always agree (WRK-04).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task WriteAsync_UnderConcurrentCalls_StampsGapFreeMonotonicSequence()
{
const int frameCount = 50;
WorkerFrameProtocolOptions options = CreateOptions();
using MemoryStream stream = new();
WorkerFrameWriter writer = new(stream, options);
await Task.WhenAll(
Enumerable.Range(0, frameCount).Select(_ => writer.WriteAsync(CreateEventEnvelope())));
stream.Position = 0;
WorkerFrameReader reader = new(stream, options);
ulong[] sequences = new ulong[frameCount];
for (int index = 0; index < frameCount; index++)
{
sequences[index] = (await reader.ReadAsync()).Sequence;
}
// On-wire order is strictly increasing 1..frameCount with no gaps or duplicates.
Assert.Equal(Enumerable.Range(1, frameCount).Select(value => (ulong)value), sequences);
}
/// <summary>
/// Verifies that when a control frame and an event frame are both queued behind an in-progress
/// write, the draining lock-holder writes the control frame first even though the event was queued
/// earlier (WRK-07).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task WriteAsync_WhenControlAndEventQueuedTogether_WritesControlFirst()
{
WorkerFrameProtocolOptions options = CreateOptions();
using GatedWriteStream stream = new();
WorkerFrameWriter writer = new(stream, options);
// First write occupies the writer and blocks inside the stream, holding the write lock.
Task firstWrite = writer.WriteAsync(CreateGatewayHelloEnvelope(), WorkerFrameWritePriority.Control);
await AwaitWithTimeoutAsync(stream.FirstWriteStarted);
// Queue an event first, then a control frame, while the writer is blocked. Both wait for the lock.
Task eventWrite = writer.WriteAsync(CreateEventEnvelope(), WorkerFrameWritePriority.Event);
Task controlWrite = writer.WriteAsync(CreateGatewayHelloEnvelope(), WorkerFrameWritePriority.Control);
await Task.Delay(50);
stream.ReleaseFirstWrite();
await AwaitWithTimeoutAsync(Task.WhenAll(firstWrite, eventWrite, controlWrite));
stream.Position = 0;
WorkerFrameReader reader = new(stream, options);
WorkerEnvelope frame1 = await reader.ReadAsync();
WorkerEnvelope frame2 = await reader.ReadAsync();
WorkerEnvelope frame3 = await reader.ReadAsync();
Assert.Equal(WorkerEnvelope.BodyOneofCase.GatewayHello, frame1.BodyCase);
// The control frame jumped ahead of the earlier-queued event.
Assert.Equal(WorkerEnvelope.BodyOneofCase.GatewayHello, frame2.BodyCase);
Assert.Equal(WorkerEnvelope.BodyOneofCase.WorkerEvent, frame3.BodyCase);
}
/// <summary>Verifies a zero negotiated frame maximum keeps the constructor default (IPC-02).</summary>
[Fact]
public void AdoptNegotiatedMaxMessageBytes_WithZero_KeepsDefault()
{
WorkerFrameProtocolOptions options = CreateOptions();
int original = options.MaxMessageBytes;
options.AdoptNegotiatedMaxMessageBytes(0);
Assert.Equal(original, options.MaxMessageBytes);
}
/// <summary>Verifies an in-range negotiated frame maximum is adopted (IPC-02).</summary>
[Fact]
public void AdoptNegotiatedMaxMessageBytes_WithInRangeValue_Adopts()
{
WorkerFrameProtocolOptions options = CreateOptions();
options.AdoptNegotiatedMaxMessageBytes(4 * 1024 * 1024);
Assert.Equal(4 * 1024 * 1024, options.MaxMessageBytes);
}
/// <summary>Verifies a negotiated frame maximum above the worker ceiling is rejected (IPC-02).</summary>
[Fact]
public void AdoptNegotiatedMaxMessageBytes_AboveCeiling_Throws()
{
WorkerFrameProtocolOptions options = CreateOptions();
WorkerFrameProtocolException exception = Assert.Throws<WorkerFrameProtocolException>(
() => options.AdoptNegotiatedMaxMessageBytes((uint)WorkerFrameProtocolOptions.MaxNegotiableFrameBytes + 1));
Assert.Equal(WorkerFrameProtocolErrorCode.InvalidConfiguration, exception.ErrorCode);
}
private static WorkerEnvelope CreateGatewayHelloEnvelope(ulong sequence = 1)
{
return new WorkerEnvelope
@@ -321,4 +418,63 @@ public sealed class WorkerFrameProtocolTests
};
}
// net48 has no Task.WaitAsync(TimeSpan); fail the test rather than hang if the writer misbehaves.
private static async Task AwaitWithTimeoutAsync(Task task)
{
Task completed = await Task.WhenAny(task, Task.Delay(TimeSpan.FromSeconds(5)));
if (completed != task)
{
throw new TimeoutException("Timed out waiting for the frame writer.");
}
await task;
}
private static WorkerEnvelope CreateEventEnvelope()
{
return new WorkerEnvelope
{
ProtocolVersion = GatewayContractInfo.WorkerProtocolVersion,
SessionId = SessionId,
WorkerEvent = new WorkerEvent
{
Event = new MxEvent { SessionId = SessionId },
},
};
}
// A MemoryStream whose first WriteAsync blocks until released, so a test can queue additional frames
// behind an in-progress write and observe the writer's priority ordering.
private sealed class GatedWriteStream : MemoryStream
{
private readonly SemaphoreSlim _release = new SemaphoreSlim(0);
private readonly TaskCompletionSource<bool> _firstWriteStarted =
new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
private int _writeCount;
public Task FirstWriteStarted => _firstWriteStarted.Task;
public void ReleaseFirstWrite() => _release.Release();
public override async Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
{
if (Interlocked.Increment(ref _writeCount) == 1)
{
_firstWriteStarted.TrySetResult(true);
await _release.WaitAsync(cancellationToken);
}
await base.WriteAsync(buffer, offset, count, cancellationToken);
}
protected override void Dispose(bool disposing)
{
if (disposing)
{
_release.Dispose();
}
base.Dispose(disposing);
}
}
}
@@ -449,6 +449,44 @@ public sealed class WorkerPipeSessionTests
await SendShutdownAndWaitAsync(pipePair, runTask, cancellation.Token);
}
/// <summary>
/// Verifies that a DrainEvents control command with <c>max_events = 0</c> is bounded by the
/// worker rather than draining the entire queue into one reply frame: the session passes a
/// capped, non-zero maximum to the runtime session (IPC-04).
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_DrainEventsWithZeroMaxEvents_BoundsTheDrain()
{
using CancellationTokenSource cancellation = new(TimeSpan.FromSeconds(5));
using PipePair pipePair = await PipePair.CreateAsync(cancellation.Token);
FakeRuntimeSession runtime = new() { SuppressDrainForBatchSize = 128 };
WorkerPipeSession session = CreatePipeSession(pipePair.WorkerStream, runtime);
runtime.EnqueueEvent(CreateWorkerEvent(sequence: 11));
Task runTask = session.RunAsync(cancellation.Token);
await CompleteGatewayHandshakeAsync(pipePair, cancellation.Token);
await pipePair.GatewayWriter
.WriteAsync(
CreateControlCommandEnvelope(
"drain-cap-1",
MxCommandKind.DrainEvents,
command => command.DrainEvents = new DrainEventsCommand { MaxEvents = 0 }),
cancellation.Token);
await ReadUntilAsync(
pipePair.GatewayReader,
WorkerEnvelope.BodyOneofCase.WorkerCommandReply,
cancellation.Token);
// The client asked for "all" (0) but the worker must pass a bounded, non-zero cap so the reply
// frame cannot grow without limit.
Assert.NotNull(runtime.LastDrainMaxEvents);
Assert.NotEqual(0u, runtime.LastDrainMaxEvents!.Value);
await SendShutdownAndWaitAsync(pipePair, runTask, cancellation.Token);
}
/// <summary>
/// Verifies that ShutdownWorker returns its OK reply BEFORE the graceful
/// shutdown runs and disposes the runtime session, and that the message
@@ -125,6 +125,14 @@ internal sealed class FakeRuntimeSession : IWorkerRuntimeSession
/// </summary>
public uint? SuppressDrainForBatchSize { get; set; }
/// <summary>
/// Records the <c>maxEvents</c> argument of the most recent non-suppressed
/// <see cref="DrainEvents"/> call — i.e. the effective cap the session passed for an explicit
/// DrainEvents control command. Lets a test assert the worker bounds the drain (IPC-04) rather
/// than forwarding the client's raw <c>max_events = 0</c>.
/// </summary>
public uint? LastDrainMaxEvents { get; private set; }
/// <inheritdoc />
public IReadOnlyList<WorkerEvent> DrainEvents(uint maxEvents)
{
@@ -133,6 +141,8 @@ internal sealed class FakeRuntimeSession : IWorkerRuntimeSession
return Array.Empty<WorkerEvent>();
}
LastDrainMaxEvents = maxEvents;
lock (gate)
{
int drainCount = maxEvents == 0
@@ -10,6 +10,14 @@ public sealed class WorkerFrameProtocolOptions
/// <summary>Default maximum message size in bytes (16 MB).</summary>
public const int DefaultMaxMessageBytes = 16 * 1024 * 1024;
/// <summary>
/// Upper ceiling the worker will accept for a gateway-negotiated frame maximum
/// (<c>GatewayHello.max_frame_bytes</c>, IPC-02). Matches the gateway's own configuration ceiling
/// so a nonsensical negotiated value is rejected at the handshake rather than driving an absurd
/// per-frame allocation. 256 MiB.
/// </summary>
public const int MaxNegotiableFrameBytes = 256 * 1024 * 1024;
/// <summary>Initializes a new instance of the WorkerFrameProtocolOptions class from WorkerOptions.</summary>
/// <param name="options">Worker initialization options.</param>
public WorkerFrameProtocolOptions(WorkerOptions options)
@@ -98,6 +106,36 @@ public sealed class WorkerFrameProtocolOptions
/// <summary>Gets the nonce for startup validation.</summary>
public string Nonce { get; }
/// <summary>Gets the maximum message size in bytes.</summary>
public int MaxMessageBytes { get; }
/// <summary>
/// Gets the maximum worker-frame message size in bytes. Initialized from the constructor and
/// then adopted once from the gateway-negotiated value during the startup handshake
/// (<c>GatewayHello.max_frame_bytes</c>, IPC-02) via <see cref="AdoptNegotiatedMaxMessageBytes"/>,
/// before the message loop starts. Not mutated afterwards, so the single-threaded handshake write
/// is safe for the reader/writer that share this instance.
/// </summary>
public int MaxMessageBytes { get; private set; }
/// <summary>
/// Adopts the gateway-negotiated frame maximum conveyed in <c>GatewayHello.max_frame_bytes</c>
/// (IPC-02). A value of 0 (an older gateway that never set the field) is ignored and the
/// constructor default is kept. A value above <see cref="MaxNegotiableFrameBytes"/> is rejected.
/// </summary>
/// <param name="negotiatedMaxFrameBytes">The gateway-negotiated maximum, or 0 for "keep default".</param>
internal void AdoptNegotiatedMaxMessageBytes(uint negotiatedMaxFrameBytes)
{
if (negotiatedMaxFrameBytes == 0)
{
return;
}
if (negotiatedMaxFrameBytes > MaxNegotiableFrameBytes)
{
throw new WorkerFrameProtocolException(
WorkerFrameProtocolErrorCode.InvalidConfiguration,
$"GatewayHello negotiated frame maximum {negotiatedMaxFrameBytes} exceeds the worker ceiling "
+ $"of {MaxNegotiableFrameBytes} bytes.");
}
MaxMessageBytes = (int)negotiatedMaxFrameBytes;
}
}
@@ -0,0 +1,17 @@
namespace ZB.MOM.WW.MxGateway.Worker.Ipc;
/// <summary>
/// Relative scheduling priority for an outbound worker frame. The single writer task drains all
/// pending <see cref="Control"/> frames before any <see cref="Event"/> frame, so a command reply,
/// fault, heartbeat, or shutdown acknowledgement is not delayed behind a backlog of queued events
/// (WRK-07). Priority only reorders the write; the frame sequence is stamped at actual write time,
/// so the on-wire order and the envelope <c>Sequence</c> always agree (WRK-04).
/// </summary>
public enum WorkerFrameWritePriority
{
/// <summary>Control-plane frame (hello, ready, command reply, heartbeat, fault, shutdown ack). Written ahead of events.</summary>
Control = 0,
/// <summary>Event frame. Written only when no control frame is pending.</summary>
Event = 1,
}
@@ -1,4 +1,5 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
@@ -7,12 +8,40 @@ using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Worker.Ipc;
/// <summary>Writes worker frames to a stream with length-prefixed protobuf serialization.</summary>
/// <summary>
/// Writes worker frames to a stream with length-prefixed protobuf serialization. Callers enqueue a
/// frame at a <see cref="WorkerFrameWritePriority"/> and then contend for a single write lock; whoever
/// holds the lock drains every queued frame, control frames first, so a reply, fault, or heartbeat is
/// never delayed behind an event backlog (WRK-07). The envelope <c>Sequence</c> is stamped by the
/// draining lock-holder at the moment of writing, so the on-wire order and the stamped sequence always
/// agree even under concurrent callers and priority reordering (WRK-04).
/// </summary>
public sealed class WorkerFrameWriter
{
private sealed class PendingFrame
{
public PendingFrame(WorkerEnvelope envelope)
{
Envelope = envelope;
Completion = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
}
public WorkerEnvelope Envelope { get; }
public TaskCompletionSource<bool> Completion { get; }
}
private readonly WorkerFrameProtocolOptions _options;
private readonly SemaphoreSlim _writeLock = new(1, 1);
private readonly SemaphoreSlim _writeLock = new SemaphoreSlim(1, 1);
private readonly Stream _stream;
private readonly object _gate = new object();
private readonly Queue<PendingFrame> _controlFrames = new Queue<PendingFrame>();
private readonly Queue<PendingFrame> _eventFrames = new Queue<PendingFrame>();
// Only ever read/written by the current write-lock holder while draining, so no interlock is
// needed. Starts at 0 and is pre-incremented, so the first written frame carries sequence 1
// (matching the previous behaviour).
private ulong _nextSequence;
/// <summary>Initializes a new instance of the WorkerFrameWriter class.</summary>
/// <param name="stream">Stream to write frames to.</param>
@@ -25,12 +54,25 @@ public sealed class WorkerFrameWriter
_options = options ?? throw new ArgumentNullException(nameof(options));
}
/// <summary>Writes a worker envelope frame to the stream with length prefix.</summary>
/// <summary>Writes a control-priority worker envelope frame to the stream with length prefix.</summary>
/// <param name="envelope">Worker envelope to write.</param>
/// <param name="cancellationToken">Token to cancel the asynchronous operation.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
/// <returns>A task that completes when the frame has been written and flushed.</returns>
public Task WriteAsync(
WorkerEnvelope envelope,
CancellationToken cancellationToken = default)
{
return WriteAsync(envelope, WorkerFrameWritePriority.Control, cancellationToken);
}
/// <summary>Queues a worker envelope frame for writing at the given priority and drains the queue.</summary>
/// <param name="envelope">Worker envelope to write.</param>
/// <param name="priority">Scheduling priority; control frames are written ahead of event frames.</param>
/// <param name="cancellationToken">Token to cancel waiting for the write lock.</param>
/// <returns>A task that completes when the frame has been written and flushed.</returns>
public async Task WriteAsync(
WorkerEnvelope envelope,
WorkerFrameWritePriority priority,
CancellationToken cancellationToken = default)
{
if (envelope is null)
@@ -38,8 +80,120 @@ public sealed class WorkerFrameWriter
throw new ArgumentNullException(nameof(envelope));
}
PendingFrame frame = new PendingFrame(envelope);
lock (_gate)
{
if (priority == WorkerFrameWritePriority.Event)
{
_eventFrames.Enqueue(frame);
}
else
{
_controlFrames.Enqueue(frame);
}
}
// Contend for the single writer: whoever wins drains every currently-queued frame in priority
// order, so this frame is written by this call or by a concurrent caller that got the lock
// first. Either way it completes via its own TaskCompletionSource.
await _writeLock.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
await DrainQueuedFramesAsync().ConfigureAwait(false);
}
finally
{
_writeLock.Release();
}
await frame.Completion.Task.ConfigureAwait(false);
}
// Runs only under _writeLock. Drains control frames before event frames, stamping and writing each.
// The stream write itself is not cancellable: a frame is written atomically or fails, never left
// half-written on the pipe because a caller gave up waiting.
private async Task DrainQueuedFramesAsync()
{
while (true)
{
PendingFrame? frame = DequeueNext();
if (frame is null)
{
return;
}
try
{
await WriteFrameAsync(frame.Envelope).ConfigureAwait(false);
frame.Completion.TrySetResult(true);
}
catch (WorkerFrameProtocolException exception) when (IsPerFrameRejection(exception))
{
// Validation, empty-payload, and oversized-frame errors are specific to this frame and
// do not damage the stream; fail only this frame and keep draining the rest.
frame.Completion.TrySetException(exception);
}
catch (Exception exception)
{
// A stream write/flush failure means the pipe is broken; fail this frame and every frame
// still queued so no caller awaits forever, then stop draining.
frame.Completion.TrySetException(exception);
FailAllQueued(exception);
return;
}
}
}
private static bool IsPerFrameRejection(WorkerFrameProtocolException exception)
{
return exception.ErrorCode is WorkerFrameProtocolErrorCode.InvalidEnvelope
or WorkerFrameProtocolErrorCode.MessageTooLarge
or WorkerFrameProtocolErrorCode.ProtocolVersionMismatch
or WorkerFrameProtocolErrorCode.SessionMismatch;
}
private PendingFrame? DequeueNext()
{
lock (_gate)
{
if (_controlFrames.Count > 0)
{
return _controlFrames.Dequeue();
}
if (_eventFrames.Count > 0)
{
return _eventFrames.Dequeue();
}
return null;
}
}
private void FailAllQueued(Exception exception)
{
lock (_gate)
{
while (_controlFrames.Count > 0)
{
_controlFrames.Dequeue().Completion.TrySetException(exception);
}
while (_eventFrames.Count > 0)
{
_eventFrames.Dequeue().Completion.TrySetException(exception);
}
}
}
private async Task WriteFrameAsync(WorkerEnvelope envelope)
{
WorkerEnvelopeValidator.Validate(envelope, _options);
// Stamp the sequence at the actual point of writing, under the write lock, so the wire order
// and the stamped sequence agree regardless of caller concurrency or priority (WRK-04).
envelope.Sequence = unchecked(++_nextSequence);
int payloadLength = envelope.CalculateSize();
if (payloadLength == 0)
{
@@ -55,26 +209,16 @@ public sealed class WorkerFrameWriter
$"Worker envelope payload length {payloadLength} exceeds the configured maximum of {_options.MaxMessageBytes} bytes.");
}
// Serialize once into a single buffer that carries the 4-byte
// length prefix followed by the payload, then issue one stream write.
// This avoids a second serialization pass (envelope.ToByteArray()
// would re-run CalculateSize internally), a separate prefix array,
// and a separate prefix write.
// Serialize once into a single buffer that carries the 4-byte length prefix followed by the
// payload, then issue one stream write. This avoids a second serialization pass, a separate
// prefix array, and a separate prefix write.
int frameLength = sizeof(uint) + payloadLength;
byte[] frame = new byte[frameLength];
WriteUInt32LittleEndian(frame, (uint)payloadLength);
envelope.WriteTo(new Span<byte>(frame, sizeof(uint), payloadLength));
await _writeLock.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
await _stream.WriteAsync(frame, 0, frameLength, cancellationToken).ConfigureAwait(false);
await _stream.FlushAsync(cancellationToken).ConfigureAwait(false);
}
finally
{
_writeLock.Release();
}
await _stream.WriteAsync(frame, 0, frameLength, CancellationToken.None).ConfigureAwait(false);
await _stream.FlushAsync(CancellationToken.None).ConfigureAwait(false);
}
private static void WriteUInt32LittleEndian(
@@ -18,6 +18,13 @@ public sealed class WorkerPipeSession
private static readonly TimeSpan BackgroundTaskStopTimeout = TimeSpan.FromSeconds(1);
private const uint EventDrainBatchSize = 128;
// Hard cap on how many events a single DrainEvents diagnostic reply may carry. DrainEvents is a
// non-streaming control command, so an unbounded drain (including the max_events = 0 "as many as
// available" request) could pack the whole queue into one session-killing reply frame (IPC-04).
// The gateway request validator rejects requests above its public ceiling; this worker-side cap is
// the backstop and defines the effective per-reply maximum. Kept in step with that public ceiling.
private const uint MaxDrainEventsPerReply = 10_000;
private readonly WorkerFrameProtocolOptions _options;
private readonly Func<int> _processIdProvider;
private readonly Func<IWorkerRuntimeSession> _runtimeSessionFactory;
@@ -28,7 +35,6 @@ public sealed class WorkerPipeSession
private readonly object _commandTaskGate = new();
private readonly HashSet<Task> _activeCommandTasks = new();
private IWorkerRuntimeSession? _runtimeSession;
private long _nextSequence;
// Mutated from the message loop, command tasks, the heartbeat loop and the
// shutdown path; volatile so cross-thread reads observe the latest state
@@ -225,6 +231,12 @@ public sealed class WorkerPipeSession
WorkerFrameProtocolErrorCode.NonceMismatch,
"GatewayHello nonce does not match the worker launch nonce.");
}
// Adopt the gateway-negotiated frame maximum so both ends frame to the same limit instead of
// matched compile-time defaults (IPC-02). Applied here, before the message loop, so every
// post-handshake frame is validated against the negotiated value; the reader and writer share
// this options instance. The hello frame itself was small and already read under the default.
_options.AdoptNegotiatedMaxMessageBytes(gatewayHello.MaxFrameBytes);
}
private Task WriteWorkerHelloAsync(CancellationToken cancellationToken)
@@ -361,8 +373,11 @@ public sealed class WorkerPipeSession
foreach (WorkerEvent workerEvent in events)
{
// Events are the low-priority frame class: the writer holds them behind any pending
// control frame (reply, fault, heartbeat, shutdown ack) so those are not delayed
// behind an event backlog (WRK-07).
await _writer
.WriteAsync(CreateEnvelope(workerEvent), cancellationToken)
.WriteAsync(CreateEnvelope(workerEvent), WorkerFrameWritePriority.Event, cancellationToken)
.ConfigureAwait(false);
}
}
@@ -527,7 +542,12 @@ public sealed class WorkerPipeSession
IWorkerRuntimeSession? runtimeSession = _runtimeSession;
if (runtimeSession is not null)
{
uint maxEvents = command.DrainEvents?.MaxEvents ?? 0;
// Bound the diagnostic drain so max_events = 0 ("as many as available") or an over-large
// request cannot pack the whole queue into one session-killing reply frame (IPC-04).
uint requested = command.DrainEvents?.MaxEvents ?? 0;
uint maxEvents = requested == 0 || requested > MaxDrainEventsPerReply
? MaxDrainEventsPerReply
: requested;
foreach (WorkerEvent workerEvent in runtimeSession.DrainEvents(maxEvents))
{
if (workerEvent.Event is not null)
@@ -1004,19 +1024,16 @@ public sealed class WorkerPipeSession
private WorkerEnvelope CreateBaseEnvelope()
{
// Sequence is deliberately left unset here: the frame writer stamps it at the actual point of
// writing, under its single drain task, so the on-wire order and the stamped sequence agree
// even under concurrent producers and priority reordering (WRK-04).
return new WorkerEnvelope
{
ProtocolVersion = _options.ProtocolVersion,
SessionId = _options.SessionId,
Sequence = NextSequence(),
};
}
private ulong NextSequence()
{
return unchecked((ulong)Interlocked.Increment(ref _nextSequence));
}
private async Task<WorkerReady> InitializeMxAccessAsync(CancellationToken cancellationToken)
{
// RunAsync constructs the runtime session via _runtimeSessionFactory()