Commit Graph

166 Commits

Author SHA1 Message Date
Joseph Doherty 5b681ee59b fix(SEC-31,SEC-32): identify a probe-slot reservation by version, not by timestamp
ReleaseProbe recognised its own reservation by comparing NextProbeAtTicks to
now + _probeIntervalTicks. RecordInto's rearm-on-trip writes that identical
expression, so a concurrent RecordFailure on the same WindowState whose `now`
lands on the claimer's tick — routine at ~1 ms clock resolution under load — was
mistaken for the caller's own claim. The release then stomped the legitimate
fresh re-arm back to the stale previousProbeAtTicks, which is already due, handing
the next arrival a free probe the re-arm had just closed.

WindowState gains a monotonic ProbeVersion bumped by every writer of
NextProbeAtTicks (TryConsumeProbe's claim and RecordInto's re-arm alike).
TryConsumeProbe returns the stamp it set as part of a ProbeClaim; ReleaseProbe
restores the previous value only while the state's version still equals that
stamp, checking and restoring in one lock(state) section and bumping the version
again on restore so no other stale release can match either.

Test: ProbeSlotRestore_DoesNotStompConcurrentRearmAtSameTick, with the clock held
still so the claim and the interleaved failure necessarily share a tick. Making it
deterministic needed a seam — the claim-to-release window is a few nanoseconds and
racing threads do not hit it (an earlier thread-based attempt passed against the
defective guard three runs out of three, and its end state was ordering-dependent
rather than correctness-dependent, so it was dropped rather than shipped as
theatre). The seam is an internal ProbeReleaseInterleaveHook, null in production,
costing one null check on the already-refused path. Verified as a genuine red
against the timestamp guard: Expected ThrottledByPeer, Actual ProbeAdmitted.
2026-08-07 06:10:14 -04:00
Joseph Doherty acebe18773 fix(SEC-31,SEC-32): make probe admission atomic and stop Reset clearing a shared fallback partition
Two defects found in code review of the limiter rework.

Probe admission was check-then-act across two lock scopes: Check() read
"probe due" under lock(state), released it, then re-acquired to advance
NextProbeAtTicks. A burst of requests arriving together at an interval boundary
could therefore all observe the slot as due and all be admitted, handing the
verifier the very burst the interval exists to bound. The claim is now a single
critical section (TryConsumeProbe). The two layers are still claimed one at a
time — holding two per-state locks at once would need a global lock ordering to
stay deadlock-free — so a slot claimed on the composite partition is compensated
via ReleaseProbe when the aggregate then refuses, which otherwise silently spent
the partition's next slot and pushed the legitimate holder out by a full
interval.

Reset() removed whatever partition the caller resolved to, including the
address's shared fallback partition when the caller's key id had been collapsed
into it by the per-peer cap (or when the token was junk-shaped). That bucket also
carries failures contributed by other key ids from the same address, so one
successful authentication became a reset button for an in-progress spray. Reset
now clears only a partition the caller owns (effectiveKeyId == presented key id);
the shared bucket decays by window expiry instead, and the caller still recovers
through probe admission. The key's aggregate is cleared either way, as designed.

Also applied from the review: closure-free GetOrAdd overload on _partitions, and
a remarks paragraph acknowledging the best-effort O(n) eviction scan under
sustained overflow. Threading the resolved partition key from Check through to
RecordFailure/Reset was declined: Check resolves with mint:false and RecordFailure
with mint:true, and the two can legitimately differ when a concurrent caller fills
the per-peer cap in between — reusing Check's key would record into the wrong
partition and bypass the cap, which is not worth saving one string concat.

Tests (limiter suite 11 -> 14): ProbeAdmission_UnderConcurrentArrivals_
GrantsExactlyOneSlot (200 rounds x 8 barrier-released threads at the boundary),
ProbeAdmission_WhenAggregateRefuses_ReturnsTheClaimedPeerSlot, and
Reset_WithOverCapKeyId_DoesNotClearSharedFallbackPartition. The latter two were
confirmed as genuine reds against the unfixed code; the concurrency test is a
guard — it is deterministically green on the fixed structure but did not
reproduce the original nanosecond-wide window on its own.
2026-08-07 05:57:26 -04:00
Joseph Doherty df710e18a9 fix(SEC-31,SEC-32): re-partition the API-key failure limiter on (peer, key id) with probe admission
The gRPC auth failure limiter partitioned on the key id parsed out of the
*unauthenticated* token and rejected with ResourceExhausted before VerifyAsync
ran. Key ids are not secret — they ride in every token and are listed on the
dashboard — so any network peer could send 10 garbage-secret requests per minute
and deny that key indefinitely: the legitimate holder's correct secret was
refused before it was ever checked, and the success-path Reset that would clear
the block sat behind the verification the block prevented (SEC-31). The tracked
map was also flushable — any `a_b_c`-shaped junk minted a fresh partition (the
`mxgw` literal was never compared), so ~4096 throwaway tokens evicted a blocked
entry and reset the window (SEC-32).

ApiKeyFailureLimiter moves from IsBlocked/RecordFailure/Reset(string peer) to a
partition-pair API: Check/RecordFailure/Reset(ApiKeyThrottlePartition) with an
ApiKeyThrottleDecision result. Two layers share one sliding window — a composite
(transport peer, key id) partition at ApiKeyFailureLimit, and a per-key-id
aggregate across all peers at the new ApiKeyFailureAggregateLimit (default 30)
that bounds a source-rotating sprayer. An over-limit state is now a valve rather
than a wall: one request per the new ApiKeyFailureProbeIntervalSeconds (default
5) is admitted through to the real verifier, so the correct secret always reaches
the constant-time compare and resets both layers. Guarantees preserved: guessing
stays bounded per window, and the failure path still spends no store read per
attempt.

SEC-32 rides the same change set: the interceptor validates token shape (literal
`mxgw` prefix, >= 3 non-empty `_` segments, key id <= 64 chars) before minting a
key-id partition, each transport peer may mint at most 32 of them before the
overflow collapses onto its fallback partition, and eviction prefers fully
expired windows and never drops an over-limit partition below a 2x transient
overshoot ceiling. Throttled attempts increment mxgateway.auth.throttled, tagged
stage=peer|aggregate only — /metrics is unauthenticated (open SEC-14), so no key
material may appear there.

Docs in the same commit: GatewayConfiguration limiter rows plus the two new keys,
the Authentication hot-path paragraph, the Authorization SEC-11 section, and the
limiter / SecurityOptions XML remarks (the old NAT rationale described the
defective keying). Tracking rows flipped to Done with a change-log entry.

Tests: new ApiKeyFailureLimiterTests (11) covering window pruning, composite vs
aggregate trip points, probe cadence, absolute-block mode, reset across both
layers, junk-spray eviction resistance, the per-peer cap, and expired-window
eviction preference; GatewayGrpcAuthorizationInterceptorTests gains the four
SEC-31 contract tests plus NonMxgwToken_FallsBackToTransportPeerPartition (20
total); GatewayOptionsValidatorTests covers both new keys including 0 as a
supported disable value (66 total).
2026-08-07 05:39:10 -04:00
Joseph Doherty 47c0b646a9 fix(logging): redact command values on the shared ILogRedactor seam
ci / windows-x86 (push) Failing after 18s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m13s
ci / portable (push) Successful in 7m6s
GatewayLogRedactor.RedactCommandValue had no production caller. Its only
one was GatewayLogRedactorAdapter on the abandoned
feat/adopt-zb-telemetry-serilog branch; when that work was re-implemented
on main as GatewayLogRedactorSeam the identity half was carried over and
the command-value half was not. Four unit tests kept the policy green, so
it read as wired while masking nothing.

No live leak today — no log statement currently emits a CommandValue
property — but the next one to do so would have written credential-bearing
MXAccess payloads (AuthenticateUser, WriteSecured, WriteSecured2) to every
sink in the clear, with passing tests suggesting otherwise.

Ports the missing half onto the seam: a non-null CommandValue is masked via
the existing policy, gated on CommandMethod. Value logging stays off — the
seam exposes no opt-in — so ordinary values are masked too, matching
RedactCommandValue's default. A null value stays null rather than becoming
the placeholder, and the property is never invented when absent.

Five tests added, three of which were red first on the leak itself
(operator01:hunter2 reaching the assertion unmasked). The other two pin
the null and absent guards. Identity redaction is untouched.

Full NonWindows suite: 785 pass, 45 pre-existing macOS NamedPipe-harness
failures unchanged from baseline (verified by stashing this change).
Build 0 warnings.
2026-07-27 17:01:04 -04:00
Joseph Doherty 2d54ace5d9 chore(health): bump ZB.MOM.WW.Health to 0.2.0
Family version-matrix alignment. No behaviour change — mxgw registers no Akka
checks, and 0.2.0's per-entry `data` object is emitted only when a check
publishes some, so its health payloads are byte-identical.

Note: this repo uses inline package pins, NOT central package management (there is
no Directory.Packages.props), so the version lives in the Server csproj.

Verified: Server builds 0 warnings.
Part of scadaproj docs/plans/2026-07-22-overview-dashboard-impl-plan.md Task 2.3.
2026-07-24 05:54:40 -04:00
Joseph Doherty 8f7ee492ba chore(secrets): bump to Secrets 0.2.3 - visible delete modal (scadaproj#2)
ci / windows-x86 (push) Successful in 1m17s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m29s
ci / portable (push) Successful in 7m59s
0.2.3's Secrets.Ui ships ConfirmDeleteModal's own styles under
collision-proof zb-secrets-* class names. This host links no Bootstrap so
it never exhibited the invisible-modal defect, but it takes the fixed
line for parity; also rides over 0.2.1/0.2.2 (Akka-replicator fixes -
inert here, no replicator in use). Tests: 780 pass, 45 fail on macOS both
before and after the bump (NamedPipeServerStream multi-instance is
Windows-only - the fake-worker pipe harness cannot run on this platform);
zero delta from the bump.

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
2026-07-19 01:22:23 -04:00
Joseph Doherty 2f0cfe33bf fix(security): stop suppressing GHSA-2m69-gcr7-jv3q, pin the patched native lib
ci / windows-x86 (push) Successful in 1m13s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m6s
ci / portable (push) Successful in 7m43s
The suppression's own comment said 'Remove once an upstream fix ships'. It has:
SQLitePCLRaw.lib.e_sqlite3 2.1.12 patches this advisory within the 2.1.x line, so
the premise that no patched release existed is no longer true.

The Server project already resolved 2.1.12, but only incidentally - transitively via
ZB.MOM.WW.Auth.ApiKeys. An explicit PackageReference makes that floor intentional, so
a change to the Auth dependency graph cannot silently regress to the vulnerable 2.1.11.
Removing the suppression restores auditing for exactly that case.

Verified: forced restore of the NonWindows solution reports no NU1903; build is
0 warnings / 0 errors; test results unchanged from baseline (781 passed, 44 failed -
all 44 pre-existing macOS Unix-domain-socket path-length failures in the fake-worker
harness, identical count before and after).
2026-07-18 05:32:28 -04:00
Joseph Doherty b021e292ee chore(secrets): bump ZB.MOM.WW.Secrets 0.1.2 -> 0.2.0
Version hygiene + picks up the G-8 KEK-rotation surface.

NOT a security fix for this repo. An earlier version of this message claimed it
closed GHSA-2m69-gcr7-jv3q; that was wrong. A/B against the 0.1.2 baseline shows
SQLitePCLRaw.lib.e_sqlite3 already resolved 2.1.12, supplied transitively by the
pre-existing ZB.MOM.WW.Auth.ApiKeys 0.1.5 reference.

Note: src/Directory.Build.props still suppresses GHSA-2m69-gcr7-jv3q on the
now-outdated rationale that no patched e_sqlite3 exists. 2.1.12 is patched and
already resolving, so that suppression looks removable - verify separately.
2026-07-18 05:23:37 -04:00
Joseph Doherty e107019a2a fix(deps): bump ZB.MOM.WW.Auth 0.1.4 -> 0.1.5 for the SQLitePCLRaw security fix
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m21s
ci / java (push) Successful in 2m18s
ci / portable (push) Successful in 9m18s
Auth.ApiKeys 0.1.4 pulled SQLitePCLRaw.lib.e_sqlite3 2.1.11, which carries
high-severity advisory GHSA-2m69-gcr7-jv3q. This gateway was genuinely exposed --
verified 2.1.11 resolving before the bump and 2.1.12 after, with the vulnerability
scan now clean.

Auth 0.1.5 is 0.1.4 plus a transitive pin, so there is no API change here.

Suite unchanged from the documented baseline: 781 pass / 44 pre-existing worker-COM
failures on macOS, none Auth/ApiKey/SQLite related.

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
2026-07-18 03:18:58 -04:00
Joseph Doherty e088dfabf7 fix(secrets): honor env-supplied LDAP bind password in live LDAP integration suite; clarify pre-host migration
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m12s
ci / java (push) Successful in 2m0s
ci / portable (push) Successful in 6m51s
2026-07-16 11:24:23 -04:00
Joseph Doherty 22a878c36e feat(secrets): mount /admin/secrets + AddSecretsAuthorization in the dashboard (G-6) 2026-07-16 11:10:13 -04:00
Joseph Doherty b79e119ada feat(secrets): source LDAP bind password via ${secret:ldap/mxgateway/bind}; drop plaintext defaults (G-4) 2026-07-16 10:56:20 -04:00
Joseph Doherty a2538039c0 feat(secrets): register runtime AddZbSecrets on the gateway host 2026-07-16 10:35:35 -04:00
Joseph Doherty 1533cd3909 feat(secrets): pre-host ${secret:} config expansion in GatewayApplication.CreateBuilder (G-4 mechanism) 2026-07-16 10:33:40 -04:00
Joseph Doherty 9552b79481 build(secrets): add ZB.MOM.WW.Secrets 0.1.2 package refs + nuget source mapping 2026-07-16 10:19:06 -04:00
Joseph Doherty 4f5371fdec Merge branch 'fix/archreview-p2' into main (P2 tier: completeness & polish)
# Conflicts:
#	archreview/remediation/00-tracking.md
#	clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayCliSecretRedactor.cs
#	clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs
#	src/ZB.MOM.WW.MxGateway.Worker.Tests/Ipc/WorkerFrameProtocolTests.cs
2026-07-12 22:12:41 -04:00
Joseph Doherty b86c6bb47f docs: complete XML-doc coverage and strip internal tracking IDs from code comments
ci / java (push) Successful in 5m51s
ci / portable (push) Successful in 6m43s
Resolve all CommentChecker findings across the gateway server, worker, tests,
and .NET client (314 -> 0 real issues): add missing <returns>/<summary>/<param>
on public and test members, convert Stream/interface overrides to <inheritdoc/>,
and remove internal task/issue tracking IDs (SEC-*, IPC-*, WRK-*, GWC-*, TST-*,
Client.Dotnet-*) from shipped code documentation while preserving the design
rationale prose. Shipped comments should not carry internal bookkeeping, and
complete XML docs keep the analyzer/TreatWarningsAsErrors gate and generated API
docs clean. The 6 remaining flags are heuristic false positives (MD5, UTC-4,
capacity-1, near-1601) left intact so real documentation is not corrupted.

Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
2026-07-10 06:15:47 -04:00
Joseph Doherty e15c3cb052 fix(dashboard): redact mirror tag values when ShowTagValues off; trim value-log doc (SEC-25, SEC-30)
SEC-25 (near-term hardening; full per-session EventsHub ACL stays deferred to
roadmap item 12): DashboardEventBroadcaster now redacts tag values from a deep
CLONE of the event before mirroring to SignalR when Dashboard:ShowTagValues is
false (the default). Clears MxEvent.value (covers OnDataChange/OnWriteComplete/
OperationComplete/OnBufferedDataChange — their bodies are empty discriminators,
values ride in the top-level field incl. buffered arrays) and the alarm body's
current_value/limit_value. Source event never mutated (shared with the gRPC
path + replay ring) - verified by a source-not-mutated test. Makes the formerly
dead ShowTagValues flag live for the mirror. EventsHub TODO(per-session-acl)
kept and tied to roadmap item 12. Tests: DashboardEventBroadcasterTests (3).

SEC-30: trim docs/Diagnostics.md to mark opt-in command-value logging as NOT
YET IMPLEMENTED (no LogCommandValues knob, RedactCommandValue has no call
sites, no values logged); wiring deferred pending secured-bulk redaction
coverage (SEC-13). No option/call sites added.

Server build clean (0 warnings); Dashboard tests 152/152.

Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
2026-07-09 15:28:24 -04:00
Joseph Doherty cf1b3d40a5 perf(gateway): replay ring LinkedList -> circular array (GWC-14)
Replace the replay buffer's LinkedList<ReplayEntry> (a node allocation per
retained event on the fan-out hot path) with a preallocated ReplayEntry[] ring
sized to ReplayBufferCapacity, tracked by _replayHead (oldest) + _replayCount.
Appending a retained event now allocates nothing.

Behavior preserved exactly: ascending-WorkerSequence append order; capacity
eviction (overwrite head + advance when full); time trim via
_timeProvider.GetUtcNow() cutoff at the same three call sites; oldest-read for
ReplayGap math; both replay-from-sequence query paths + highest-seen tracking;
same _replayLock. Capacity-0 stays retain-nothing (guarded early return, no
modulo-by-zero).

Server build clean (0 warnings); Distributor/Replay tests 33/33 incl. two new
cases (multi-wrap ring keeps newest in order; capacity-1 overwrite + gap).

Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
2026-07-09 15:17:11 -04:00
Joseph Doherty 5e2e40a927 perf(gateway): trim event/command hot-path allocations (GWC-06/07/15, IPC-05)
Behavior-preserving allocation cuts on the per-event/per-command path:
- GWC-06: StreamEvents send-timing uses Stopwatch.GetTimestamp() +
  GetElapsedTime() instead of a per-event Stopwatch allocation (same measured
  span).
- GWC-07/IPC-05 (event): MapEvent transfers ownership of the inner MxEvent
  instead of .Clone()-ing it. Safe: WorkerEvent is parsed fresh per pipe frame
  with the distributor pump as its single consumer (GWC-01), MapEvent runs once
  before fan-out, and every downstream consumer (subscribers, replay ring)
  only READS the event (WorkerSequence is stamped upstream; verified no
  post-mapping mutation). Comment documents the invariant + restore-clone
  caveat if a second consumer is added.
- IPC-05 (command): CreateCommandEnvelope no longer re-clones; MapCommand
  already isolated the graph from the caller-owned gRPC message.
- GWC-15: grpc_stream_queue.depth converts from a per-event push counter to an
  ObservableGauge summing registered channel sources at scrape time only
  (name/semantics unchanged); removes all per-event .Count/lock work.

Kept every load-bearing isolation clone (MapCommand, Invoke, bulk filters,
MapCommandReply). Server build clean (0 warnings); EventStream/Metrics/
Distributor/Mapper tests 62/62 (incl. formerly-flaky queue-depth tests, now
green under the lazy gauge, + 2 new MapEvent ownership tests). Docs: Metrics.md,
Grpc.md.

Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
2026-07-09 15:12:43 -04:00
Joseph Doherty baae59ce3b perf(gateway): pool+coalesce worker pipe frame I/O (GWC-08, IPC-13, IPC-14)
Gateway server-side named-pipe frame path, wire format byte-identical:
- Writer (GWC-08/IPC-13): serialize once into a single ArrayPool-rented buffer
  holding the 4-byte LE length prefix + payload, then one WriteAsync. Removes
  the second serialization pass (ToByteArray re-ran CalculateSize), the
  separate prefix array + second stream write, and per-frame heap allocation.
- Reader (IPC-14): rent the payload buffer from ArrayPool instead of new byte[]
  per frame; read/parse bounded to the real length, return in finally. Matches
  the worker side which already pools.
- Correctness: length tracked separately from (larger) pool capacity; every
  rented buffer returned exactly once in finally incl. the malformed-protobuf
  path; ParseFrom copies into the message so the envelope never aliases the
  returned buffer.

Cross-checked LE-prefix framing agrees across gateway+worker writer/reader.
Server build clean (0 warnings); WorkerFrameProtocol tests 10/10 incl. a new
large-payload-near-cap round-trip (forces pool buffer > frame). The 2 failing
WorkerClient/FakeWorkerHarness tests are the known macOS UDS 104-char path
limit, not a regression.

Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
2026-07-09 14:58:09 -04:00
Joseph Doherty 7d42c85345 fix(dashboard): conditionally restore __Host- cookie prefix (SEC-03)
Restore the __Host- browser guarantees for the default secure deployment
without breaking plaintext/dev:
- DashboardServiceCollectionExtensions PostConfigure now resolves the cookie
  name as: explicit MxGateway:Dashboard:CookieName override wins; else
  __Host-MxGatewayDashboard when SecurePolicy==Always (RequireHttpsCookie
  true); else the plain MxGatewayDashboard default. Guard: never apply the
  __Host- prefix without Secure (browsers silently drop it).
- DashboardAuthenticationDefaults: add SecureCookieName const; keep the plain
  CookieName as the non-secure fallback.
- Docs corrected to the actual conditional contract (five stale claims):
  gateway.md, GatewayProcessDesign.md, ImplementationPlanGateway.md,
  GatewayDashboardDesign.md, CLAUDE.md; GatewayConfiguration.md phrasing
  tightened.
- Test: DashboardCookieOptionsTests asserts the name flips with
  RequireHttpsCookie and that an explicit override wins.

Server build clean (0 warnings); Dashboard tests 149/149.

Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
2026-07-09 14:53:19 -04:00
Joseph Doherty a038363e74 feat(security): apikey --expires CLI flag + dashboard expiry/staleness badge (SEC-10 polish)
ci / portable (push) Failing after 44s
ci / java (push) Failing after 50s
ci / java (pull_request) Failing after 37s
ci / portable (pull_request) Failing after 55s
ci / windows (push) Failing after 8s
ci / windows (pull_request) Failing after 8s
ci / live-mxaccess (push) Has been cancelled
ci / live-mxaccess (pull_request) Has been cancelled
Gateway-side follow-up to the shared auth-lib expiry core (delivered via G-2):

- apikey create-key gains optional --expires — absolute ISO-8601 UTC or relative
  <N>d/<N>h from now; omitted means non-expiring (opt-in, unchanged default).
  Threaded ApiKeyAdminCommand -> parser -> runner into the library's
  CreateKeyAsync(..., expiresUtc, ...). list-keys shows an expiry column and an
  active/expired/revoked status.
- Dashboard API Keys page surfaces expiry: an Expires column (Never when unset)
  and a status badge reading Expired (red) / Expiring (<=7d, amber) / Revoked /
  Active. DashboardApiKeySummary.ExpiresUtc projected in DashboardSnapshotService;
  StatusBadge maps the new states.

Tests: parser (absolute/relative/invalid/none) + end-to-end past-expiry rejection
through the live verifier; dashboard summary suite green. Docs: Authentication.md
(verification-flow expiry step, CLI table + examples, dashboard badge).

Closes the tracked SEC-10 polish (SEC-10 core was already Done).
2026-07-09 09:48:09 -04:00
Joseph Doherty 32d6b48910 feat(gateway): decouple worker event enqueue from the read loop (GWC-04)
The read loop awaited EnqueueWorkerEventAsync inline, which blocks in the bounded
event channel's timed WriteAsync (up to EventChannelFullModeTimeout, default 5s)
when the channel is full with no/slow consumer. A WorkerCommandReply or heartbeat
queued behind an event frame was then stalled, so an in-flight InvokeAsync could
hit CommandTimeout even though the worker replied in time.

Mirror the existing outbound WriteLoopAsync: add an unbounded event staging
channel and a dedicated EventWriteLoopAsync. DispatchEnvelope is now fully
synchronous — the WorkerEvent branch hands the event off with a non-blocking
TryWrite and the read loop never awaits. The event write loop owns the timed
WriteAsync into the bounded channel and the sustained-overflow ProtocolViolation
fault (unchanged contract). Registered in WaitForBackgroundTasks + completed on
close/fault/dispose.

Test: reply arriving after events with a full, consumer-less event channel is
dispatched promptly (no CommandTimeout) — pipe-harness, verified on windev.
Docs: GatewayProcessDesign read/write/event-loop section.
2026-07-09 09:28:13 -04:00
Joseph Doherty c8b3a2281a feat(ipc): negotiate worker frame max, add gRPC headroom, bound DrainEvents (IPC-02/03/04 gateway half)
Proto foundation + gateway-side of the size/backpressure pass:

- IPC-02: add GatewayHello.max_frame_bytes (regen Generated/); gateway sends
  its negotiated worker-frame max in the handshake so the worker can adopt it
  instead of a hard-coded default. Worker read-half lands separately.
- IPC-03: give the pipe frame max envelope-overhead headroom above the public
  gRPC cap (WorkerFrameProtocolOptions.EnvelopeOverheadReserveBytes = 64 KiB;
  default Worker.MaxMessageBytes bumped to 16 MiB + reserve), cross-validate the
  headroom at startup, and pre-check command envelope size in WorkerClient so an
  oversized command fails only that correlation (ResourceExhausted) instead of
  faulting the whole session.
- IPC-04: reject DrainEvents max_events above a public ceiling in the request
  validator (worker per-reply cap lands with the worker half).

Docs: GatewayConfiguration, WorkerFrameProtocol, gateway.md.
Tests: headroom validation, DrainEvents bound, oversized-command per-command
failure (pipe-harness, verified on windev).
2026-07-09 08:49:59 -04:00
Joseph Doherty 11a716a07f test+fix: eliminate Windows full-suite temp-file-lock flakiness
Full-host tests passed in isolation but failed 2-4/run under parallel execution on
Windows with 'the process cannot access the file ... because it is being used by
another process' (macOS never sees it — Unix deletes open files). Two shared-state
parallel collisions, discovered while verifying TST-08:

1. Self-signed cert generation. GatewayTlsBootstrapTests sets process-global
   Kestrel/TLS env vars that GatewayApplication.Build reads; a parallel host-building
   test inherits them mid-run and both generate a cert at the same path, racing on the
   fixed-name '<path>.tmp'. Fixes:
   - SelfSignedCertificateProvider: stage the PFX in a unique '<path>.<guid>.tmp' instead
     of a fixed name, so concurrent/interrupted writers never collide on the temp file
     (real robustness: two instances or a restart-during-write no longer clash). The
     final atomic Move (last-writer-wins) still yields an equivalent cert.
   - TestHostEnvironmentInitializer: default MxGateway__Tls__SelfSignedCertPath to a
     per-process temp path so the suite never writes the shared ProgramData default or
     fights the deployed service; first host-building test generates, the rest load it.
   - GatewayTlsBootstrapTests: [Collection] with DisableParallelization so its global
     env-var mutation cannot bleed into parallel tests (new GlobalEnvironmentCollection).

2. AuthStoreHealthCheckTests opened a real SQLite file; Microsoft.Data.Sqlite's
   connection pool keeps the .db handle open after 'await using', so the finally
   File.Delete threw. Reuse the existing TempDatabaseDirectory helper, which calls
   SqliteConnection.ClearAllPools() before deleting.

Server build clean; affected classes 17/17 on macOS. Windows full-suite verified separately.
2026-07-09 08:14:55 -04:00
Joseph Doherty 74e815c4d7 fix(archreview-p1): SEC-02/12/20 dashboard + observability hardening
- SEC-02: DashboardAuthorizationHandler restricts the loopback and
  Authentication:Mode=Disabled bypasses to read-only. They now satisfy only a
  Viewer-bearing requirement (AnyDashboardRole), never AdminOnly, so anonymous
  localhost can view the dashboard but cannot reach API-key CRUD or session
  Close/Kill at the policy layer (previously guarded only by service re-checks).
- SEC-12: DashboardSessionAdminService emits canonical AuditEvents through
  IAuditWriter (actions dashboard-close-session / dashboard-kill-worker, category
  SessionAdmin) on Success/Failure/Denied, mirroring the API-key audit path so
  destructive session actions leave durable, queryable rows.
- SEC-20: drop the unbounded session_id tag from the exported
  mxgateway.heartbeats.failed counter (per-session detail stays in the snapshot/log).

Docs updated same-change: CLAUDE.md (read-only loopback + 5-min bearer),
GatewayDashboardDesign.md (bypass scoping + session-admin audit), Metrics.md.
Server build clean; 30/30 targeted + 295/295 Dashboard/Security/App/Metrics sweep.
2026-07-09 07:47:51 -04:00
Joseph Doherty 17f16ea181 fix(archreview): security authz+hub cluster (SEC-07/05/08/11) + validation hardening
SEC-07: add QueryActiveAlarmsRequest -> events:read scope arm; fix two tests that
  constructed StreamAlarmsRequest instead of QueryActiveAlarmsRequest.
SEC-05: shorten hub-token lifetime 30m -> 5m; document that the ?access_token= query
  carriage must never be request-logged.
SEC-08: gateway-side CachingApiKeyVerifier (short TTL, keyed on a hash of the presented
  secret) skips the per-call store read+last_used write; CoalescingMarkApiKeyStore bounds
  last_used writes to <=1/key/min; identity constraints are cached. Invalidation is wired
  at the gateway admin sites (revoke/rotate/delete); short TTL backstops out-of-process CLI.
SEC-11: fixed-window rate limit on POST /auth/login + a per-peer (key-id) failure limiter
  checked before VerifyAsync; new MxGateway:Security options bound + validated.

Also fixes regressions from the SEC-01/04/06 commit (c185f62) that a narrow test filter
missed (all now covered by a full-suite checkpoint):
- Rooting check is cross-platform: accepts Windows C:\/UNC forms on Unix so the shipped
  appsettings path validates on the macOS dev box, still rejecting bare filenames.
- AddGatewayConfiguration TryAdds a non-production IHostEnvironment fallback so the validator
  resolves in minimal test/tooling containers; the real host + apikey CLI register the actual
  environment first (TryAdd no-op there).
- Test assembly defaults ASPNETCORE_ENVIRONMENT=Development (ModuleInitializer) so full-host
  tests exercise wiring instead of tripping the SEC-04/06 production guards.
- GatewayOptionsTests asserts the SEC-01 CommonApplicationData-derived default (platform-correct).

archreview: SEC-07/05/08/11 (P1). Verified: NonWindows build clean; full gateway suite
747 passed / 42 failed, where all 42 are the pre-existing macOS named-pipe-harness failures
(Unix-socket path limit) and 0 are validation/regression failures.
2026-07-09 07:31:35 -04:00
Joseph Doherty c185f621a4 fix(archreview): security config guards (SEC-01/04/06)
- SEC-01: derive AuthenticationOptions.SqlitePath / TlsOptions.SelfSignedCertPath
  defaults from SpecialFolder.CommonApplicationData (was Windows-literal strings
  that became CWD-relative files off-Windows); validator rejects non-rooted
  overrides; delete the stray C:\ProgramData\...gateway-auth.db files that
  materialized under src/; add a tree-hygiene test failing on any *.db under src/.
- SEC-04: GatewayOptionsValidator fails startup when IsProduction() && DisableLogin.
- SEC-06: validator rejects LDAP Transport=None in Production; document the
  MxGateway__Ldap__ServiceAccountPassword env override + dev-credential rotation.

Galaxy SnapshotCachePath rooting could not be validator-enforced (bound by the
external GalaxyRepository package, not GatewayOptions) — documented instead.

archreview: SEC-01/04/06 (P1). Verified on the Auth-0.1.4 baseline: Server build
clean, GatewayOptionsValidator + GatewayTreeHygiene tests 53/53.
2026-07-09 06:40:57 -04:00
Joseph Doherty 197731ae2d chore(auth): consume ZB.MOM.WW.Auth 0.1.4 — gain API-key expiry enforcement (archreview G-2)
Bump all four ZB.MOM.WW.Auth.* package refs 0.1.2 -> 0.1.4. The shared
ApiKeyVerifier now rejects any key whose ExpiresUtc is in the past;
existing keys have NULL expiry (never expire), so nothing breaks, and the
auth SQLite DB auto-migrates to schema v3 (nullable expires_utc column) on
first boot. Implement the two IApiKeyAdminStore members added in 0.1.3
(SetScopesAsync/SetEnabledAsync) on the test FakeApiKeyAdminStore.

Build green; no new test failures (the macOS worker-pipe IPC failures are
pre-existing/environmental, identical to the pre-bump baseline).
2026-07-09 06:37:10 -04:00
Joseph Doherty 20392cf246 fix(archreview): gateway core P0 remediation (GWC-01/02/03, TST-02, TST-12)
Interlocking changes across the gateway server (shared GatewaySession.cs /
SessionManager.cs), committed together:

- GWC-01 (Critical): alarm monitor now attaches as an internal
  (non-counted) distributor subscriber instead of a second raw drain of the
  single worker event channel; WorkerClient._events -> SingleReader with a
  claimed-once guard so a future dual-consumer regression throws loudly.
- GWC-02 (High): faulted sessions are swept in CloseExpiredLeasesAsync
  (IsFaultedReapable + FaultedReason); new FaultedGraceSeconds (default 0).
- GWC-03 (High): configurable MaxSparseArrayLength (default 1_000_000)
  enforced before allocation.
- TST-02 (High, security): StreamEvents attach now enforces the opening key
  id -> PermissionDenied on owner mismatch.
- TST-12 (Medium): CLAUDE.md retention-defaults sentence corrected.

Verified: NonWindows build clean; targeted tests 135/135 on macOS, plus
WorkerClientTests 18/18 on the Windows host.
2026-07-09 05:51:57 -04:00
Joseph Doherty fca978de07 docs(src): add missing XML docs and strip tracking-ID comments
Sweep of 203 source files resolving CommentChecker findings: add
<summary>/<param>/<returns>/<inheritdoc> where missing, and remove
resolved task/issue tracking markers (Tests-NNN, Worker-NNN, Server-NNN,
Task N) from code comments. Comment/doc-only — no logic changes.
Server+Tests build clean under TreatWarningsAsErrors.
2026-07-07 14:09:49 -04:00
Joseph Doherty 9ae6bce0c8 fix(dashboard): copy Galaxy summary volatile fields fresh; memoize only O(N) breakdown
Resolves Server-059, Tests-041, Server-060 (2026-06-25 re-review).

DashboardSnapshotService memoized the whole Galaxy summary keyed on the cache
Sequence, but the shared library bumps Sequence only on a heavy refresh: the
steady-state tick, the refresh-failure path, and the age-based status getter all
replace the entry via 'previous with { ... }' at the SAME Sequence. The dashboard
therefore froze LastQueriedAt and, during a Galaxy SQL outage, kept showing
Healthy with no error for the whole outage.

Split DashboardGalaxySummaryProjector into ComputeBreakdown (the O(N) template/
category work, the only sequence-bound part) and BuildSummary (cheap volatile
fields copied fresh). ResolveGalaxySummary now memoizes only the breakdown by
Sequence and rebuilds the summary from the current entry each tick. Removed the
redundant DashboardGalaxyProjector wrapper.

Tests: same-sequence status/error/timestamp now reflected (the regression);
memoization-hit and sequence-invalidation guards; GatewayApplicationTests asserts
the DI container resolves IGalaxyBrowseScopeProvider to GatewayBrowseScopeProvider
(pins the registration-order invariant over the library's no-op default).
2026-06-25 13:42:39 -04:00
Joseph Doherty 80bf4acc4f perf(dashboard): memoize Galaxy summary by cache sequence; document scope-provider identity invariant 2026-06-25 11:48:27 -04:00
Joseph Doherty 8e196a7c83 refactor(gateway): adopt ZB.MOM.WW.GalaxyRepository 0.2.0; delete inline Galaxy code 2026-06-25 11:33:22 -04:00
Joseph Doherty 8678b6cb87 feat(dashboard): host-side Galaxy summary projector over lib cache entry 2026-06-25 11:10:55 -04:00
Joseph Doherty 9c25cc75de build(gateway): add ZB.MOM.WW.GalaxyRepository 0.2.0 package reference
Add packageSourceMapping entry for ZB.MOM.WW.GalaxyRepository in nuget.config
and add the PackageReference (Version 0.2.0) to the Server csproj.  Also set
NuGetAuditMode=direct to suppress the pre-existing NU1903 transitive vulnerability
in SQLitePCLRaw.lib.e_sqlite3 2.1.11 (no upstream fix available yet).
2026-06-25 11:02:26 -04:00
Joseph Doherty 2671639250 fix(gateway): resolve 2026-06-18 array-write review findings
- Server-057: extend []-suffix normalization to AddItemBulk/AddBufferedItem so bulk-added
  array tags bind write-capable handles (authz check, worker bind, and registration kept
  consistent); update gateway.md + client READMEs. Tests: AddItemBulk/AddBufferedItem wiring.
- Server-058: assert []-fallback-resolved bare array names are still denied when out of
  read/write scope and that MaxWriteClassification is enforced on suffixed array registrations.
- Contracts-023/024/025: round-trip + field-19 descriptor pin for MxSparseArray; document
  MxSparseArray in docs/Contracts.md; enumerate it in the protocol-version-3 test summary.
- Tests-040: add wiring tests for the six uncovered sparse-write arms (WriteSecured, Write2,
  WriteSecured2, Write2Bulk, WriteSecuredBulk, WriteSecured2Bulk).

dotnet build + targeted tests green (184 passed).
2026-06-18 10:58:42 -04:00
Joseph Doherty 8a1f037d5a fix(gateway): resolve array attribute constraints by bare name via [] fallback 2026-06-18 03:25:48 -04:00
Joseph Doherty f0ef7ea0a8 feat(gateway): normalize array AddItem suffix and expand sparse writes at the worker boundary 2026-06-18 03:10:13 -04:00
Joseph Doherty 627c17fae1 fix(gateway): reject oversized sparse array total_length with InvalidArgument
Guard against proto uint32 total_length values that exceed Array.MaxLength
before casting; the previous checked cast threw OverflowException (gRPC
Internal) instead of the intended InvalidArgument. Adds tests for the new
guard, for the null-value ArgumentNullException path, and removes the
checked keyword (redundant after the guard).
2026-06-18 02:58:03 -04:00
Joseph Doherty 34a99c783b feat(gateway): add SparseArrayExpander for default-fill partial array writes 2026-06-18 02:52:33 -04:00
Joseph Doherty 52cd0da9f5 feat(gateway): add ArrayAddressNormalizer for bare-name array AddItem 2026-06-18 02:51:37 -04:00
Joseph Doherty 6b5fe6aa82 fix: resolve code-review findings (locally verified)
Server-054/055/056, Contracts-020/021/022, Tests-036/038/039,
IntegrationTests-030/031/032 (+033 deferred to live rig),
Client.Dotnet-026/028/029 (+027 won't-fix), Client.Go-030..034,
Client.Python-032..036, Client.Rust-033..038.

Key fix: SessionEventDistributor orphaned a subscriber that registered after
the pump completed but before disposal (Server-056) -> register paths now
complete late registrants under _lifecycleLock; regression test added. The
racy dashboard-mirror gRPC test made deterministic (Tests-039).

Verified green locally: gateway Tests targeted classes (GatewaySession,
SessionEventDistributor, GatewayOptionsValidator, ProtobufContractRoundTrip,
GatewaySessionDashboardMirror) + dotnet/go/python/rust client suites.
2026-06-17 05:23:14 -04:00
Joseph Doherty 4ab3bd55e5 fix(server): single-clock poll loop + drop dead sync GetReadyWorkerClient (Task 8 review) 2026-06-16 17:04:18 -04:00
Joseph Doherty 4966ef3359 feat(server): bounded worker-ready wait in GatewaySession (default off) 2026-06-16 16:48:02 -04:00
Joseph Doherty ea17528767 feat(server): add MxGateway:Sessions:WorkerReadyWaitTimeoutMs (default off)
Adds WorkerReadyWaitTimeoutMs to SessionOptions (default 0 = disabled),
validates >= 0 in GatewayOptionsValidator, documents it in
GatewayConfiguration.md, and adds validator + default-value tests.
No wait/poll logic is implemented here (that is Task 8).
2026-06-16 16:38:31 -04:00
Joseph Doherty 121ab7e263 fix(galaxy): include undeployed areas in browse + re-root orphaned objects
The hierarchy query returned deployed objects only (deployed_package_id <> 0), so
areas whose containing area is undeployed were orphaned and hidden from /browse —
on wonder, only the lone deployed root area surfaced. Include category-13 Area
objects regardless of deployment, and in GalaxyHierarchyIndex re-root any object
whose parent is absent from the set (e.g. a deleted container area) so nothing
disappears under a phantom parent id.
2026-06-16 12:49:03 -04:00
Joseph Doherty ca443b1903 test+docs(dashboard): assert hub policy under disable-login; correct warning/doc wording 2026-06-16 09:30:13 -04:00
Joseph Doherty a0b21ca225 refactor(dashboard): consistent effective-user in disable-login warning; doc the config read 2026-06-16 09:23:13 -04:00