Commit Graph

819 Commits

Author SHA1 Message Date
Joseph Doherty 756296886b fix(dashboard): generation-scoped idle gate in UnsubscribeAsync — no zero-subscriber pump survives 2026-08-15 20:43:17 -04:00
Joseph Doherty b0f5941e46 fix(dashboard): generation-tagged feed subscribers survive pump teardown races; drain-timeout observability 2026-08-15 20:32:07 -04:00
Joseph Doherty 38dd7678f2 fix(dashboard): guard stale-session batches inside the renderer dispatch; register IDashboardSessionEventSubscriber 2026-08-15 20:26:42 -04:00
Joseph Doherty 10406a3541 fix(dashboard): mutate provider-status state inside the renderer dispatch 2026-08-15 20:17:07 -04:00
Joseph Doherty e245237c2b feat(dashboard): in-process snapshot feed replaces the pages' loopback /hubs/snapshot hop 2026-08-15 20:16:29 -04:00
Joseph Doherty e23f816bfb feat(dashboard): in-process session event subscription feeds the viewer registry — SessionDetailsPage drops its /hubs/events hop 2026-08-15 20:15:52 -04:00
Joseph Doherty d44fe1d6b5 feat(dashboard): AlarmsPage reads provider status from IGatewayAlarmService in-process 2026-08-15 20:10:51 -04:00
Joseph Doherty 59420d8568 refactor(sessions): _subscribers to plain Dictionary — every access is under _lifecycleLock 2026-08-15 20:07:15 -04:00
Joseph Doherty 94dbe9f1af perf(sessions): fold the ReadEventsAsync pass-through into MapWorkerEventsAsync — one fewer iterator per event 2026-08-15 20:07:11 -04:00
Joseph Doherty 935f002dbf perf(grpc): consume the subscriber channel directly in StreamEventsAsync — drops the ReadAllAsync iterator hop 2026-08-15 20:06:48 -04:00
Joseph Doherty 25f07f89dd fix(tests): clear the SQLite pool before deleting the secrets path-guard temp dir — Windows sharing violation 2026-08-15 20:04:59 -04:00
Joseph Doherty a756e47682 docs(plans): deferred-findings remediation plan — 14 tasks over the six deferred findings + the Windows secrets-test bug 2026-08-15 20:03:52 -04:00
Joseph Doherty 15f188e4f9 Merge perf/review-remediation: full 2026-08-15 perf-review remediation — pipe buffers, signal-driven drains, STA message-driven waits, viewer-gated dashboard mirror, pull-model metrics, async audit pipeline, parallel teardown, truncation-safe alarms 2026-08-15 19:25:09 -04:00
Joseph Doherty 2faf243189 chore(plans): all 25 tasks complete — final integration review: ready to merge
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m6s
ci / portable (push) Successful in 8m57s
2026-08-15 18:07:33 -04:00
Joseph Doherty f4b065b9f6 docs(worker): reviewer follow-up comments and tests from the remediation reviews
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m11s
ci / portable (push) Successful in 11m5s
The worker-side half of the review tail. Tests and comments only — nothing here
changes worker behavior, and none of it compiles on the macOS tree (net48/x86),
so it was reviewed line by line against the already-windev-validated files.

- MxAccessHandleRegistryTests gains the multi-candidate case behind
  MxAccessSession.TryGetCachedReadFor's fall-through: one tag under two item
  handles, the lower registered-but-unadvised and the higher advised. Asserted
  at the registry rather than the session because the session's read path needs
  a live MXAccess COM instance; what the registry owes the scan is the stable
  ascending candidate order and a per-item-handle (not per-tag) advice index,
  and both are pinned here along with the fall-through contract in prose.
- A single adversarial lifecycle test — register, advise, re-register the same
  item handle under a new tag, unadvise, unregister the server — asserting every
  index agrees after each step. The individual transitions were already covered;
  what was not was that they compose, and a stale entry in any one index
  resurrects a handle MXAccess has already retired.
- StaWaitHelperTests.WaitForSignalOrMessages_PreSignalledHandle_ReturnsImmediately
  drains pending messages first, like the other two wait tests. Without it a
  stale message can end the wait instead of the handle, failing the
  signal-consumed post-condition for an unrelated reason.
- GatewayTesting.md records the two findings from the Task 24 windev gate:
  SecretsStorePathGuardTests.CreateBuilder_AcceptsSecretsStoreOutsideContentRoot_AndCreatesIt
  fails deterministically on Windows on main too (SQLite pooling holds secrets.db
  open across the cleanup's recursive delete; pre-existing, tracked separately),
  and the StaWaitHelper timing tests' flake signature on a loaded box is a
  message wake — the helper working as designed — not a broken wait.
2026-08-15 17:56:16 -04:00
Joseph Doherty dc2df628e3 chore(followups): reviewer-recommended tests, comments, and hardening from the remediation reviews
The remediation reviews approved every task but left a tail of small notes.
This lands the gateway-side half of them.

Hardening (behavior changes, all narrow):

- BuildFilteredWriteBulkCommand's unreachable default case failed OPEN: a
  fifth bulk-write kind added upstream without a filter case here would have
  shipped the DENIED entries to the worker while reporting them denied to the
  caller. It now throws UnreachableException.
- SqliteCanonicalAuditStore.ListRecentAsync no longer throws on a row it
  cannot date. The retention sweep deliberately preserves such rows (SQLite's
  datetime() yields NULL, so the DELETE never matches), which guaranteed the
  dashboard's recent-audit view would meet one eventually and lose the whole
  page to it. The row is now reported at DateTimeOffset.MinValue with every
  other column intact, behind an optional logger.
- The audit drain loop's finally now completes the channel writer alongside
  detaching the drain, so a producer that raced past the attached check takes
  the write-through branch instead of stranding its event in a buffer nobody
  reads until shutdown. TryComplete is idempotent, so StopAsync is unaffected.

Tests:

- MapCommandReply ownership (Assert.Same on the inner reply), mirroring the
  existing MapEvent ownership test.
- Redactor key-id length boundary at exactly 64 and 65 characters, pinning
  which way it fails. Nothing validates key-id length at creation, so
  docs/Diagnostics.md's "which no issued key id does" is now stated as the
  heuristic it is.
- ApiKeyFailureLimiter.Reset with a PartitionResolution whose partition was
  evicted between the Check and the Reset: inert, and clears nobody else's
  block.
- Constraint-cache concurrency stress: the cap is enforced by the inserting
  thread, so overshoot must be transient and proportional to the in-flight
  inserters, and the cache must settle at or under the cap.
- ListRecentAsync against a raw-SQL undateable row.

Comment/doc accuracy:

- EventsHubViewerRegistry.ReleaseConnection records that it relies on
  SignalR's default sequential per-connection dispatch
  (MaximumParallelInvocationsPerClient = 1).
- A PERF(followup) note on Invoke's double session resolve and why removing it
  needs a SessionManager overload.
- SessionEventDistributor: the volatile-field comment named the pump as the
  lock-free reader, but the pump's single capture point is inside _replayLock;
  the genuinely lock-free reader is SubscriberCount. OnSubscriberOverflow's
  "cannot be observed here" now excepts the DisposeAsync abandon path. The
  churn test names its ConcurrentDictionary bucket-order assumption and that a
  violation surfaces as a read timeout, not a silent pass.
- The two "restores the sequential drain's behavior" claims (SessionManager,
  docs/Sessions.md) were wrong: the sequential drain leaked too, because
  KillWorkerAsync's entry ThrowIfCancellationRequested aborted the whole loop
  on the first session for zero kills. Reworded to "fixes a leak the
  sequential drain also had", with the sweep-bound/shutdown-unbound
  ParallelOptions asymmetry explained.
- ISessionManager.ShutdownAsync's token doc: it degrades the drain to a kill
  sweep rather than cancelling it, with the bounded overrun stated.
  SessionShutdownHostedService.StopAsync records that its cancellation-logging
  branch is now unreachable.
2026-08-15 17:54:31 -04:00
Joseph Doherty 7755745f2f chore(plans): Task 24 windev gate green — slnx+x86 builds clean, worker 499/499, gateway 1039/1040 (1 pre-existing main failure, 1 isolated-pass load flake) 2026-08-15 17:44:27 -04:00
Joseph Doherty b2d8dd70ed chore(plans): Phase B implementation complete; as-built note for Task 17
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m12s
ci / java (push) Successful in 2m26s
ci / portable (push) Successful in 7m56s
2026-08-15 17:35:05 -04:00
Joseph Doherty 58d97ad4e8 perf(worker): memoize event frame size at enqueue; drain stops sizing under the STA's lock 2026-08-15 17:27:47 -04:00
Joseph Doherty b5ea6bb461 fix(alarms): fetch/poll ceilings; truncation-semantics docs; log-format conformance 2026-08-15 17:19:41 -04:00
Joseph Doherty 7c9add3d73 fix(worker): StaWaitHelper regression tests + full-drain contract notes; consolidate wait P/Invoke 2026-08-15 17:17:10 -04:00
Joseph Doherty 896d81e286 docs(worker): record the SemaphoreSlim/STA dispatch invariant + single-waiter contract; harden fake 2026-08-15 17:13:17 -04:00
Joseph Doherty 25cbe5cd3e fix(worker): guard timestamp-format derivation against pathological culture patterns 2026-08-15 17:10:58 -04:00
Joseph Doherty 13583322b5 perf(worker): launcher-configurable event queue capacity 2026-08-15 17:10:54 -04:00
Joseph Doherty f3e1de5f37 fix(alarms): truncation-safe transitions; perf: single-pass alarm parse; configurable poll cadence 2026-08-15 17:04:06 -04:00
Joseph Doherty 94fdc18c3c perf(worker): exact-format timestamp parse and compiled status-field accessors on the event path 2026-08-15 16:59:52 -04:00
Joseph Doherty f4a6cb1db2 perf(worker): signal-driven event drain — removes the 25 ms latency floor and idle wakeups 2026-08-15 16:59:13 -04:00
Joseph Doherty dc9424d3bd perf(worker): message-driven completion waits — the STA pumps continuously while waiting 2026-08-15 16:58:54 -04:00
Joseph Doherty afec56d03b perf(worker): reverse tag index + memoized views + indexed removals in the handle registry 2026-08-15 16:58:02 -04:00
Joseph Doherty f56798aeb9 perf(worker): pooled frame buffers — brings the net48 codec up to the gateway side's GWC-30 pattern 2026-08-15 13:28:29 -04:00
Joseph Doherty 13df92fbd8 chore(plans): Phase A complete — gate green at 1015/1015 2026-08-15 13:25:05 -04:00
Joseph Doherty 95f8ba918d fix(sessions): exception-total shutdown body with non-cancellable kill fallback 2026-08-15 13:19:08 -04:00
Joseph Doherty 0bc13b5292 perf(sessions): bounded-parallel teardown in lease sweep and shutdown 2026-08-15 12:43:30 -04:00
Joseph Doherty a1a38b5538 fix(ipc): cancellation-priority timeout classification; structurally-enforced no-throw cancel send 2026-08-15 12:41:38 -04:00
Joseph Doherty 1742e38c10 fix(events): graceful unregister no longer masquerades as overflow under FailFast 2026-08-15 12:38:20 -04:00
Joseph Doherty 7b2d04605e fix(audit): write-through on completed channel, poison-batch isolation, drain-fault fallback 2026-08-15 12:37:09 -04:00
Joseph Doherty 07b83561d1 docs(events): correct the capture-under-replayLock rationale 2026-08-15 12:34:36 -04:00
Joseph Doherty f920b4cbf5 fix(dashboard): clamped CAS retry loop in snapshot hub connection counter + direct counter tests
Decrement was decrement-first with a single non-retried repair CAS. From zero,
two unmatched decrements (SignalR calls OnDisconnectedAsync for a connection
whose OnConnectedAsync faulted) capture -1 and -2; a real Increment then makes
the count -1, and the first decrementer's stale CompareExchange(0, -1) matches
and resets to zero — erasing a live connection, so the idle gate freezes an open
dashboard. The same lost race also made Decrement report 0 when it had not
written 0.

Clamping now happens inside the compare-and-swap: read, clamp, publish, retry on
loss. A lost race re-reads the fresh value instead of repairing a stale one.

The counter moves to its own file per the one-public-type-per-file convention and
gains direct tests: the zero floor under concurrent unmatched decrements, matched
pairs settling at zero, and an interleaved connect/disconnect stress round. The
stress test asserts the observable invariants only — the specific interleaving
cannot be forced through the public API (verified: the previous implementation
passes it), which its remarks now state rather than implying a reproducer. A hub
wiring test is skipped for the EventsHub reason: driving Hub.OnConnectedAsync
needs caller-clients and connection-context fakes, and the overrides are two
lines of delegation to the tested type.

Also documents that the API-key refresh's pre-gate time check races benignly.
2026-08-15 12:32:58 -04:00
Joseph Doherty 44ca7c8623 fix(dashboard): enforce/document the advised-set cap honestly; cover handle-0 and oversize-read paths 2026-08-15 12:31:36 -04:00
Joseph Doherty 7c1ea12331 perf(ipc): WaitAsync command timeouts + forward WorkerCancel so a timed-out COM call frees the STA 2026-08-15 12:28:09 -04:00
Joseph Doherty 7171892984 perf(grpc): O(1) unconstrained bulk fast path, direct filtered-command build, cache eviction, capacity hints 2026-08-15 12:24:17 -04:00
Joseph Doherty 88d38bb900 perf(auth): allocation-free token parsing; single partition-key build per RPC 2026-08-15 12:23:25 -04:00
Joseph Doherty 3ff073d1ea perf(grpc): transfer reply ownership instead of deep-cloning every worker reply 2026-08-15 12:22:59 -04:00
Joseph Doherty 8e2066b4bd docs(dashboard): restore the advised-set LRU paragraph dropped by the snapshot commit
77c5731 committed this file from a working copy that predated 75e3dc2's
advised-set section, silently deleting it. Puts the paragraph back verbatim;
no other content changes.
2026-08-15 12:22:33 -04:00
Joseph Doherty 9735ac3b7c perf(metrics): pull-model worker queue gauge (fixes last-writer-wins), Interlocked command counters 2026-08-15 12:21:51 -04:00
Joseph Doherty 77c5731b7b perf(dashboard): idle-gate the snapshot tick; cache static config; bound key-list refresh
The snapshot publisher broadcast to Clients.All on every ~1s tick forever, with
zero viewers. Each tick cost a session-registry snapshot and sort, a metrics
snapshot that copies dictionaries under the global metrics lock, a full rebuild
of EffectiveGatewayConfiguration, and a SQLite read of the API key table.

DashboardSnapshotHub now counts live connections into the singleton
DashboardSnapshotHubConnectionCounter (clamped at zero, since SignalR can call
OnDisconnectedAsync for a connection whose OnConnectedAsync faulted). The
publisher drives the snapshot enumerator by hand instead of await foreach: with
no connections it does not call MoveNextAsync at all, so the producing iterator
stays suspended at its yield and no snapshot is built — the gate removes the
build, not just the broadcast. It re-checks once a second, so the first viewer
resumes the tick within about one interval; that viewer is seeded immediately by
DashboardPageBase's synchronous GetSnapshot() and by the hub's OnConnectedAsync.

Two per-tick costs are bounded independently of the gate: the effective
configuration is startup-static (options are bound once at boot and never
reloaded), so it is built once and cached; and the API key summaries refresh at
most every 15s, since the list only changes when an operator creates, rotates,
or revokes a key. Only a successful refresh restarts the interval, so a failed
or timed-out read is still retried on the next tick with the previous summaries
left on screen.
2026-08-15 12:21:49 -04:00
Joseph Doherty e04b1c9199 perf(events): copy-on-write subscriber snapshot in fan-out pump 2026-08-15 12:21:42 -04:00
Joseph Doherty 75e3dc2794 perf(dashboard): LRU cap on the shared live-read session's advised set 2026-08-15 12:21:13 -04:00
Joseph Doherty f1e26fed4f perf(alarms): memoize CurrentAlarms projection, invalidate on mutation 2026-08-15 12:20:51 -04:00
Joseph Doherty ca34a2d65d fix(logging): fail-closed bearer redaction; hoist per-request logger creation 2026-08-15 12:17:27 -04:00