Commit Graph

1081 Commits

Author SHA1 Message Date
Joseph Doherty e69cd0321e fix(dashboard): close three low hub/page races (mxaccessgw#174)
D11: BrowsePage search CTS is cancelled+disposed exactly once under a lock.
D12: the hub refreshes the cached health status before seeding a new connection.
D13: DashboardEventBroadcaster gains DisposeAsync that awaits the pumps (bounded).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:30:04 -05:00
Joseph Doherty f3a452810e fix(dashboard): validate __Host- cookie vs RequireHttpsCookie; surface login flags (mxaccessgw#196)
O19: reject a __Host- CookieName with RequireHttpsCookie=false at startup.
O20: project DisableLogin, AutoLoginUser, RequireHttpsCookie and CookieName into
EffectiveDashboardConfiguration and show them on the Settings page.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:28:21 -05:00
Joseph Doherty 417b9e5a19 feat(apikey): add --pepper-stdin and warn on --pepper (mxaccessgw#194)
--pepper on the command line lands in shell history and the process list.
Add --pepper-stdin (first stdin line), warn on stderr when --pepper is used,
and reject both together. Documented in GatewayProcessDesign.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:27:37 -05:00
Joseph Doherty be0d585f8c fix(server): cap Serilog file-sink retention and size (mxaccessgw#208)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m31s
ci / java (push) Successful in 2m46s
ci / portable (push) Failing after 5m16s
Set retainedFileCountLimit=30, fileSizeLimitBytes=50 MiB and rollOnFileSizeLimit=true on the
file sink in appsettings.json and appsettings.Development.json; document under a new Logging
section in docs/GatewayConfiguration.md; add a guard test over the shipped config.
2026-11-09 12:10:20 -05:00
Joseph Doherty 0d156b56cc refactor(server): SparseArrayExpander throws a domain exception, mapped to InvalidArgument in the gRPC layer (mxaccessgw#190)
Adds SparseArrayException so the Sessions layer no longer throws Grpc.Core.RpcException;
MxAccessGatewayService.MapException maps it to StatusCode.InvalidArgument (client-visible
behaviour unchanged). Adds a service-level mapping test.
2026-11-09 12:10:20 -05:00
Joseph Doherty 8ba5667b7a fix(server): detect a missing API-key pepper by type, not message text (mxaccessgw#195)
Auth.ApiKeys 0.3.0 throws a plain InvalidOperationException("pepper unavailable")
with no typed exception to catch, so take the issue's second option: check
IApiKeyPepperProvider before Create/Rotate and drop the message-matching catch.
Adds create/rotate missing-pepper tests.
2026-11-09 12:10:20 -05:00
Joseph Doherty 6130b1338e fix(server): compare API-key ids ordinal-ignore-case in the verifier cache guard (mxaccessgw#173)
The generation counter and key-id eviction index used ordinal comparison, so an
Invalidate spelled with different case than the presented token missed the guard.
Both now use OrdinalIgnoreCase; two tests added (fail without the fix).
2026-11-09 12:10:20 -05:00
Joseph Doherty 42f69e4f00 fix(server): remove inert worker startup probe retry options (mxaccessgw#188)
WorkerProcessStartedProbe only checks HasExited, so a retry could never change
the outcome. Remove StartupProbeRetryAttempts/StartupProbeRetryDelayMilliseconds,
their validation, the launcher's Polly retry pipeline and the docs; the probe now
runs once under StartupTimeoutSeconds. Stale config keys are ignored by the binder.
2026-11-09 12:10:20 -05:00
Joseph Doherty fad1c11e44 fix(client-dotnet): size the retry outer budget for all attempts (mxaccessgw#197)
ExecuteSafeUnaryAsync capped the whole retry pipeline at one DefaultCallTimeout
while each attempt carried the same deadline, so a DeadlineExceeded first
attempt consumed the whole budget and no retry ever ran. The outer cap is now
MaxAttempts x DefaultCallTimeout + (MaxAttempts-1) x MaxDelay in both
MxGatewayClient and GalaxyRepositoryClient. Adds a slow-first-attempt test
(fails without the fix) and a budget unit test.
2026-11-09 12:10:20 -05:00
Joseph Doherty 20138b60cc fix(client-dotnet): remove duplicate InternalsVisibleTo (mxaccessgw#200)
Keep the csproj AssemblyAttribute item; delete Properties/AssemblyInfo.cs.
2026-11-09 12:10:20 -05:00
Joseph Doherty 58e9b6a05a fix(clients/python): concurrency-safe Session.close() caching the real reply (mxaccessgw#202)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m22s
ci / java (push) Successful in 2m24s
ci / portable (push) Failing after 4m37s
2026-11-09 12:04:13 -05:00
Joseph Doherty 9007966292 fix(clients/go): counter-based fallback when crypto/rand fails in newCorrelationID (mxaccessgw#201) 2026-11-09 12:03:29 -05:00
Joseph Doherty 0ec247e88c feat(clients/go): mxgw-go -require-certificate-validation flag (mxaccessgw#199) 2026-11-09 12:03:00 -05:00
Joseph Doherty f13a747084 feat(clients/go): typed ErrUnauthenticated/ErrPermissionDenied on GatewayError (mxaccessgw#198) 2026-11-09 12:02:31 -05:00
Joseph Doherty e5e7f1854f chore(clients/dotnet): bump Contracts + Client 0.2.1 -> 0.3.0 for publish (#214)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m23s
ci / java (push) Successful in 2m19s
ci / portable (push) Failing after 4m35s
0.2.1 on the feed predates the 488649c AcknowledgeAlarmRequest
operator_password/operator_domain fields; 0.3.0 ships them so OtOpcUa can
adopt operator-authenticated alarm ack (lmxopcua#749).
2026-11-06 16:00:21 -05:00
Joseph Doherty 777fef9cad fix(sessions): drop the deny-NETWORK pipe ACE (#187)
ci / windows-x86 (push) Successful in 1m26s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m30s
ci / portable (push) Failing after 4m47s
NT AUTHORITY\NETWORK marks a network logon, not a remote caller. On windev
a gateway run from an SSH session carries it in its own token, so the deny
entry locked the gateway's same-machine workers out (UnauthorizedAccess on
connect; 15 fake-worker tests failed). The ACL now holds exactly one entry:
FullControl for the gateway's user.
2026-11-06 15:52:46 -05:00
Joseph Doherty 34a7453ac3 fix(sessions): restrict the worker named pipe to the gateway's account (#187)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m23s
ci / java (push) Successful in 2m20s
ci / portable (push) Failing after 4m46s
On Windows each worker pipe is created via NamedPipeServerStreamAcl with a
protected DACL granting FullControl only to the gateway process user (the
worker is launched under the same account) and denying NETWORK logons. On
non-Windows hosts (test matrix) the pipe uses PipeOptions.CurrentUserOnly.
Tests assert the factory ACL and the ACL read back from the created pipe
(Windows), and that a same-user client still connects.
2026-11-06 15:45:50 -05:00
Joseph Doherty 9354d69bf9 fix(diagnostics): redact WriteSecuredBulk/WriteSecured2Bulk payloads (#193)
Adds both secured bulk kinds to GatewayLogRedactor's credential-bearing set,
plus a contract cross-check test that walks every MxCommand payload descriptor
and fails if a secured or password-bearing command kind is not redacted.
2026-11-06 15:45:50 -05:00
Joseph Doherty 5a235776b6 docs: point markdown backlogs at Gitea issues (status lives in Gitea)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m28s
ci / java (push) Successful in 2m29s
ci / portable (push) Failing after 4m58s
2026-11-06 15:32:27 -05:00
Joseph Doherty 8c0a51dc90 Merge branch 'fix/review-2026-10-21' into main (2026-10-21 system review remediation) 2026-10-27 09:56:50 -04:00
Joseph Doherty b2bce9c58b fix(go-client): go vet copylocks in types_alias_test (declare pointer types)
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / portable (push) Failing after 4m8s
ci / windows-x86 (push) Successful in 6m28s
2026-10-22 10:15:58 -04:00
Joseph Doherty c9597f7adc docs(archreview): record stream M final-review fixes 2026-10-22 08:48:09 -04:00
Joseph Doherty abd135a220 Merge stream M (cross-stream review fixes: terminal-worker attach, fault double-count, cancelled/stopping opens, docs) 2026-10-22 08:47:59 -04:00
Joseph Doherty dd623356f4 docs(clients): ReadBulk timeout is one shared deadline for the command, not per tag
Stream E made the worker wait once for all uncached tags. Fix the Go, Python, Rust, Java
and .NET library docs and CLI help text, the proto comment (default is the worker's
1000 ms), and the worker design doc. Regenerated Contracts Generated/ and the Java
generated protos; both diffs are comment-only.
2026-10-22 08:46:29 -04:00
Joseph Doherty 3ffba83723 docs: dashboard live-data session reaping, subscriber-refusal code, distributor-based StreamEvents model
- Sessions.md / GatewayConfiguration.md: the dashboard Browse live-data session is
  unary-only, is reaped by UnattachedSessionGraceSeconds when idle, and reopens via
  EnsureReadyAsync; only the alarm monitor is exempt.
- GatewayConfiguration.md: a refused second subscriber gets ResourceExhausted, not AlreadyExists.
- Grpc.md: rewrite the event streaming model around the SessionEventDistributor pump and
  replay ring; add EventSubscriberLimitReached to the mapping snippet; describe what
  clients see while the gateway is stopping.
2026-10-22 08:43:41 -04:00
Joseph Doherty a0b5e91227 fix(grpc): count a worker fault once, not once per StreamEvents subscriber
WorkerClient.SetFaulted already records mxgateway.faults; EventStreamService added a
WorkerFaulted fault per subscriber, so one fault read as 1+K. The subscriber now records
only its WorkerFault disconnect reason. docs/Metrics.md updated.
2026-10-22 08:42:35 -04:00
Joseph Doherty a178d68b60 fix(sessions): refuse a dead worker at attach, fault on pump readiness failure, keep cancel/stopping out of OpenFailed
- TryAttachWorkerClientAndMarkReadyAsync returns SessionAttachOutcome and refuses a
  worker client that is already Faulted/Closed; the open discards it and fails OpenFailed.
- MapWorkerEventsAsync calls ObserveWorkerTerminalState when its first readiness check
  fails, so a worker that dies between attach and the first pump read faults the session
  and the faulted sweep reaps it.
- A caller-cancelled open rolls back, records no fault and rethrows the cancellation
  (gRPC Cancelled); an attach refused because the drain closed the session reports
  GatewayStopping.
- docs/Sessions.md updated.
2026-10-22 08:41:02 -04:00
Joseph Doherty 2375df5df3 chore(java-client): regenerate protos for stream L comment-only proto edits 2026-10-22 08:30:34 -04:00
Joseph Doherty 4962fa57cb docs(archreview): mark stream E/L rows done after merge; defer MxStatusDetailText test 2026-10-22 08:29:14 -04:00
Joseph Doherty ad67c486e8 Merge stream K (2026-10-21 tracker, cycle-1 re-triage, 2026-08-19 header fix) 2026-10-22 08:29:03 -04:00
Joseph Doherty 20135734b3 Merge stream E (worker: cache eviction, ReadBulk shared deadline + dedupe, alarm poll threshold, STA loop fault, unhandled-exception hook, live control-command facts) 2026-10-22 08:28:55 -04:00
Joseph Doherty ac4edc6e0a docs(archreview): 2026-10-21 remediation tracker, cycle-1 re-triage, fix 2026-08-19 header
Add archreview/2026-10-21 (overall summary + tracker with R21-* rows).
Re-triage the 68 cycle-1 Not-started rows in place: 21 Done, 13 Won't fix,
34 Open (promoted as R21-O1..O34). Correct the 2026-08-19 tracker header
so CLI-47..50, IPC-36/37, TST-35 read Done like their rows.
2026-10-22 08:28:44 -04:00
Joseph Doherty 2fad124bf8 Merge stream L (proto reserved/propagation comments + regen, stale docs, gateway test gaps) 2026-10-22 08:25:25 -04:00
Joseph Doherty 908cf7ccca test(server): cover GatewayConfigPathRules, DashboardRedactor, EventSubscriberReplayAttachment 2026-10-22 08:25:11 -04:00
Joseph Doherty dd3ea056b0 docs: note operator_full_name not propagated; refresh stillpending and deferred-followups status 2026-10-22 08:24:04 -04:00
Joseph Doherty 7f665bf25b docs(contracts): document unpropagated operator_full_name, reserved enum values, correlation echo 2026-10-22 08:23:26 -04:00
Joseph Doherty 38fbbde191 fix(worker): review follow-up — ghost cache entries, ReadBulk dedupe/pumping, fault hygiene
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m15s
ci / portable (push) Failing after 4m19s
ci / windows-x86 (push) Successful in 6m16s
- AddItem/AddItem2/AddBufferedItem drop stale cache entries for a reused handle id
- ReadBulk: dedupe handles within a batch (advise/teardown once, fan out result);
  pump every 16 items in add/advise and teardown phases
- alarm poll stops without recording a fault once StaRuntime.LoopFault is set
- StaRuntime skips the loop-fault handler during a requested shutdown
- StaHung loop fault diagnostic carries type + HRESULT only
- gateway.md documents ReadBulk's shared deadline
2026-10-22 08:23:05 -04:00
Joseph Doherty 7f0720f956 Merge stream A (session lifecycle: startup orphan, sweep metrics, stopping gate, kill-fallback Closed, unattached grace, worker-exit faults session) 2026-10-22 08:21:47 -04:00
Joseph Doherty cc1e895474 fix(sessions): fault a Ready session whose worker reached a terminal state on its own
Live windev finding: after a client sent the ShutdownWorker control command
the worker exited 0 and WorkerClient moved to Closed on the shutdown ack,
but GatewaySession.State stayed Ready (even with an event stream attached)
until its lease expired.

IWorkerClient has no state-change callback, so the session observes the
worker from its own worker-event pump source: WorkerClient enters
Closed/Faulted before completing the event channel, so when the enumeration
ends ObserveWorkerTerminalState sees the terminal state and faults a Ready
session. Faulted rather than Closed: Closed means the gateway closed it and
would linger in the registry; a faulted session stays observable with the
reason and the faulted-session sweep reaps it, accounting SessionClosed
exactly once. Only a Ready session follows its worker, so a gateway-
initiated close is unaffected.

FakeWorkerHarness-based tests cover the ShutdownWorker exit and a broken
worker pipe.
2026-10-22 06:48:31 -04:00
Joseph Doherty 3f9448f619 Merge stream C (gRPC surface: audit aggregation, streaming auth, alarm overflow/snapshot, write cancellation, correlation echo, disconnect reasons, verifier single-flight, opt-in /metrics auth) 2026-10-22 06:47:49 -04:00
Joseph Doherty 82a856b61e fix(metrics-auth): DisableLogin no longer opens /metrics; log correlation pair; host-level tests
- The metrics scheme skips its dashboard-cookie fallback while
  Dashboard:DisableLogin is on, so the dev auto-login handler cannot
  satisfy RequireAuthentication=true.
- Invoke logs client_correlation_id <-> worker envelope id once at Debug.
- MetricsEndpointAuthenticationTests drives the real pipeline: anonymous
  loopback 401 (no /login redirect), Viewer cookie 200, admin key 200,
  non-admin key 403, DisableLogin 401, /health/active 200.
- Docs: DisableLogin behaviour; the /metrics limiter partition is per IP
  vs gRPC per connection, so only the per-key aggregate is shared.
2026-10-22 06:47:36 -04:00
Joseph Doherty ec9f26400a fix(sessions): review follow-up — never orphan a late worker, exempt internal subscribers from unattached grace, tolerate lost close races
- Open path: the attach gate now waits on CancellationToken.None and the
  manager discards (kills + disposes) the late worker client on ANY
  exception out of the attach step, including the caller's own cancellation
  (checked first, so an abandoned open never yields a session). Previously
  an OperationCanceledException from the gate wait escaped before the
  discard and orphaned the x86 worker. The gate Release is guarded against
  ObjectDisposedException.
- Unattached grace: AttachInternalEventSubscriber sets a separate latch so
  the gateway's own alarm-monitor session (internal subscriber only) is
  never reaped; the idle clock starts at Ready (MarkReady stamps
  _readyAtUtc) instead of at open, so a slow worker start does not eat the
  window.
- KillWorkerAsync / CloseSessionCoreAsync: an ObjectDisposedException from
  the close gate on a session that is no longer registered means a
  concurrent close already removed, accounted and disposed it; both now
  return AlreadyClosed with no fault, no second gauge decrement and no sweep
  failure. The general kill catch only decrements the gauge when the
  session is still registered. The sweep no longer counts AlreadyClosed
  results among the sessions it closed.
2026-10-22 06:43:32 -04:00
Joseph Doherty 4ea3672b1e test(integration): ShutdownWorker live fact pins reply-then-exit; log gateway session state
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m10s
ci / portable (push) Failing after 4m15s
ci / windows-x86 (push) Successful in 6m31s
2026-10-22 06:37:55 -04:00
Joseph Doherty 65bca804bb feat(security): opt-in authentication for /metrics (SEC-14)
New MxGateway:Metrics:RequireAuthentication (default false). When true,
/metrics requires an API key with the admin scope (verified through the
cached verifier and the shared API-key failure limiter) or a dashboard
login with a real dashboard role; anonymous localhost and
Authentication:Mode=Disabled do not qualify. An anonymous scrape gets a
plain 401 Bearer challenge, a throttled peer 429. /health stays anonymous
because the off-repo Overview dashboard polls /health/active.

MapZbMetrics returns IEndpointConventionBuilder, so the policy is attached
with RequireAuthorization.
2026-10-22 06:37:31 -04:00
Joseph Doherty ed3a2c9c8b fix(auth): single-flight API-key verification misses; prune key-id index on eviction
CachingApiKeyVerifier's key-id index was only pruned by Invalidate, so it
grew with every token ever verified. A post-eviction callback now drops the
slot (under the same lock as Set + index, so a late callback cannot
unindex a re-cached entry). Concurrent misses for the same token and key
generation now share one inner verification, so a TTL expiry under load no
longer stampedes SQLite; keying the flight on the generation keeps the
revoke guarantee for joiners.
2026-10-22 06:33:18 -04:00
Joseph Doherty 8212b0e550 Merge stream D (dashboard: bounded hub fan-out, alarm reset marker, browse debounce, snapshot config once, health status, per-viewer LRU, circuit retention, doc rewrite) 2026-10-22 06:32:17 -04:00
Joseph Doherty bd7abf4e35 fix(dashboard): re-check hub viewers under lock before starting a pump; document alarm send-timeout tradeoff 2026-10-22 06:32:17 -04:00
Joseph Doherty 285ef64160 feat(sessions): add UnattachedSessionGraceSeconds to reap never-subscribed idle sessions
A client that opens a session and vanishes before it ever streams (e.g.
OtOpcUa's GalaxyDriver on a failed start) pinned a session slot and a live
x86 worker for the full DefaultLeaseSeconds (1800 s), because detach-grace
only applies to sessions that once had a subscriber.

MxGateway:Sessions:UnattachedSessionGraceSeconds (int, default 0 = off,
validated >= 0) makes the existing lease sweep reap a Ready session that
has never attached an event subscriber once it has been idle (no unary
activity) for the window, with the close reason unattached-grace-expired.
Ready-only, idle-from-last-activity and first-subscriber-cancels keep it
from catching slow starts or unary-only clients.

GatewaySession.ExtendLease had no production callers and did not fit the
rule, so it is removed; tests move leases through TouchClientActivity.
2026-10-22 06:31:39 -04:00
Joseph Doherty 069972aff0 test(integration): attach event stream in ShutdownWorker live fact so the gateway observes the worker exit
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m19s
ci / portable (push) Failing after 4m16s
ci / windows-x86 (push) Successful in 6m30s
2026-10-22 06:31:36 -04:00
Joseph Doherty 68d6b5fd6f perf(metrics): cache MxEventFamily names for per-event metric tags
WorkerClient.StageWorkerEvent and StreamEvents called Enum.ToString per
event (per subscriber on the stream path). MxEventFamilyNames serves the
contract-fixed names from a frozen lookup, falling back to ToString for
undefined values so exported tag values are unchanged.
2026-10-22 06:30:57 -04:00