Commit Graph

17 Commits

Author SHA1 Message Date
Joseph Doherty afd504d5ae feat(events): opt-in stream-attached acknowledgement on StreamEvents (#227)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m28s
ci / java (push) Failing after 2m21s
ci / portable (push) Failing after 3m33s
A fresh StreamEvents is never replayed, so an initial value published before
the gateway registers the stream's subscriber is lost (lmxopcua#865 —
slow-changing ScanState). Clients had nothing deterministic to wait on.

Contract (additive, proto3):
- StreamEventsRequest.emit_stream_attached = 3 (bool, default false)
- MxEvent.stream_attached = 15 (optional StreamAttached)
- message StreamAttached { uint64 live_after_sequence = 1; }
- OpenSessionReply capability "stream-events-attached-ack"
  (GatewayCapabilities.StreamEventsAttachedAck)

When requested, EventStreamService yields the sentinel as the FIRST message,
after AttachEventSubscriber/AttachEventSubscriberWithReplay has returned, so
receipt implies registration; refused attaches never see it. It precedes the
ReplayGap sentinel and replay on a resume. Shape mirrors ReplayGap: family
UNSPECIFIED, no body, worker_sequence 0.

Opt-in rather than always-on: ScadaBridge's MxGateway adapter stores every
streamed event's worker_sequence as its resume cursor, so an unrequested
sentinel would reset it to 0. Default wire behaviour is unchanged.

.NET client: StreamEventsAsync/StreamEventItemsAsync(after, emitStreamAttached),
MxEventStreamItem.IsStreamAttached, MxGatewaySession.SupportsStreamAttachedAck,
and OpenEventStreamAsync -> MxGatewayEventStream (awaits the sentinel, with
attach timeout; refuses older gateways).

Regenerated Contracts/Generated, the client protoset, Go and Python bindings
(also picks up comment-only drift already on main), synced the Rust vendored
proto and set emit_stream_attached in the Rust StreamEventsRequest literals.
No package versions bumped.

Tests: ordering (sentinel first, then events), race proof (event fanned right
after the sentinel is delivered — mutation-checked: fails when the sentinel is
yielded before registration), resume-with-gap order, refused attach emits no
sentinel, default emits none, contract field numbers, capability, and .NET
client wire tests (await, no-capability, older gateway, timeout).
2026-11-12 07:27:17 -05:00
Joseph Doherty fccc21ba8e feat(events): distinguish ResourceExhausted causes with an mxgw-reason trailer (mxaccessgw#216)
Event-queue overflow, alarm-feed overflow, subscriber cap, session limit and
oversized commands all surfaced as ResourceExhausted. The status code is
unchanged; the gateway now attaches a machine-readable `mxgw-reason` trailer
(constants in Contracts ResourceExhaustedReasons). The .NET client maps
ResourceExhausted to MxGatewayResourceExhaustedException (an
MxGatewayException subclass; additive) exposing Reason/IsOverflow/
IsSubscriberLimit. Documented in docs/ClientBehaviorFixtures.md; the other
clients get a doc note only.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:02:24 -05:00
Joseph Doherty 0d156b56cc refactor(server): SparseArrayExpander throws a domain exception, mapped to InvalidArgument in the gRPC layer (mxaccessgw#190)
Adds SparseArrayException so the Sessions layer no longer throws Grpc.Core.RpcException;
MxAccessGatewayService.MapException maps it to StatusCode.InvalidArgument (client-visible
behaviour unchanged). Adds a service-level mapping test.
2026-11-09 12:10:20 -05:00
Joseph Doherty 82a856b61e fix(metrics-auth): DisableLogin no longer opens /metrics; log correlation pair; host-level tests
- The metrics scheme skips its dashboard-cookie fallback while
  Dashboard:DisableLogin is on, so the dev auto-login handler cannot
  satisfy RequireAuthentication=true.
- Invoke logs client_correlation_id <-> worker envelope id once at Debug.
- MetricsEndpointAuthenticationTests drives the real pipeline: anonymous
  loopback 401 (no /login redirect), Viewer cookie 200, admin key 200,
  non-admin key 403, DisableLogin 401, /health/active 200.
- Docs: DisableLogin behaviour; the /metrics limiter partition is per IP
  vs gRPC per connection, so only the per-key aggregate is shared.
2026-10-22 06:47:36 -04:00
Joseph Doherty 0b4a1682c8 fix(grpc): typed alarm-feed overflow, memoized alarm snapshot, cancellable stream writes, uniform reply correlation id
- StreamAlarms: slow subscribers are completed with
  AlarmFeedSubscriberOverflowException and surface as ResourceExhausted
  (like a StreamEvents overflow) instead of warning-with-stack + Unavailable.
- GatewayAlarmMonitor.StreamAsync reuses the memoized alarm-set projection
  instead of cloning the whole set under the broadcast lock per attach; the
  prefix filter runs outside the lock.
- StreamEvents/StreamAlarms pass the call token to response writes; a write
  failure after client cancel maps to Cancelled without a warning.
- Every MxCommandReply.correlation_id now echoes the request's
  client_correlation_id (previously only constraint-denied replies did;
  normal replies leaked the internal envelope id).
2026-10-22 06:27:36 -04:00
Joseph Doherty 2858e42e80 feat(gateway): forward operator credentials on alarm acknowledgement
Claude-Session: https://claude.ai/code/session_016J6isfcgqy4SB2RK3ZvFvT
2026-10-19 19:40:23 -04:00
Joseph Doherty f40409a688 docs: strip task-tracking IDs from code comments and fill missing XML docs
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m16s
ci / java (push) Successful in 2m5s
ci / portable (push) Failing after 5m9s
Tracking IDs are review bookkeeping that rots in shipped source; the rationale they tagged is kept. Comment-only change (CommentChecker 458 -> 16 false positives).
2026-10-07 15:34:32 -04:00
Joseph Doherty a212e145ac feat(security): DashboardTags on API-key constraints; sessions inherit owner tags (SEC-25 groundwork)
Adds a dashboard event-visibility tag to ApiKeyConstraints, riding in the existing
constraints JSON blob so no auth-store schema migration is needed (design
docs/plans/2026-07-10-dashboard-session-acl-tst15.md sections 3/3.1, open call
settled per its own recommendation). The tag is visibility-only: no read, write,
browse, or subscribe path consults it, and HasRead/HasWriteConstraints ignore it.

GatewaySession gains an immutable, ordinal-ignore-case Tags set stamped at
construction from the owning API key, forwarded by MxAccessGatewayService.OpenSession
from the resolved ApiKeyIdentity — never from the wire request, so a client cannot
label its own session with another tenant's tag. ISessionManager gains a tag-carrying
OpenSessionAsync overload whose default implementation forwards to the tagless one, so
an implementation that does not model tags opens an untagged (least visible) session.

apikey create-key gains --dashboard-tags team-a,team-b (repeatable, trimmed,
de-duplicated; an empty segment is rejected rather than dropped) and list-keys prints
the tags column. No enforcement yet — the EventsHub ACL that consumes the tag is a
later change.
2026-08-17 03:58:14 -04:00
Joseph Doherty fa9eb0c0b4 refactor(sessions): remove the dead ISessionManager.ReadEventsAsync chain
ISessionManager.ReadEventsAsync had zero production call sites: the worker
event channel is drained once by GatewaySession.MapWorkerEventsAsync (the
distributor pump), and every consumer — gRPC subscribers, the dashboard
mirror, the alarm monitor — attaches to the distributor. The interface
member, SessionManager's forwarder, and GatewaySession.ReadEventsAsync are
gone; IWorkerClient/WorkerClient.ReadEventsAsync is untouched, it is the
live worker-channel claim.

No test was removed or rewired: nothing invoked the member through the
interface. Nine ISessionManager test fakes carried a required-member stub
(seven threw NotSupportedException or yielded nothing; EventStreamServiceTests
and GatewaySessionDashboardMirrorTests forwarded to the session; the two
MxAccessGatewayService fakes yielded their Events list) — all nine stubs were
deleted. The MxAccessGatewayService suites' streaming tests already run
through FakeEventStreamService, which reads the same Events list, so their
coverage is unchanged; only the now-inaccurate doc comments on Events /
LastReadEventsSessionId were reworded.

The MapWorkerEventsAsync comment no longer describes a twin to keep in step;
it now states the single-reader claim directly. docs/Sessions.md drops
ReadEventsAsync from the SessionManager member list and from the Run-state
prose. The 2026-08-15 deferred-remediation as-built note records the removal.
2026-08-17 03:46:41 -04:00
Joseph Doherty 3b6a239ed6 fix(GWC-26): attach the alarm monitor's lease before SubscribeAlarms
RunMonitorAsync issued SubscribeAlarms and the first reconcile before the
internal distributor subscriber was attached (via ISessionManager
.ReadAlarmEventsAsync). The pump has been running since MarkReady started the
dashboard mirror and only fans to subscribers registered at fan-out time, so
every transition raised in that two-round-trip window bypassed the alarm feed —
and a missed Acknowledge was never repaired, because ApplyReconcile broadcast
presence deltas only.

- The monitor now takes the internal lease directly from its session BEFORE
  SubscribeAlarms and drains it after the first reconcile; window transitions
  buffer in the lease's bounded channel. Processing them after ApplyReconcile is
  order-safe (ApplyTransition handles alarms the snapshot already placed).
- ISessionManager.ReadAlarmEventsAsync removed — zero remaining callers.
- ApplyReconcile broadcasts an Acknowledge feed transition when a both-present
  alarm's state advanced to ActiveAcked. This is a feed-level repair on the
  AlarmFeedMessage/StreamAlarms surface rebuilt from the worker's own snapshot,
  not MxEvent emission, so the "never synthesize events" rule is untouched;
  the reasoning is recorded on ApplyReconcile.

The alarm-monitor test fakes now hand the monitor a real Ready GatewaySession
with a dashboard mirror, which is what makes the window reproducible.

Docs: docs/Sessions.md and gateway.md alarm-monitor ordering notes.

Refs: archreview/2026-07-12/remediation/10-gateway-core.md GWC-26
2026-08-07 05:40:33 -04:00
Joseph Doherty 20392cf246 fix(archreview): gateway core P0 remediation (GWC-01/02/03, TST-02, TST-12)
Interlocking changes across the gateway server (shared GatewaySession.cs /
SessionManager.cs), committed together:

- GWC-01 (Critical): alarm monitor now attaches as an internal
  (non-counted) distributor subscriber instead of a second raw drain of the
  single worker event channel; WorkerClient._events -> SingleReader with a
  claimed-once guard so a future dual-consumer regression throws loudly.
- GWC-02 (High): faulted sessions are swept in CloseExpiredLeasesAsync
  (IsFaultedReapable + FaultedReason); new FaultedGraceSeconds (default 0).
- GWC-03 (High): configurable MaxSparseArrayLength (default 1_000_000)
  enforced before allocation.
- TST-02 (High, security): StreamEvents attach now enforces the opening key
  id -> PermissionDenied on owner mismatch.
- TST-12 (Medium): CLAUDE.md retention-defaults sentence corrected.

Verified: NonWindows build clean; targeted tests 135/135 on macOS, plus
WorkerClientTests 18/18 on the Windows host.
2026-07-09 05:51:57 -04:00
Joseph Doherty fca978de07 docs(src): add missing XML docs and strip tracking-ID comments
Sweep of 203 source files resolving CommentChecker findings: add
<summary>/<param>/<returns>/<inheritdoc> where missing, and remove
resolved task/issue tracking markers (Tests-NNN, Worker-NNN, Server-NNN,
Task N) from code comments. Comment/doc-only — no logic changes.
Server+Tests build clean under TreatWarningsAsErrors.
2026-07-07 14:09:49 -04:00
Joseph Doherty a43b2ee6af test(sessions): cover OwnerKeyId service-layer forwarding; doc 11-param ctor
Add LastOwnerKeyId capture to FakeSessionManager and assert it equals
"operator01" in OpenSession_WithValidRequest_ReturnsSessionDetails, closing
the gap where OwnerKeyId threading through the service layer had no test
coverage. Add a <remarks> to the 11-param GatewaySession convenience ctor
documenting that OwnerKeyId is null there and authenticated call sites must
use the 12-param overload.
2026-06-15 12:29:16 -04:00
Joseph Doherty f5479f3ca3 feat(sessions): record OwnerKeyId on session creation
Add a nullable string? OwnerKeyId property to GatewaySession that captures
the API key identifier (KeyId) of the authenticated caller that opened the
session. Wire it through ISessionManager.OpenSessionAsync → SessionManager
→ GatewaySession constructor. The gRPC service passes identityAccessor
.Current?.KeyId; internal callers (GatewayAlarmMonitor, DashboardLiveDataService)
pass null. Covers the positive and null cases with two new TDD-first tests.
2026-06-15 12:24:29 -04:00
Joseph Doherty 6bae5ea3a3 Resolve Tests-027..031: flake root cause + coverage gaps
Tests-027  GatewayMetrics exposes its internal Meter; the
           StreamEvents_WhenEventIsWritten_RecordsSendDuration listener
           now filters by ReferenceEquals(instrument.Meter, metrics.Meter)
           instead of Meter.Name, so parallel tests with their own
           GatewayMetrics no longer cross-contaminate the families list.
Tests-028  FakeWorkerClient.Kill now captures LastKillReason;
           SessionManager.KillWorkerAsync tests pin the reason
           propagation end-to-end and cover the blank/null guard. The
           DashboardSessionAdminService kill test pins the literal
           dashboard-admin-kill reason.
Tests-029  Added CloseSessionAsync_BlankSessionId_ReturnsFailure to mirror
           the existing KillWorkerAsync blank-id coverage.
Tests-030  DeleteAsync_WhenStoreRefuses_ReportsFriendlyError renamed and
           extended to assert the dashboard-delete-key audit row with
           Details = not-found-or-active. Added
           DeleteAsync_BlankKeyId_ReturnsFailure.
Tests-031  DashboardSnapshotPublisher reconnect test now measures the
           gap from the first throw inside the fake (firstThrowAt) to
           secondSubscribeAt, isolating Task.Delay from StartAsync /
           scheduling overhead.

All resolved at 2026-05-24; 512/512 gateway tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 09:28:54 -04:00
Joseph Doherty c5e7479ee4 Dashboard: admin-only Close session / Kill worker
Add IDashboardSessionAdminService (Admin-role gate, friendly errors,
audit logging) wrapping a new ISessionManager.KillWorkerAsync that
skips graceful shutdown and cleans up registry/metrics. Sessions,
Workers, and SessionDetails pages render Close / Kill buttons only
when CanManage; the service re-checks the role on every call so
forged clicks return Unauthenticated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 07:10:32 -04:00
Joseph Doherty dc9c0c950c rename: prefix gateway projects/namespaces with ZB.MOM.WW + sln→slnx
Apply the ZB.MOM.WW. prefix to all gateway-side projects, folders,
.csproj/.sln contents, C# namespaces, using directives, generated proto
C# (csharp_namespace + checked-in generated files), InternalsVisibleTo
attributes, project-name string literals (LoadProject, .sln lookups,
worker exe paths, staticwebassets manifest), and the install/script/doc
references that point at any of the above. Migrate the solution from
.sln to .slnx via `dotnet sln migrate` and delete the old file.

External-runtime identifiers are intentionally NOT prefixed so external
configuration keeps working:
- GatewayMetrics.cs MeterName ("MxGateway.Server")
- DashboardAuthenticationDefaults Scheme/Policy ("MxGateway.Dashboard")
- GatewayRequestLoggingMiddleware logger category ("MxGateway.Request")
- StaRuntime thread name ("MxGateway.Worker.STA")
- appsettings.json root section "MxGateway" + env-var prefix
  MxGateway__... and secret-name MxGateway:ApiKeyPepper
- C:\ProgramData\MxGateway\ data dir paths

Also fixes two tests that were not rename-related but became visible
while validating the rename:

- WorkerLiveMxAccessSmokeTests.ShutDownAsync: cancellation that the
  gateway service correctly maps to RpcException(Cancelled) per gRPC
  convention was being misclassified as a stream fault. Added a sibling
  catch on RpcException with StatusCode.Cancelled.

- IntegrationTestEnvironment.ResolveRepositoryRoot: extracted IsRepositoryRoot
  and made it accept either a .git marker OR a .sln/.slnx next to src/
  so the worker-exe walker works in non-git working copies.

clients/proto/proto-inputs.json's protoRoot updated to point at
src/ZB.MOM.WW.MxGateway.Contracts/Protos.

Verified by `dotnet build` and a full `dotnet test` of the .slnx with
MXGATEWAY_RUN_LIVE_{MXACCESS,LDAP,GALAXY}_TESTS=1:
  Tests: 472/472 pass
  Worker.Tests: 280/280 pass (4 dev-rig [Fact(Skip=...)] skipped)
  IntegrationTests: 18/18 pass

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 16:22:23 -04:00