Compare commits

...

35 Commits

Author SHA1 Message Date
Joseph Doherty 6f4c7c3bf1 Keep subtag acks and alarm reads safe on the managed ack path
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (pull_request) Successful in 2m0s
ci / nightly-windev (pull_request) Has been skipped
ci / java (pull_request) Successful in 2m16s
ci / java (push) Successful in 2m19s
ci / portable (push) Failing after 2m31s
ci / portable (pull_request) Failing after 2m36s
ci / windows-x86 (push) Successful in 3m11s
Review fixes for the opt-in managed GUID ack:

- Subtag-mode GUID acks (forced subtag, or failover on the standby) stay on
  the consumer; their GUIDs are synthetic and only the subtag consumer can
  resolve them. Forced-subtag sessions no longer build a managed client.
- A failed managed Initialize/Register/Subscribe drops back to the wnwrap
  ack path instead of failing the alarm subscription.
- Drop stale crefs to a nonexistent Unverified member in
  WnalNativeAckConsumer.
- Unit tests for routing, fallback, and disposal; document
  MXGATEWAY_USE_MANAGED_ALARM_ACK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 13:11:01 -04:00
Joseph Doherty 6affbd57ea feat(worker): add opt-in managed GUID alarm-ack path
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m8s
ci / portable (push) Failing after 2m23s
ci / nightly-windev (pull_request) Has been skipped
ci / windows-x86 (pull_request) Successful in 1m18s
ci / java (pull_request) Successful in 2m3s
ci / portable (pull_request) Failing after 2m17s
Adds a managed acknowledge path over AVEVA's modern alarm client so GUID
acks carry full operator identity (domain + full name) and work on builds
where the wnwrap COM AlarmAckByGUID is E_NOTIMPL. Default off; opt in with
MXGATEWAY_USE_MANAGED_ALARM_ACK=1 (wnwrap behavior unchanged otherwise).

- AaManagedAlarmAckConsumer: reflection wrapper over
  aaAlarmManagedClient.AlarmClient (InitializeConsumer/RegisterConsumer/
  Subscribe/AlarmAckByGUID); EnvGatedFactory degrades to null (wnwrap) if the
  DLL can't load, so alarm reads never break.
- IManagedAlarmAckClient seam. AlarmDispatcher takes an optional client and
  routes GUID acks to it (subscribing + disposing it alongside the reader).
- AlarmCommandHandler gains an optional managedAckFactory; WorkerPipeSession
  wires AaManagedAlarmAckConsumer.EnvGatedFactory at the production sites.
- WnalNativeAckConsumer: confirmed native alarmlst.dll WNAL submit layer
  (WNALalarmReqSetText / WNALUserRequest / WNALUserRequestEx + lpAlmAck
  layout). List acquisition is intentionally not hand-rolled: the alarm-record
  layout and selection semantics are provider-defined and not recoverable from
  alarmlst.dll, so inferring them risks acking the wrong alarm.
- AlarmManagedGuidAckLiveSmokeTests: skip-gated live smoke that pulls a real
  (non-degraded) alarm GUID and asserts the ack APPLIES (Acknowledge
  transition / acked snapshot), not merely rc=0.

By-name acks stay on wnwrap (AlarmClient has no AlarmAckByName). Grounded in
static RE of alarmlst.dll, aaAlarmManagedClient.dll and alarmmgr.exe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-18 12:42:07 -04:00
dohertj2 6fc423f608 Merge pull request 'Retain fake worker fault diagnostics in test fixtures' (#155) from fix/fake-worker-fault-diagnostics into main
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m3s
ci / portable (push) Successful in 8m23s
2026-10-13 10:46:14 -04:00
Joseph Doherty 83aa76c33d Retain fake worker fault diagnostics in test fixtures
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m20s
ci / java (push) Successful in 2m19s
ci / windows-x86 (pull_request) Successful in 1m20s
ci / nightly-windev (pull_request) Has been skipped
ci / java (pull_request) Successful in 2m23s
ci / portable (push) Successful in 9m10s
ci / portable (pull_request) Successful in 9m2s
2026-10-13 10:35:28 -04:00
dohertj2 d779daf6af Merge pull request 'Make framed worker and readiness test gates deterministic' (#153) from fix/framed-worker-test-gates into main
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m22s
ci / java (push) Successful in 2m2s
ci / portable (push) Successful in 8m26s
2026-10-13 09:59:45 -04:00
Joseph Doherty 42c120aa83 Preserve framed reads across fake worker emission gates
ci / windows-x86 (push) Successful in 1m20s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m17s
ci / windows-x86 (pull_request) Successful in 1m25s
ci / nightly-windev (pull_request) Has been skipped
ci / java (pull_request) Successful in 2m27s
ci / portable (push) Successful in 9m43s
ci / portable (pull_request) Successful in 9m47s
2026-10-13 09:42:09 -04:00
dohertj2 19920b13da Merge pull request 'Allow redundant must_use only in generated Rust service traits' (#152) from fix/generated-rust-must-use into main
ci / windows-x86 (push) Successful in 1m25s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m15s
ci / portable (push) Successful in 9m6s
2026-10-13 09:37:53 -04:00
Joseph Doherty a99070005f Fix Rust 1.99 lint for generated service traits
ci / windows-x86 (push) Successful in 1m27s
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m45s
ci / java (pull_request) Successful in 2m55s
ci / windows-x86 (pull_request) Successful in 1m20s
ci / nightly-windev (pull_request) Has been skipped
ci / portable (push) Successful in 10m40s
ci / portable (pull_request) Successful in 10m18s
2026-10-13 09:24:07 -04:00
Joseph Doherty f40409a688 docs: strip task-tracking IDs from code comments and fill missing XML docs
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m16s
ci / java (push) Successful in 2m5s
ci / portable (push) Failing after 5m9s
Tracking IDs are review bookkeeping that rots in shipped source; the rationale they tagged is kept. Comment-only change (CommentChecker 458 -> 16 false positives).
2026-10-07 15:34:32 -04:00
Joseph Doherty f2e0861709 docs: note that psbridge and winrm remote access to the wonder boxes are disabled (2026-09-11)
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m25s
ci / java (push) Successful in 3m34s
ci / portable (push) Successful in 11m28s
Claude-Session: https://claude.ai/code/session_01PpXxRCFznFqE65ay6wAJwe
2026-09-11 09:05:35 -04:00
Joseph Doherty 4f931e4b7e chore(deps): bump ZB.MOM.WW.Auth 0.2.1->0.3.0
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m20s
ci / java (push) Successful in 2m20s
ci / portable (push) Successful in 10m43s
Bumps ZB.MOM.WW.Auth.Abstractions, .ApiKeys, .AspNetCore, and .Ldap in
ZB.MOM.WW.MxGateway.Server.csproj and ZB.MOM.WW.MxGateway.IntegrationTests.csproj.
0.3.0 is additive (new IApiKeyStoreMigrator, SqliteAuthStoreMigrator implements it,
old ctors retained) — no source changes needed. Restore, Release build (0 warnings/
0 errors), and dotnet test on the NonWindows solution are unchanged versus the
pre-bump baseline: 1198 passed, 0 failed, 19 skipped (live-only tests, opt-in via
env vars).

Claude-Session: https://claude.ai/code/session_01Wzg6NUCWHipSexLs2cgRFv
2026-09-03 10:57:10 -04:00
Joseph Doherty d16ae4e267 docs(claude): note the umbrella index's short-row convention for cross-repo propagation
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m34s
ci / portable (push) Successful in 9m14s
Claude-Session: https://claude.ai/code/session_01Wzg6NUCWHipSexLs2cgRFv
2026-09-03 10:47:48 -04:00
Joseph Doherty 74ba889dea docs: record cycle-3 client release
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m21s
ci / java (push) Successful in 2m29s
ci / portable (push) Successful in 9m46s
2026-08-25 12:47:54 -04:00
Joseph Doherty 5776021970 ci(rust): allow generated result size lint
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m24s
ci / portable (push) Successful in 9m31s
2026-08-25 12:28:12 -04:00
Joseph Doherty a11786cbea docs: record batch N portable gates
ci / portable (push) Failing after 17m19s
ci / java (push) Successful in 2m29s
ci / windows-x86 (push) Successful in 5m55s
ci / nightly-windev (push) Has been skipped
2026-08-25 12:09:40 -04:00
Joseph Doherty aea2c74fdf docs: start batch N release gate
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m18s
ci / java (push) Successful in 2m11s
ci / portable (push) Failing after 8m43s
2026-08-25 12:06:34 -04:00
Joseph Doherty 8b147a32a2 docs: record batch M verification
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m19s
ci / java (push) Successful in 2m26s
ci / portable (push) Failing after 5m27s
2026-08-25 12:02:19 -04:00
Joseph Doherty cc143b52cf ci(TST-35): run gateway suite on windev nightly
ci / nightly-windev (push) Has been skipped
ci / java (push) Successful in 2m44s
ci / windows-x86 (push) Successful in 5m56s
ci / portable (push) Failing after 9m21s
2026-08-25 11:56:35 -04:00
Joseph Doherty ad79238c84 docs: record batch L verification
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 1m17s
ci / java (push) Successful in 2m28s
ci / portable (push) Failing after 10m12s
2026-08-25 11:50:48 -04:00
Joseph Doherty 986c6c0ce4 fix(IPC-36): import obsolete attribute namespace
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Successful in 2m4s
ci / java (push) Successful in 2m22s
ci / portable (push) Failing after 9m44s
2026-08-25 11:49:02 -04:00
Joseph Doherty 484028e721 feat(GWC-37,WRK-32): refresh live alarm watch lists
ci / nightly-windev (push) Has been skipped
ci / windows-x86 (push) Failing after 51s
ci / java (push) Successful in 2m8s
ci / portable (push) Failing after 1h1m54s
2026-08-25 11:46:07 -04:00
Joseph Doherty 1b11e89f29 docs: record batch K verification 2026-08-25 11:26:12 -04:00
Joseph Doherty 5cdb3da8c0 fix(CLI-47,CLI-48,CLI-49,CLI-50): align alarm feed clients 2026-08-25 11:25:47 -04:00
Joseph Doherty feadcf5740 docs: record batch J verification 2026-08-25 11:14:33 -04:00
Joseph Doherty d6c5a11ae6 fix(IPC-36,IPC-37): guard secondary contracts 2026-08-25 11:14:14 -04:00
Joseph Doherty f3827c71c9 docs: record batch I verification 2026-08-25 11:01:38 -04:00
Joseph Doherty 1875426e4b fix(SEC-38,SEC-39): preserve aggregate and fallback throttles 2026-08-25 11:00:29 -04:00
Joseph Doherty 780720165a fix(GWC-35): repair re-raised alarm snapshots 2026-08-25 10:58:21 -04:00
Joseph Doherty b0156c685e fix(GWC-33): unregister terminal worker queue gauge 2026-08-25 10:56:23 -04:00
Joseph Doherty f2090fe1b5 docs: record batch H verification 2026-08-25 10:53:03 -04:00
Joseph Doherty 92aae28f07 chore(GWC-36,IPC-38,TST-33,TST-34,TST-36): close cycle-3 currency gaps 2026-08-25 10:52:39 -04:00
Joseph Doherty 4614080e24 docs: normalize continuation design formatting 2026-08-25 10:41:22 -04:00
Joseph Doherty 8e30ec5be9 docs: plan and track cycle-3 P2 continuation 2026-08-25 10:40:55 -04:00
Joseph Doherty 30548da71c docs: design cycle-3 P2 continuation 2026-08-25 10:37:30 -04:00
Joseph Doherty 914495a9f3 chore: ignore project-local worktrees 2026-08-20 12:14:23 -04:00
186 changed files with 7698 additions and 2537 deletions
+9 -6
View File
@@ -4,16 +4,16 @@
# .NET Framework 4.8 / x86 and need MXAccess COM installed, so they build ONLY on a Windows host.
# They are out of scope for this Linux `portable` job — the `windows-x86` job below builds and
# tests them on windev (10.100.0.48) over SSH (see scripts/ci/run-windev-ci.sh), and the nightly
# `nightly-windev` job additionally runs the live-MXAccess smoke, which is scheduled-only so it
# never gates a push.
# `nightly-windev` job additionally runs the full solution build, gateway suite, and live-MXAccess
# smoke, which are scheduled-only so they never gate a push.
name: ci
on:
push:
pull_request:
schedule:
# 06:00 UTC nightly: the `nightly-windev` job re-runs the x86 Worker build + full Worker.Tests on
# windev, then the live-MXAccess smoke (MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1) and a full-slnx build.
# 06:00 UTC nightly: x86 Worker build + Worker.Tests, full-slnx build, gateway tests, then the
# live-MXAccess smoke (MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1), serialized on windev's host lock.
# Push/PR runs skip it — the job is gated `if: github.event_name == 'schedule'`.
- cron: '0 6 * * *'
@@ -91,6 +91,9 @@ jobs:
shell: pwsh
run: ./scripts/check-client-versions.ps1
- name: Code-reviews README consistency
run: python code-reviews/regen-readme.py --check
- name: Gateway fake-worker tests
run: dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj -c Release --no-build
@@ -179,7 +182,7 @@ jobs:
run: ./scripts/ci/run-windev-ci.sh test
nightly-windev:
# TST-25: scheduled deep run — full Worker.Tests + live-MXAccess smoke + full-slnx build on windev.
# TST-35: scheduled deep run — Worker, full-slnx, gateway, and live-smoke tiers on windev.
# Gated to the schedule event via `if:` (not an unsatisfiable `runs-on`), so push/PR runs mark it
# skipped, not queued. Never gates a push. On failure it opens a Gitea issue so a red nightly is
# visible even though nobody watches the Actions page.
@@ -195,7 +198,7 @@ jobs:
PUBLIC_SERVER_URL: https://gitea.dohertylan.com
steps:
- uses: actions/checkout@v4
- name: Full Worker.Tests + live-MXAccess smoke on windev
- name: Worker + gateway tests + live-MXAccess smoke on windev
env:
# WINDEV_SSH_KEY is stored base64-encoded (single line) so Gitea's line-oriented secret
# masker redacts it in the step env echo; run-windev-ci.sh decodes it. A raw multiline PEM
+1
View File
@@ -155,3 +155,4 @@ generated-scratch/
# Agent worktrees (subagent isolation) — never commit
.claude/worktrees/
.worktrees/
+13 -2
View File
@@ -49,7 +49,18 @@ mxaccessgw changes here — remote/push status, the `.proto` contracts (`mxacces
`mxaccess_worker.proto`, `galaxy_repository.proto`), the two-process architecture, shared-lib
consumption, or per-project commands — update the **MxAccessGateway entry in
[`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md)** in the same change so the umbrella index
never drifts from this repo. (Mirrors the same rule in the peer repos.)
never drifts from this repo. (Mirrors the same rule in the peer repos.) **Keep the index row to
a few lines** — since 2026-09-03 the umbrella keeps only short summary rows and parks long
per-project narrative in `../scadaproj/.claude/skills/sister-project-history/<project>.md`
(create `mxaccessgateway.md` there if this gateway ever needs one) and shared-library history
in `../scadaproj/<lib>/CLAUDE.md`.
> ⛔ **Wonder box remote access (2026-09-11): no `psbridge`, no `winrm`.** Both skills are retired/disabled —
> the WinRM remote-exec channel set off the security software on the wonder boxes (vd03/vq03/vq04). Do not
> run remote commands (service control, publish/swap, PowerShell) on those boxes and do not stand up a
> replacement channel until the user clears one. Still allowed: UNC/SMB file copies and logging directly
> into the SQL Servers. Deploy steps that need a command on the box are blocked; see the umbrella
> [`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md) for the current hold.
## Build, Test, Run
@@ -157,7 +168,7 @@ External analysis sources referenced by design docs:
Gateway gRPC clients authenticate with an API key in metadata: `authorization: Bearer mxgw_<key-id>_<secret>`. Keys are stored hashed (with a peppered SHA) in a gateway-owned SQLite DB (default `C:\ProgramData\MxGateway\gateway-auth.db`). Scopes (`session:open`, `session:close`, `invoke:read`, `invoke:write`, `invoke:secure`, `events:read`, `metadata:read`, `admin`) gate specific RPCs; missing → `Unauthenticated`, insufficient → `PermissionDenied`. The `apikey` subcommand on the server exe manages keys; see `src/ZB.MOM.WW.MxGateway.Server/Security/Authentication/`.
Session event streaming is **owner-scoped**: the API key that opened a session is recorded on the session, and every `StreamEvents` attach/reattach is rejected with `PermissionDenied` unless the caller's key id matches the owner. Possessing the `event` scope and knowing a session id is not sufficient — this closes the reconnect/fan-out trust boundary (detach-grace and replay retention are on by default) so an `event`-scoped key cannot attach to another key's retained session.
Session event streaming is **owner-scoped**: the API key that opened a session is recorded on the session, and every `StreamEvents` attach/reattach is rejected with `PermissionDenied` unless the caller's key id matches the owner. Possessing the `events:read` scope and knowing a session id is not sufficient — this closes the reconnect/fan-out trust boundary (detach-grace and replay retention are on by default) so an `events:read`-scoped key cannot attach to another key's retained session.
Dashboard auth is LDAP-backed (separate from the gRPC API-key model). `/login` binds against `MxGateway:Ldap` and maps the user's LDAP groups to `Admin` or `Viewer` via `MxGateway:Dashboard:GroupToRole`, then issues an HTTP-only secure cookie named `__Host-MxGatewayDashboard` when `Dashboard:RequireHttpsCookie` is true (default) and no `Dashboard:CookieName` override is set, else the plain `MxGatewayDashboard` (the `__Host-` prefix requires a Secure cookie). SignalR hubs at `/hubs/{snapshot,alarms,events}` accept either the cookie or a 5-minute bearer minted at `/hubs/token`; `/hubs/alarms` and the `/alarms` page additionally require a **real authenticated dashboard role** (`MxGateway.Dashboard.Alarms`, a requirement type the environment-bypass handler does not handle), because the alarm feed is gateway-central with no per-session provenance and so cannot be tag-scoped — its metadata stays galaxy-wide for authenticated Viewers, but anonymous localhost and `Authentication:Mode=Disabled` do not receive it at all. Dashboard event visibility is **tag-scoped per session** (`IDashboardSessionAcl`, gating both the events hub and the session-details page's in-process subscribe): an Administrator sees every session, while any other caller sees a session only when its tags — inherited from the owning API key's `apikey --dashboard-tags`, never from the client's request — intersect the tags their LDAP groups grant via `Dashboard:GroupToTag`. That grant is **re-resolved from the caller's LDAP group claims against current options on every subscribe** (not read from the tag claims stamped at login, which would outlive a config-revoked grant for the cookie's 8-hour sliding life); login-stamped tag claims are the fallback only for a principal carrying no group claims — i.e. the hub-token principal, whose grant the mint already re-resolved. Untagged sessions follow `Dashboard:UntaggedSessionVisibility` (default `AdminOnly`), and a principal with neither claim kind (anonymous localhost included) is an empty-grant Viewer. `Dashboard:AllowAnonymousLocalhost` (default `true`) grants **read-only** loopback access — it satisfies the Viewer requirement but never the Admin-only requirement, so anonymous localhost can view the dashboard but not reach API-key CRUD, session Close/Kill, or the alarm feed (`Authentication:Mode=Disabled` is scoped the same way). `Dashboard:DisableLogin` (default `false`) auto-authenticates every dashboard request — including remote browsers — as `Dashboard:AutoLoginUser` (default `multi-role`) with both Admin and Viewer roles; dev/test only, never enable in production.
+3 -2
View File
@@ -133,8 +133,9 @@ finding carries an unrecognised Status value. The PowerShell wrapper
`scripts/check-code-reviews-readme.ps1` runs that check and is the intended hook
for CI or a pre-commit step.
> The repo's installed `python` is the real interpreter; the bare `python3`
> alias resolves to the Windows Store stub and fails. Use `python`.
> On Windows, the installed interpreter may be available only as `python`
> because `python3` can resolve to the Windows Store stub. macOS and Linux
> normally use `python3`; CI uses the configured `python` executable.
The per-module `findings.md` files are the source of truth; `README.md` is the
aggregated index and must always agree with them — which the script guarantees.
@@ -155,6 +155,14 @@ Sequence these together rather than piecemeal — several are one change set spa
- ~~When CLI-38 lands, close old **CLI-08** with a pointer here.~~ Done 2026-08-07: CLI-38 landed and old CLI-08 is now `Done` in the first-cycle tracker, pointing at [CLI-38](50-clients.md#cli-38--align-netgojava-on-hresult--0-lands-prior-cli-08-cures-the-doc-drift---medium--p1).
- ~~When WRK-26 lands, its doc section also discharges the WorkerFrameProtocol gap~~ Done 2026-08-07: WRK-26 landed and **IPC-29** is discharged, pointing at [WRK-26](20-worker.md#wrk-26--write-priority-and-overflow-doc-drift-from-the-wrk-07-change---low--p1). ~~When TST-25 lands, revisit old **TST-05** (scheduled live smoke) and **TST-24** (client wire tests), which it unlocks.~~ TST-05 revisited 2026-08-10: `Partially done` in the first-cycle tracker — the `nightly-windev` job closes the scheduled-cadence half, but the finding's coverage-audit half stays open (the live suite reaches all six late-added COM commands and none of the five control commands). TST-24 revisited in the same change and closed `Done`: Go/Rust/Java already had real-server wire tests, and the two genuine gaps (.NET, Python) now have them plus a CI step.
## 2026-08-25 documentation correction
The 2026-08-07 IPC-23 change-log entry's `docs/Grpc.md` deviation rationale is inaccurate:
`DrainEvents` is not private worker IPC. It is a public `MxCommandKind` accepted by the
`Invoke` RPC. Cycle-3 IPC-38 adds the missing public count-cap, byte-cap,
drain-until-empty, and destructive-stream interaction contract to `docs/Grpc.md` while
preserving the historical entry below.
## Change log
| Date | Change |
@@ -27,7 +27,10 @@ This plan is executed as **seven parallel batches**, each by one subagent workin
|---|---|---|
| TST-32 | Blocked (external) | Gitea runner registration-token reset is a web-UI action on the Gitea admin page; no repo change. Operator to-do. |
| SEC-37 | Blocked (external) | Family-wide: TLS on the shared GLAuth lives in `scadaproj/infra/glauth/` (separate repo + docker host). Plan separately; after it lands, move the deployed service to `Production`. |
| CLI-47/48/49/50, IPC-36, IPC-37, TST-33/34/35/36, GWC-33/35/36, SEC-38/39, IPC-38, WRK-34 | Deferred (P2) | Roadmap tier 10; WRK-34 may be picked up by batch F if trivial. |
| CLI-47/48/49/50, IPC-36/37, TST-35 | Not started | Activated by the 2026-08-25 P2 continuation below. |
| GWC-33/35, SEC-38/39 | Done | Closed by continuation batch I; regression counts and commits are recorded below. |
| GWC-36, IPC-38, TST-33/34/36 | Done | Closed by continuation batch H; verification and commit are recorded below. |
| WRK-34 | Done | Completed opportunistically in batch F (`3679e59`): watchdog faults use `CommandMethod=Watchdog` and carry the stuck correlation ID in the diagnostic message. Tracker correction recorded by the continuation audit. |
## Batch specifications
@@ -90,3 +93,167 @@ C is merged **last** (its comment sweep touches the same files E and G edit). Or
## Integration gate (2026-08-20)
All seven branches merged into `main` (A `ee68cc3`, B `f59c34a`, C `f2ab9cc`, D `9f0c3fd`, E `cfbc995`, F `aa07497`, G `c6650c1`). Merged tree: `NonWindows.slnx` 0 warnings; full gateway suite **1183/1183** passed; windev `windev-worker-ci.ps1 -Sha f2ab9cc -Mode test`: x86 Worker build succeeded, `Worker.Tests` **524 passed / 11 skipped / 0 failed**. Follow-ups surfaced by the batches: a worker command to refresh a live alarm subscription's watch-list (would let GWC-37's re-resolve be applied, not just reported); WRK-32's proto reply-field comment deferred until the next proto regen wave; the five clients are bumped but not yet published.
## P2 continuation (planned 2026-08-25)
Current-state audit: refreshed `origin/main` and re-checked every deferred finding and integration-gate follow-up. Sixteen of the seventeen P2 findings remain open; WRK-34 was already completed by `3679e59`. All three follow-ups remain open. Exact target client coordinates are still absent from Gitea (HTTP 404), and `clients/go/v0.2.1` does not exist.
Approved design: [`../../../docs/plans/2026-08-25-cycle-3-p2-followups-design.md`](../../../docs/plans/2026-08-25-cycle-3-p2-followups-design.md) (`30548da`).
Execution plan: [`../../../docs/plans/2026-08-25-cycle-3-p2-followups.md`](../../../docs/plans/2026-08-25-cycle-3-p2-followups.md).
The continuation runs serially in the isolated `.worktrees/c3-p2-followups` worktree on `codex/c3-p2-followups`, with a tracker update at every batch boundary. Codex equivalents are selected from the implementation risk: `gpt-5.6-terra`/medium for contained documentation, CI, and client work; `gpt-5.6-sol`/high for concurrency, authorization, protocol, and live alarm lifecycle changes.
| Batch | Findings / follow-ups | Codex model | Status | Exit gate |
|---|---|---|---|---|
| H — tracker and currency | WRK-34 record, GWC-36, IPC-38, TST-33/34/36 | `gpt-5.6-terra` medium | Done | Currency greps clean; README check in portable CI; NonWindows build clean |
| I — gateway correctness/security | GWC-33/35, SEC-38/39 | `gpt-5.6-sol` high | Done | Focused regression tests green; NonWindows build clean |
| J — secondary contracts | IPC-36/37 | `gpt-5.6-sol` high | Done | Shared launch constants and descriptor parity tests green |
| K — client alarm parity | CLI-47/48/49/50 | `gpt-5.6-terra` high | Done | All five client suites green |
| L — live watch-list/proto wave | watch-list refresh follow-up + WRK-32 proto comment | `gpt-5.6-sol` high | Done | Codegen clean; portable alarm/contract tests green; windev Worker.Tests green |
| M — Windows gateway CI | TST-35 | `gpt-5.6-terra` medium | Done | windev `live` mode proves Worker, gateway, full-slnx, and live smoke tiers |
| N — integration and release | publish five clients + Contracts | `gpt-5.6-sol` high | Done | Full local/windev gates; merged `main`; registry coordinates and Go tag verified |
### Continuation tasklist
#### H — tracker and currency
- [x] Correct WRK-34 from Deferred to Done with commit `3679e59`.
- [x] GWC-36: correct `EventSubscriberReplayAttachment.OldestAvailableSequence` semantics.
- [x] IPC-38: add public `DrainEvents` behavior to `docs/Grpc.md` and correct the cycle-2 omission rationale.
- [x] TST-33: reconcile the five stale/self-contradictory `stillpending.md` claims and mark its live-status boundary.
- [x] TST-34: run `code-reviews/regen-readme.py --check` in portable CI and qualify the interpreter note.
- [x] TST-36: replace prose ``event`` scope references with canonical ``events:read``.
- [x] Record batch-H commit and verification evidence here.
Batch-H verification (2026-08-25): `python3 code-reviews/regen-readme.py --check`
reported the generated README current and consistent; the TST-33 and live-document
TST-36 stale-pattern greps returned no matches; `docs/Grpc.md` contains the public
`DrainEvents` contract; and `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`
succeeded with **0 warnings / 0 errors** in 1.59 s. Batch commit: `92aae28`.
#### I — gateway correctness and security
- [x] GWC-33: unregister terminal clients from the event-queue-depth gauge without shortening fault retention.
- [x] GWC-35: repair acknowledged-to-active snapshot deltas with a Raise transition and instance identity.
- [x] SEC-38: feed/consult the aggregate for valid key IDs that overflow the peer partition cap.
- [x] SEC-39: prove a null effective key ID cannot reset the shared fallback partition.
- [x] Run focused tests and clean NonWindows build; record counts and commits here.
Batch-I verification (2026-08-25): the combined `WorkerClientTests`,
`GatewayAlarmMonitorAttachOrderTests`, and `ApiKeyFailureLimiterTests` filter passed
**55/55** tests in 12 s; `dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx`
succeeded with **0 warnings / 0 errors** in 3.42 s. Finding commits: GWC-33
`b0156c6`, GWC-35 `7807201`, and SEC-38/SEC-39 `1875426`.
#### J — secondary contracts
- [x] IPC-36: centralize all six worker-launch environment names in Contracts and consume them on both sides.
- [x] IPC-36: document variable/config/default mapping and test launcher completeness.
- [x] IPC-37: add normalized retained-vs-served Galaxy descriptor parity test.
- [x] Run focused tests and clean NonWindows build; record counts and commits here.
Batch-J verification (2026-08-25): the combined `WorkerProcessLauncherTests` and
`GalaxyContractParityTests` filter passed **10/10** tests in 1 s; the descriptor
mutation proof identified the changed protobuf message; and `dotnet build
src/ZB.MOM.WW.MxGateway.NonWindows.slnx` succeeded with **0 warnings / 0
errors** in 2.20 s. IPC-36/IPC-37 commit: `d6c5a11`.
#### K — client alarm-feed parity
- [x] CLI-47: make .NET text status flags explicit and document deliberate JSON differences.
- [x] CLI-48: test provider/snapshot status arms in .NET, Go, and Java; add Go stream-alarm scaffold.
- [x] CLI-49: export Go aliases for `AlarmProviderStatus` and `AlarmSnapshotStatus`; fix five-arm documentation.
- [x] CLI-50: stream Python alarm rows incrementally with an unbounded default and bounded opt-in.
- [x] Run .NET, Go, Python, Rust, and Java client suites; record counts and commits here.
Batch-K verification (2026-08-25): `dotnet test
clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx` passed **134/134** portable
tests (1 live test skipped); `go test ./...` passed both tested packages;
`python -m pytest` passed **169/169** portable tests (1 opt-in TLS test
skipped); `cargo test --workspace` passed **108/108** tests; and
`JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test` passed **132/132** tests.
CLI-47/CLI-48/CLI-49/CLI-50 commit: `5cdb3da`.
#### L — live watch-list refresh and protobuf wave
- [x] Add explicit fallback-enabled subscription semantics for an initially empty discovered list.
- [x] Add and round-trip `UpdateAlarmWatchListCommand` and its `MxCommandKind`.
- [x] Route the command through the worker executor and alarm handler on the STA.
- [x] Build/subscribe/attach a replacement standby before swap; preserve the old standby on failure.
- [x] Preserve current primary/subtag mode during a successful replacement.
- [x] Re-resolve on the existing gateway reconciliation cadence and send only material changes/recovery.
- [x] Refresh immediately at failover edge, update acknowledged fingerprint only after worker OK, and reconcile after success.
- [x] Expand WRK-32's public reply-field comment to the sequential stale-callback/recency case.
- [x] Replace IPC-36's internal env-var name in the proto comment with the public configuration key.
- [x] Regenerate Contracts and all client generated trees once; run codegen and all affected suites.
- [x] Verify x86 Worker build and Worker.Tests on windev; record SHA/counts here.
Batch-L verification (2026-08-25): `pwsh scripts/check-codegen.ps1` passed all
four descriptor/C#/Rust/Go/Python freshness checks, and Java
`checkGeneratedClean` passed. The exact portable alarm/contract/descriptor
filter passed **77/77**; `NonWindows.slnx` built with **0 warnings / 0 errors**;
the .NET, Go, Python, Rust, and Java client suites were green (respectively
**134 passed / 1 live skip**, both Go packages, **169 passed / 1 opt-in TLS
skip**, **108 passed**, and **132 passed**). Windev `test` mode at
`986c6c0ce4f02c62b4f59ec44780de5af6d291b8` built the x86/net48 Worker with
**0 warnings / 0 errors** and passed Worker.Tests **529 passed / 11 opt-in
skips / 0 failed**. Feature/proto commit: `484028e`; Batch-J x86 compile
follow-up: `986c6c0`.
#### M — Windows gateway CI
- [x] TST-35: add the serialized net10 gateway suite to windev `live` mode.
- [x] Update `docs/GatewayTesting.md` and `scripts/ci/README.md` with order/failure semantics.
- [x] Execute `live` mode on windev and record Worker, gateway, full-slnx, and live-smoke evidence here.
Batch-M verification (2026-08-25): windev `live` mode at exact commit
`cc143b52cf04be0b2f08350922a987b2e250bc1d` completed the serialized sequence in
**4m45s** with remote exit code 0. The x86/net48 Worker build succeeded with **0
warnings / 0 errors** in 22.59 s; Worker.Tests passed **529**, skipped **11**
opt-in probes, and failed **0**; the full Windows `slnx` build succeeded with **0
warnings / 0 errors** in 43.24 s; the net10 gateway suite passed **1193/1193**
in 53 s; and the live-MXAccess smoke passed **8/8** in 1m46s. TST-35 commit:
`cc143b5`.
#### N — integration and release
- [x] Run `git diff --check`, codegen, client-version, README, NonWindows build, and full gateway suite gates.
- [x] Run all five client suites on the final candidate.
- [x] Run final windev `live` gate at the exact candidate SHA.
- [x] Merge/push the verified continuation to `main`; wait for green CI.
- [x] Re-query all package coordinates and Go tag immediately before publish; abort on collision.
- [x] Publish .NET Client + Contracts 0.2.1, Python 0.2.1, Rust 0.2.1, Java 0.2.2.
- [x] Create/push `clients/go/v0.2.1` at the exact release commit.
- [x] Verify every registry coordinate and Go module/tag, update `docs/ClientPackaging.md`, and record immutable release evidence here.
Batch-N portable verification (2026-08-25): `git diff --check`, codegen freshness,
client-version freshness, and the generated README check all passed without changing
the worktree. `NonWindows.slnx` built with **0 warnings / 0 errors**; the gateway
suite passed **1193/1193**, and the portable integration suite passed **5** with
**19** live-only skips. The final .NET, Go, Python, Rust, and Java client gates were
green (respectively **134 passed / 1 live skip**, all Go packages built/tested with
clean `gofmt`, **169 passed / 1 opt-in TLS skip** from a clean virtual environment,
**108 passed** plus clean `cargo fmt`/Clippy, and **132 passed / 0 failed** with the
generated Java tree unchanged).
Batch-N integration/release verification (2026-08-25): `main` was integrated and
released at exact commit `5776021970cbbe59813ba57579abe00d9fc61a46`; Gitea CI
run **783** concluded success. Windev `live` mode at that SHA built the x86/net48
Worker with **0 warnings / 0 errors**, passed Worker.Tests **529 passed / 11
opt-in skips / 0 failed**, built the full Windows solution with **0 warnings / 0
errors**, passed the Windows gateway suite **1193/1193**, and passed the live
MXAccess smoke **8/8**. One preceding full-suite attempt hit a transient fake-worker
disconnect in the reconnect/replay test; the test then passed **50/50** isolated
macOS runs and **20/20** isolated windev runs before the clean full-gate pass.
Immediately before publication, all five package coordinates returned HTTP 404
and `clients/go/v0.2.1` was absent locally/remotely. The published coordinates are
NuGet `ZB.MOM.WW.MxGateway.Client` **0.2.1** and
`ZB.MOM.WW.MxGateway.Contracts` **0.2.1**, PyPI
`zb-mom-ww-mxaccess-gateway-client` **0.2.1**, Cargo
`zb-mom-ww-mxgateway-client` **0.2.1**, and Maven
`com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client` **0.2.2**. Each exact Gitea
package API coordinate returned HTTP 200 after publication. Annotated Go tag
`clients/go/v0.2.1` resolves locally, remotely, and through `go list -m` to
`5776021970cbbe59813ba57579abe00d9fc61a46`.
+6 -1
View File
@@ -166,6 +166,11 @@ when the gateway's alarm monitor restarts and drops a truncated verdict. Track
it if you need set completeness on a live feed without polling
`QueryActiveAlarmsAsync`.
The CLI's text rows always spell out `degraded=false` and `truncated=false` so
healthy and clearing frames remain visible. Its protobuf-JSON modes retain the
standard proto3 omission rule, so those same false-valued payloads appear as
`{"providerStatus": {}}` and `{"snapshotStatus": {}}`; absent means false.
`MxGatewaySession.CloseAsync` is explicit and idempotent. Repeated calls return
the first `CloseSessionReply` instead of sending another close request.
@@ -285,7 +290,7 @@ dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- advise --s
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- write --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- write2 --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --timestamp 2026-01-01T00:00:00Z --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-events --session-id <id> --max-events 1 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-alarms --filter-prefix Area001 --max-events 1 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- stream-alarms --filter-prefix Area001 --max-events 4 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --comment "ack from cli" --operator operator1 --json
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- smoke --endpoint http://localhost:5000 --api-key-env MXGATEWAY_API_KEY --item Area001.Pump001.Speed --json
```
@@ -347,13 +347,13 @@ public static class MxGatewayClientCli
/// <summary>
/// Canonical CLI credential environment variable, shared by every official
/// client CLI (CLI-45) so one exported variable drives the same operator
/// client CLI so one exported variable drives the same operator
/// workflow in all five languages.
/// </summary>
private const string DefaultVerifyPasswordEnvironmentName = "MXGATEWAY_VERIFY_PASSWORD";
/// <summary>
/// Pre-CLI-45 environment variable, still honoured as a deprecated fallback
/// Legacy environment variable, still honoured as a deprecated fallback
/// for one release so existing scripts keep working.
/// </summary>
private const string LegacyVerifyPasswordEnvironmentName = "MXGATEWAY_VERIFY_USER_PASSWORD";
@@ -379,14 +379,14 @@ public static class MxGatewayClientCli
}
/// <summary>
/// Resolves the effective MXAccess verify-user credential in the CLI-45
/// Resolves the effective MXAccess verify-user credential in this
/// order: <c>--password</c>, the deprecated <c>--verify-user-password</c>
/// alias, the environment variable named by <c>--password-env</c> (default
/// <c>MXGATEWAY_VERIFY_PASSWORD</c>), then the deprecated
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c>. An empty value from any source is
/// treated as absent. The credential is never echoed; this resolver exists so
/// the error-redaction catch block can strip it from any surfaced error
/// (CLI-04), mirroring <see cref="TryResolveApiKey" />.
/// the error-redaction catch block can strip it from any surfaced error,
/// mirroring <see cref="TryResolveApiKey" />.
/// </summary>
private static string? TryResolveVerifyUserPassword(CliArguments arguments)
{
@@ -416,7 +416,7 @@ public static class MxGatewayClientCli
/// Resolves the verify-user credential for <c>authenticate-user</c>, throwing
/// a redaction-safe error when no source yields a non-empty value. Failing
/// fast keeps a misconfigured environment from becoming a real MXAccess
/// authentication attempt with an empty credential (CLI-45); the thrown
/// authentication attempt with an empty credential; the thrown
/// message names only the option/env var, never the value.
/// </summary>
private static string ResolveVerifyUserPassword(CliArguments arguments)
@@ -758,9 +758,9 @@ public static class MxGatewayClientCli
{
// The credential is resolved from --password or its env var (default
// MXGATEWAY_VERIFY_PASSWORD) and is never echoed; a missing or empty value
// fails fast before the invoke rather than reaching the wire (CLI-45).
// fails fast before the invoke rather than reaching the wire.
// On any surfaced error the RunCoreAsync catch block routes
// it through MxGatewayCliSecretRedactor so it cannot reach stderr (CLI-04).
// it through MxGatewayCliSecretRedactor so it cannot reach stderr.
return InvokeAndWriteAsync(
arguments,
client,
@@ -1425,7 +1425,7 @@ public static class MxGatewayClientCli
else if (gatewayEvent.ReplayGap is { } replayGap)
{
// Render the ReplayGap sentinel as the typed cross-CLI row instead of the raw
// sentinel MxEvent (NEXT-02, mirroring the Go/Python/Rust CLIs).
// sentinel MxEvent, mirroring the Go/Python/Rust CLIs.
output.WriteLine(FormatReplayGapRow(replayGap));
}
else
@@ -1561,9 +1561,9 @@ public static class MxGatewayClientCli
AlarmFeedMessage.PayloadOneofCase.Transition =>
$"transition {ProtobufJsonFormatter.Format(feedMessage.Transition)}",
AlarmFeedMessage.PayloadOneofCase.ProviderStatus =>
$"provider-status {ProtobufJsonFormatter.Format(feedMessage.ProviderStatus)}",
$"provider-status mode={feedMessage.ProviderStatus.Mode} degraded={feedMessage.ProviderStatus.Degraded.ToString().ToLowerInvariant()} reason=\"{feedMessage.ProviderStatus.Reason}\"",
AlarmFeedMessage.PayloadOneofCase.SnapshotStatus =>
$"snapshot-status {ProtobufJsonFormatter.Format(feedMessage.SnapshotStatus)}",
$"snapshot-status truncated={feedMessage.SnapshotStatus.Truncated.ToString().ToLowerInvariant()}",
_ => $"unknown-payload {feedMessage.PayloadCase}",
};
}
@@ -1849,7 +1849,7 @@ public static class MxGatewayClientCli
}
/// <summary>
/// Formats the typed ReplayGap row shared by the CLIs (NEXT-02). Hand-built so the
/// Formats the typed ReplayGap row shared by the CLIs. Hand-built so the
/// cursors are JSON numbers like the Go/Python/Rust rows, not the protobuf JSON
/// formatter's quoted uint64 strings.
/// </summary>
@@ -134,6 +134,7 @@ public sealed class MxGatewayClientCliTests
}
/// <summary>Verifies that write-secured builds a WriteSecured command with the value and user ids.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_WriteSecured_BuildsWriteSecuredCommand()
{
@@ -178,6 +179,7 @@ public sealed class MxGatewayClientCliTests
/// Verifies that authenticate-user builds an AuthenticateUser command sourcing the
/// credential from the flag, and that the credential never appears in stdout/stderr.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_AuthenticateUser_BuildsCommandAndDoesNotEchoCredential()
{
@@ -218,9 +220,10 @@ public sealed class MxGatewayClientCliTests
}
/// <summary>
/// CLI-04: a surfaced error for authenticate-user must have the credential
/// Verifies that a surfaced error for authenticate-user must have the credential
/// redacted (never echoed to stderr), mirroring the API-key redaction seam.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_AuthenticateUser_ErrorOutput_RedactsCredential()
{
@@ -248,7 +251,7 @@ public sealed class MxGatewayClientCliTests
}
/// <summary>
/// CLI-45: <c>--password</c> is the primary credential flag, matching the other
/// <c>--password</c> is the primary credential flag, matching the other
/// four CLIs. The credential reaches the wire but never stdout/stderr.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
@@ -290,10 +293,12 @@ public sealed class MxGatewayClientCliTests
}
/// <summary>
/// CLI-45: the credential is read from the environment variable named by
/// the credential is read from the environment variable named by
/// <c>--password-env</c>, whose default is the canonical
/// <c>MXGATEWAY_VERIFY_PASSWORD</c> shared by all five CLIs.
/// </summary>
/// <param name="passwordEnvArgument">The value passed to <c>--password-env</c>, or null to use the default.</param>
/// <param name="environmentName">The environment variable that holds the credential.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[InlineData(null, "MXGATEWAY_VERIFY_PASSWORD")]
@@ -340,7 +345,7 @@ public sealed class MxGatewayClientCliTests
}
/// <summary>
/// CLI-45: the pre-rename names stay usable for one release — the deprecated
/// the pre-rename names stay usable for one release — the deprecated
/// <c>--verify-user-password</c> flag and the deprecated
/// <c>MXGATEWAY_VERIFY_USER_PASSWORD</c> environment variable both still resolve.
/// </summary>
@@ -413,7 +418,7 @@ public sealed class MxGatewayClientCliTests
}
/// <summary>
/// CLI-45: a missing or empty credential fails fast before the invoke — the CLI
/// a missing or empty credential fails fast before the invoke — the CLI
/// never sends a fabricated empty password to the wire. The error names the flag
/// and the environment variable, never a value.
/// </summary>
@@ -463,6 +468,9 @@ public sealed class MxGatewayClientCliTests
private readonly string _name;
private readonly string? _original;
/// <summary>Sets the environment variable and remembers its original value.</summary>
/// <param name="name">The environment variable name.</param>
/// <param name="value">The temporary value, or null to clear it.</param>
public EnvironmentVariableScope(string name, string? value)
{
_name = name;
@@ -470,6 +478,7 @@ public sealed class MxGatewayClientCliTests
Environment.SetEnvironmentVariable(name, value);
}
/// <summary>Restores the original environment variable value.</summary>
public void Dispose()
{
Environment.SetEnvironmentVariable(_name, _original);
@@ -588,7 +597,7 @@ public sealed class MxGatewayClientCliTests
/// <summary>
/// Verifies stream-events renders the ReplayGap sentinel as the typed cross-CLI row —
/// numeric cursors under a replayGap key — instead of the raw sentinel MxEvent (NEXT-02).
/// numeric cursors under a replayGap key — instead of the raw sentinel MxEvent.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -705,6 +714,50 @@ public sealed class MxGatewayClientCliTests
Assert.Equal(string.Empty, error.ToString());
}
/// <summary>Verifies that false alarm-feed status flags remain explicit in text while protobuf JSON omits defaults.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task RunAsync_StreamAlarms_StatusFramesDistinguishTextFromProtobufJson()
{
FakeCliClient fakeClient = new();
fakeClient.AlarmFeedMessages.Add(new AlarmFeedMessage
{
ProviderStatus = new AlarmProviderStatus { Degraded = false },
});
fakeClient.AlarmFeedMessages.Add(new AlarmFeedMessage
{
SnapshotStatus = new AlarmSnapshotStatus { Truncated = false },
});
using var textOutput = new StringWriter();
using var textError = new StringWriter();
int textExit = await MxGatewayClientCli.RunAsync(
["stream-alarms", "--endpoint", "http://localhost:5000", "--api-key", "test-api-key", "--max-events", "2"],
textOutput,
textError,
_ => fakeClient);
Assert.Equal(0, textExit);
Assert.Equal(
["provider-status mode=Unspecified degraded=false reason=\"\"", "snapshot-status truncated=false"],
textOutput.ToString().Split(Environment.NewLine, StringSplitOptions.RemoveEmptyEntries));
Assert.Equal(string.Empty, textError.ToString());
using var jsonOutput = new StringWriter();
using var jsonError = new StringWriter();
int jsonExit = await MxGatewayClientCli.RunAsync(
["stream-alarms", "--endpoint", "http://localhost:5000", "--api-key", "test-api-key", "--max-events", "2", "--jsonl"],
jsonOutput,
jsonError,
_ => fakeClient);
Assert.Equal(0, jsonExit);
Assert.Equal(
["{ \"providerStatus\": { } }", "{ \"snapshotStatus\": { } }"],
jsonOutput.ToString().Split(Environment.NewLine, StringSplitOptions.RemoveEmptyEntries));
Assert.Equal(string.Empty, jsonError.ToString());
}
/// <summary>Verifies that acknowledge-alarm builds a request and prints the JSON reply.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -228,6 +228,7 @@ public sealed class MxGatewayClientSessionTests
/// <see cref="MxEventStreamItem"/> with the gap populated, while normal
/// events pass through unchanged with IsReplayGap false.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StreamEventItemsAsync_SurfacesReplayGapSentinel()
{
@@ -430,6 +431,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that unregister builds an unregister command with the server handle.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task UnregisterAsync_BuildsUnregisterCommand()
{
@@ -451,6 +453,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that advise-supervisory builds the supervisory advise command.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AdviseSupervisoryAsync_BuildsAdviseSupervisoryCommand()
{
@@ -473,6 +476,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that add-buffered-item returns the item handle from the typed reply.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AddBufferedItemAsync_BuildsCommandAndReturnsItemHandle()
{
@@ -498,6 +502,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that set-buffered-update-interval builds the command with the interval.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task SetBufferedUpdateIntervalAsync_BuildsCommandWithInterval()
{
@@ -520,6 +525,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that suspend builds the command and returns the reply status.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task SuspendAsync_BuildsCommandAndReturnsStatus()
{
@@ -545,6 +551,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that activate builds the command and returns the reply status.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task ActivateAsync_BuildsCommandAndReturnsStatus()
{
@@ -568,6 +575,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that write-secured builds a WriteSecured command with value and user ids.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task WriteSecuredAsync_BuildsWriteSecuredCommand()
{
@@ -598,6 +606,7 @@ public sealed class MxGatewayClientSessionTests
/// natively. The client must surface that scripted failure (as an
/// <see cref="MxAccessException"/>) rather than pre-validating it away.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task WriteSecuredAsync_SurfacesNativeFailureWhenNotAuthenticated()
{
@@ -621,6 +630,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that write-secured2 builds a WriteSecured2 command with value and timestamp.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task WriteSecured2Async_BuildsWriteSecured2Command()
{
@@ -647,6 +657,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that authenticate-user builds the command and returns the resolved user id.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AuthenticateUserAsync_BuildsCommandAndReturnsUserId()
{
@@ -675,6 +686,7 @@ public sealed class MxGatewayClientSessionTests
/// must never contain the credential — the error path is built only from
/// reply-derived diagnostics, not the request payload.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AuthenticateUserAsync_FailureDoesNotLeakCredentialInErrorMessage()
{
@@ -698,6 +710,7 @@ public sealed class MxGatewayClientSessionTests
}
/// <summary>Verifies that archestra-user-to-id builds the command and returns the resolved user id.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task ArchestraUserToIdAsync_BuildsCommandAndReturnsUserId()
{
@@ -8,6 +8,7 @@ namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// </summary>
public sealed class MxGatewaySecretRedactionTests
{
/// <summary>Test: Redact replaces every occurrence of secret.</summary>
[Fact]
public void Redact_ReplacesEveryOccurrenceOfSecret()
{
@@ -19,6 +20,7 @@ public sealed class MxGatewaySecretRedactionTests
Assert.Equal("pw=<redacted> retry pw=<redacted> again <redacted>", result);
}
/// <summary>Test: Redact scrubs both secrets when one is substring of the other.</summary>
[Fact]
public void Redact_ScrubsBothSecretsWhenOneIsSubstringOfTheOther()
{
@@ -33,6 +35,7 @@ public sealed class MxGatewaySecretRedactionTests
Assert.DoesNotContain("secret", result, StringComparison.Ordinal);
}
/// <summary>Test: Redact with null secrets array returns message unchanged.</summary>
[Fact]
public void Redact_WithNullSecretsArray_ReturnsMessageUnchanged()
{
@@ -43,6 +46,7 @@ public sealed class MxGatewaySecretRedactionTests
Assert.Equal(message, result);
}
/// <summary>Test: Redact with empty secrets array returns message unchanged.</summary>
[Fact]
public void Redact_WithEmptySecretsArray_ReturnsMessageUnchanged()
{
@@ -53,6 +57,7 @@ public sealed class MxGatewaySecretRedactionTests
Assert.Equal(message, result);
}
/// <summary>Test: Redact ignores whitespace only secret.</summary>
[Fact]
public void Redact_IgnoresWhitespaceOnlySecret()
{
@@ -64,6 +69,7 @@ public sealed class MxGatewaySecretRedactionTests
Assert.Equal(message, result);
}
/// <summary>Test: Redacted preserves concrete subtype and does not chain secret bearing original.</summary>
[Fact]
public void Redacted_PreservesConcreteSubtypeAndDoesNotChainSecretBearingOriginal()
{
@@ -4,15 +4,16 @@ using ZB.MOM.WW.MxGateway.Contracts.Proto;
namespace ZB.MOM.WW.MxGateway.Client.Tests;
/// <summary>
/// Tests for the credential-scrub (CLI-40) and malformed-reply (CLI-41) contracts on the
/// Tests for the credential-scrub and malformed-reply contracts on the
/// credential and id-returning session helpers, driven from shared behavior fixtures.
/// </summary>
public sealed class MxGatewaySessionReplyContractTests
{
/// <summary>
/// CLI-40: when MXAccess echoes the submitted credential back in its failure diagnostic,
/// Verifies that when MXAccess echoes the submitted credential back in its failure diagnostic,
/// the surfaced exception message must scrub it to the library redaction marker.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AuthenticateUserAsync_RedactsEchoedCredentialInFailureMessage()
{
@@ -33,13 +34,14 @@ public sealed class MxGatewaySessionReplyContractTests
}
/// <summary>
/// CLI-40: the redacted exception must not leak the echoed credential through any structured
/// Verifies that the redacted exception must not leak the echoed credential through any structured
/// accessor either — <see cref="MxAccessException.Reply"/> (protocol message, diagnostic
/// message, and each MXSTATUS_PROXY diagnostic text) and <see cref="MxGatewayException.Statuses"/>
/// all carry the server-echoed credential verbatim before the fix. Both the OK+negative-HRESULT
/// and the MXACCESS_FAILURE reply route to <see cref="MxAccessException"/>, so both must scrub.
/// </summary>
/// <param name="fixture">The echoed-credential reply fixture to drive.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[InlineData("authenticate-user.echoed-credential.reply.json")]
[InlineData("authenticate-user.echoed-credential-mxaccess-failure.reply.json")]
@@ -71,9 +73,10 @@ public sealed class MxGatewaySessionReplyContractTests
}
/// <summary>
/// CLI-41: an OK reply that carries neither the typed AuthenticateUser payload nor an
/// Verifies that an OK reply that carries neither the typed AuthenticateUser payload nor an
/// int32 return_value is a malformed reply, surfaced as a typed exception rather than an NRE.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AuthenticateUserAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
@@ -87,9 +90,10 @@ public sealed class MxGatewaySessionReplyContractTests
}
/// <summary>
/// CLI-41: an OK reply that omits the typed payload but carries an int32 return_value
/// Verifies that an OK reply that omits the typed payload but carries an int32 return_value
/// resolves to that return value.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AuthenticateUserAsync_ReturnValueOnly_ResolvesReturnValue()
{
@@ -104,9 +108,10 @@ public sealed class MxGatewaySessionReplyContractTests
}
/// <summary>
/// CLI-41: the AddBufferedItem fallback shares the malformed-reply contract — an OK reply
/// Verifies that the AddBufferedItem fallback shares the malformed-reply contract — an OK reply
/// with neither a typed item handle nor an int32 return_value throws the typed exception.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task AddBufferedItemAsync_MissingPayloadAndReturnValue_ThrowsMalformedReply()
{
@@ -32,6 +32,9 @@ public sealed class MxStatusProxyExtensionsTests
}
/// <summary>Verifies that the raw success member never overrides the authoritative category.</summary>
/// <param name="category">The status category under test.</param>
/// <param name="success">The raw success member, which must not influence the result.</param>
/// <param name="expected">The expected <c>IsSuccess</c> result.</param>
[Theory]
[InlineData(MxStatusCategory.Ok, 0, true)]
[InlineData(MxStatusCategory.Ok, 1, true)]
@@ -96,7 +96,8 @@ internal sealed class WireFakeGatewayServer : IAsyncDisposable
DefaultCallTimeout = TimeSpan.FromSeconds(30),
});
/// <inheritdoc />
/// <summary>Stops and disposes the underlying web application.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
public async ValueTask DisposeAsync()
{
await _app.StopAsync().ConfigureAwait(false);
@@ -854,6 +854,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// </summary>
/// <param name="serverHandle">The ServerHandle from register.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
public async Task UnregisterAsync(
int serverHandle,
CancellationToken cancellationToken = default)
@@ -888,6 +889,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// <param name="serverHandle">The ServerHandle from register.</param>
/// <param name="itemHandle">The ItemHandle from add-item.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
public async Task AdviseSupervisoryAsync(
int serverHandle,
int itemHandle,
@@ -985,6 +987,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// <param name="serverHandle">The ServerHandle from register.</param>
/// <param name="updateIntervalMilliseconds">The buffered update interval, in milliseconds.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
public async Task SetBufferedUpdateIntervalAsync(
int serverHandle,
int updateIntervalMilliseconds,
@@ -1125,6 +1128,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// <param name="currentUserId">The current operator user id.</param>
/// <param name="verifierUserId">The verifier (secondary approver) user id.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
public async Task WriteSecuredAsync(
int serverHandle,
int itemHandle,
@@ -1204,6 +1208,7 @@ public sealed class MxGatewaySession : IAsyncDisposable
/// <param name="currentUserId">The current operator user id.</param>
/// <param name="verifierUserId">The verifier (secondary approver) user id.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
public async Task WriteSecured2Async(
int serverHandle,
int itemHandle,
+6 -1
View File
@@ -162,6 +162,11 @@ on every verdict change — including the clearing frame sent when the gateway's
alarm monitor restarts and drops a truncated verdict. Track it if you need set
completeness on a live feed without polling `QueryActiveAlarms`.
The CLI's text rows always spell out `degraded=false` and `truncated=false`.
Its protobuf-JSON mode retains the standard proto3 omission rule, so those
false-valued payloads appear as `{"providerStatus":{}}` and
`{"snapshotStatus":{}}`; absent means false.
## Write Semantics And Common Pitfalls
These are MXAccess parity behaviors that surprise new callers. The gateway
@@ -439,7 +444,7 @@ go run ./cmd/mxgw-go write -session-id <id> -server-handle 1 -item-handle 1 -typ
go run ./cmd/mxgw-go write-bulk -session-id <id> -server-handle 1 -item-handles 1,2 -values 10,20 -type int32 -plaintext -json
go run ./cmd/mxgw-go read-bulk -session-id <id> -item-handles 1,2 -plaintext -json
go run ./cmd/mxgw-go stream-events -session-id <id> -plaintext -json
go run ./cmd/mxgw-go stream-alarms -plaintext -json
go run ./cmd/mxgw-go stream-alarms -plaintext -limit 4 -json
go run ./cmd/mxgw-go smoke -item Area001.Tag.Value -plaintext -json
go run ./cmd/mxgw-go galaxy-test-connection -plaintext -json
go run ./cmd/mxgw-go galaxy-last-deploy -plaintext -json
+62
View File
@@ -283,6 +283,68 @@ type pingFakeGateway struct {
pb.UnimplementedMxAccessGatewayServer
}
// alarmStatusFakeGateway emits both status arms with their proto3 boolean
// defaults so the CLI contract covers values that protobuf JSON omits.
type alarmStatusFakeGateway struct {
pb.UnimplementedMxAccessGatewayServer
}
func (g *alarmStatusFakeGateway) StreamAlarms(
_ *pb.StreamAlarmsRequest,
stream grpc.ServerStreamingServer[pb.AlarmFeedMessage],
) error {
if err := stream.Send(&pb.AlarmFeedMessage{
Payload: &pb.AlarmFeedMessage_ProviderStatus{
ProviderStatus: &pb.AlarmProviderStatus{Degraded: false},
},
}); err != nil {
return err
}
return stream.Send(&pb.AlarmFeedMessage{
Payload: &pb.AlarmFeedMessage_SnapshotStatus{
SnapshotStatus: &pb.AlarmSnapshotStatus{Truncated: false},
},
})
}
func startAlarmStatusGateway(t *testing.T) string {
t.Helper()
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
server := grpc.NewServer()
pb.RegisterMxAccessGatewayServer(server, &alarmStatusFakeGateway{})
go func() { _ = server.Serve(listener) }()
t.Cleanup(func() {
server.Stop()
_ = listener.Close()
})
return listener.Addr().String()
}
func TestRunStreamAlarmsPrintsStatusArmsInTextAndProtobufJSON(t *testing.T) {
endpoint := startAlarmStatusGateway(t)
baseArgs := []string{"stream-alarms", "-endpoint", endpoint, "-plaintext", "-api-key", "test", "-limit", "2"}
var textOut, textErr bytes.Buffer
if err := runWithIO(t.Context(), baseArgs, &textOut, &textErr); err != nil {
t.Fatalf("text stream-alarms error = %v; stderr = %s", err, textErr.String())
}
if got, want := strings.TrimSpace(textOut.String()), "provider-status mode=ALARM_PROVIDER_MODE_UNSPECIFIED degraded=false reason=\"\"\nsnapshot-status truncated=false"; got != want {
t.Fatalf("text stream-alarms output = %q, want %q", got, want)
}
var jsonOut, jsonErr bytes.Buffer
jsonArgs := append(append([]string{}, baseArgs...), "-json")
if err := runWithIO(t.Context(), jsonArgs, &jsonOut, &jsonErr); err != nil {
t.Fatalf("JSON stream-alarms error = %v; stderr = %s", err, jsonErr.String())
}
if got, want := strings.TrimSpace(jsonOut.String()), "{\"providerStatus\":{}}\n{\"snapshotStatus\":{}}"; got != want {
t.Fatalf("JSON stream-alarms output = %q, want protobuf JSON %q", got, want)
}
}
func (g *pingFakeGateway) Invoke(_ context.Context, req *pb.MxCommandRequest) (*pb.MxCommandReply, error) {
echo := "pong:" + req.GetCommand().GetPing().GetMessage()
return &pb.MxCommandReply{
File diff suppressed because it is too large Load Diff
+7 -2
View File
@@ -127,9 +127,14 @@ type (
QueryActiveAlarmsRequest = pb.QueryActiveAlarmsRequest
// StreamAlarmsRequest is the gateway StreamAlarms request message.
StreamAlarmsRequest = pb.StreamAlarmsRequest
// AlarmFeedMessage is one message on the StreamAlarms feed — an
// active-alarm snapshot row, a snapshot-complete sentinel, or a transition.
// AlarmFeedMessage is one message on the StreamAlarms feed — an active-alarm
// snapshot row, snapshot-complete sentinel, transition, provider status, or
// snapshot-completeness status.
AlarmFeedMessage = pb.AlarmFeedMessage
// AlarmProviderStatus reports the alarm provider mode and degradation state.
AlarmProviderStatus = pb.AlarmProviderStatus
// AlarmSnapshotStatus reports whether the cached alarm snapshot is truncated.
AlarmSnapshotStatus = pb.AlarmSnapshotStatus
// ActiveAlarmSnapshot is one row in a ConditionRefresh stream.
ActiveAlarmSnapshot = pb.ActiveAlarmSnapshot
// OnAlarmTransitionEvent is the body carried by alarm-transition MxEvents.
+17
View File
@@ -0,0 +1,17 @@
package mxgateway_test
import (
"testing"
"gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/mxgateway"
)
// TestAlarmFeedStatusAliasesArePublic is a compile-time consumer contract for
// the two status payloads carried by AlarmFeedMessage.
func TestAlarmFeedStatusAliasesArePublic(t *testing.T) {
t.Helper()
var provider mxgateway.AlarmProviderStatus
var snapshot mxgateway.AlarmSnapshotStatus
_ = provider
_ = snapshot
}
+6 -1
View File
@@ -135,6 +135,11 @@ the clearing frame sent when the gateway's alarm monitor restarts and drops a
truncated verdict. Track it if you need set completeness on a live feed without
polling `queryActiveAlarms`.
The CLI's text rows always spell out `degraded=false` and `truncated=false`.
Its protobuf-JSON mode retains the standard proto3 omission rule, so those
false-valued payloads appear as `{"providerStatus":{}}` and
`{"snapshotStatus":{}}`; absent means false.
## Write Semantics And Common Pitfalls
These are MXAccess parity behaviors that surprise new callers. The gateway
@@ -411,7 +416,7 @@ gradle :zb-mom-ww-mxgateway-cli:run --args="advise-supervisory --endpoint localh
gradle :zb-mom-ww-mxgateway-cli:run --args="authenticate-user --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --server-handle 1 --verify-user operator --password-env MXGATEWAY_VERIFY_PASSWORD --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="write-secured --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --server-handle 1 --item-handle 1 --type int32 --value 123 --current-user-id 100 --verifier-user-id 100 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="stream-events --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --session-id <id> --limit 1 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="stream-alarms --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --filter-prefix Galaxy --limit 1 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="stream-alarms --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --filter-prefix Galaxy --limit 4 --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="acknowledge-alarm --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --reference \"\\Galaxy\Area001.Pump001.PumpFault\" --json"
gradle :zb-mom-ww-mxgateway-cli:run --args="smoke --endpoint localhost:5000 --api-key-env MXGATEWAY_API_KEY --plaintext --item TestObject.TestInt --json"
```
File diff suppressed because it is too large Load Diff
@@ -27,6 +27,8 @@ import mxaccess_gateway.v1.MxaccessGateway.ActiveAlarmSnapshot;
import mxaccess_gateway.v1.MxaccessGateway.AddItemReply;
import mxaccess_gateway.v1.MxaccessGateway.AlarmConditionState;
import mxaccess_gateway.v1.MxaccessGateway.AlarmFeedMessage;
import mxaccess_gateway.v1.MxaccessGateway.AlarmProviderStatus;
import mxaccess_gateway.v1.MxaccessGateway.AlarmSnapshotStatus;
import mxaccess_gateway.v1.MxaccessGateway.AlarmTransitionKind;
import mxaccess_gateway.v1.MxaccessGateway.BulkReadResult;
import mxaccess_gateway.v1.MxaccessGateway.BulkWriteResult;
@@ -685,6 +687,21 @@ final class MxGatewayCliTests {
assertTrue(run.output().contains("\"snapshotComplete\""), run.output());
}
@Test
void streamAlarmsCommandPrintsStatusArmsInTextAndProtobufJson() {
FakeClientFactory factory = new FakeClientFactory();
CliRun text = execute(factory, "stream-alarms");
List<String> textRows = text.output().lines().filter(line -> !line.isBlank()).toList();
assertEquals("provider-status mode=ALARM_PROVIDER_MODE_UNSPECIFIED degraded=false reason=", textRows.get(3));
assertEquals("snapshot-status truncated=false", textRows.get(4));
CliRun json = execute(factory, "stream-alarms", "--json");
List<String> jsonRows = json.output().lines().filter(line -> !line.isBlank()).toList();
assertEquals("{\"providerStatus\":{}}", jsonRows.get(3));
assertEquals("{\"snapshotStatus\":{}}", jsonRows.get(4));
}
@Test
void acknowledgeAlarmCommandForwardsOptionsAndPrintsReply() {
FakeClientFactory factory = new FakeClientFactory();
@@ -1409,6 +1426,12 @@ final class MxGatewayCliTests {
.setTransitionKind(AlarmTransitionKind.ALARM_TRANSITION_KIND_ACKNOWLEDGE)
.setSeverity(700))
.build());
observer.onNext(AlarmFeedMessage.newBuilder()
.setProviderStatus(AlarmProviderStatus.newBuilder().setDegraded(false))
.build());
observer.onNext(AlarmFeedMessage.newBuilder()
.setSnapshotStatus(AlarmSnapshotStatus.newBuilder().setTruncated(false))
.build());
observer.onCompleted();
return new MxGatewayAlarmFeedSubscription();
}
+9 -1
View File
@@ -132,6 +132,14 @@ sent when the gateway's alarm monitor restarts and drops a truncated verdict.
Track it if you need set completeness on a live feed without polling
`query_active_alarms`.
`mxgw-py stream-alarms` writes one JSON row as each feed message arrives. The
default `--max-messages 0` is unbounded; use a positive value for a finite
sample, remembering that the provider-status and snapshot-status preamble both
count toward it. `--timeout` ends the command after an idle wait and the stream
is closed in every exit path. Protobuf JSON omits false-valued scalar defaults,
so healthy/clearing frames appear as `{"providerStatus": {}}` and
`{"snapshotStatus": {}}`; absent means false.
Canceling a Python task cancels the client-side gRPC call or stream wait. It
does not abort an in-flight MXAccess COM call inside the worker process.
@@ -448,7 +456,7 @@ mxgw-py register --session-id <id> --client-name python-client --json
mxgw-py add-item --session-id <id> --server-handle 1 --item Object.Attribute --json
mxgw-py advise --session-id <id> --server-handle 1 --item-handle 2 --json
mxgw-py stream-events --session-id <id> --max-events 1 --json
mxgw-py stream-alarms --max-messages 1 --json
mxgw-py stream-alarms --max-messages 4 --json
mxgw-py acknowledge-alarm --reference "\\Galaxy\Area001.Pump001.PumpFault" --json
mxgw-py write --session-id <id> --server-handle 1 --item-handle 2 --type int32 --value 123 --json
```
File diff suppressed because one or more lines are too long
@@ -511,16 +511,15 @@ def stream_events(**kwargs: Any) -> None:
@main.command("stream-alarms")
@gateway_options
@click.option("--filter-prefix", default="", help="Alarm-reference prefix filter.")
@click.option("--max-messages", default=1, type=int, show_default=True)
@click.option("--max-messages", default=0, type=int, show_default=True, help="Maximum rows to emit; 0 means unbounded.")
@click.option("--timeout", default=5.0, type=float, show_default=True)
@click.option("--correlation-id", default="", help="Client correlation id.")
@click.option("--json", "output_json", is_flag=True, help="Emit JSON output.")
def stream_alarms(**kwargs: Any) -> None:
"""Stream a bounded number of messages from the gateway's central alarm feed."""
"""Stream messages from the gateway's central alarm feed as they arrive."""
_run(
_run_streaming(
_stream_alarms(**kwargs),
output_json=kwargs["output_json"],
secrets=_secrets(kwargs),
)
@@ -1119,19 +1118,39 @@ async def _stream_events(**kwargs: Any) -> dict[str, Any]:
return {"events": [_event_row(event) for event in events]}
async def _stream_alarms(**kwargs: Any) -> dict[str, Any]:
async with await _connect(kwargs) as client:
messages = await _collect_alarm_messages(
client.stream_alarms(
pb.StreamAlarmsRequest(
client_correlation_id=kwargs["correlation_id"],
alarm_filter_prefix=kwargs["filter_prefix"],
),
),
max_messages=kwargs["max_messages"],
timeout=kwargs["timeout"],
async def _stream_alarms(**kwargs: Any) -> None:
max_messages = int(kwargs["max_messages"])
if max_messages > MAX_AGGREGATE_EVENTS:
raise click.BadParameter(
f"must be less than or equal to {MAX_AGGREGATE_EVENTS}",
param_hint="--max-messages",
)
return {"messages": [_message_dict(message) for message in messages]}
async with await _connect(kwargs) as client:
messages = client.stream_alarms(
pb.StreamAlarmsRequest(
client_correlation_id=kwargs["correlation_id"],
alarm_filter_prefix=kwargs["filter_prefix"],
),
)
iterator = messages.__aiter__()
emitted = 0
try:
while max_messages == 0 or emitted < max_messages:
try:
message = await asyncio.wait_for(
iterator.__anext__(),
timeout=kwargs["timeout"],
)
except (StopAsyncIteration, asyncio.TimeoutError):
break
_emit(_message_dict(message), output_json=kwargs["output_json"])
sys.stdout.flush()
emitted += 1
finally:
close = getattr(iterator, "aclose", None)
if close is not None:
await close()
async def _acknowledge_alarm(**kwargs: Any) -> dict[str, Any]:
@@ -1489,6 +1508,17 @@ def _run(
_emit(payload, output_json=output_json)
def _run_streaming(
awaitable: Awaitable[None],
*,
secrets: list[str | None],
) -> None:
try:
asyncio.run(awaitable)
except MxGatewayError as error:
raise click.ClickException(redact_secret(str(error), secrets)) from error
def _emit(
payload: dict[str, Any],
*,
@@ -1536,34 +1566,6 @@ async def _collect_events(
return collected
async def _collect_alarm_messages(
messages: Any,
*,
max_messages: int,
timeout: float,
) -> list[pb.AlarmFeedMessage]:
if max_messages > MAX_AGGREGATE_EVENTS:
raise click.BadParameter(
f"must be less than or equal to {MAX_AGGREGATE_EVENTS}",
param_hint="--max-messages",
)
collected: list[pb.AlarmFeedMessage] = []
iterator = messages.__aiter__()
try:
while len(collected) < max_messages:
collected.append(await asyncio.wait_for(iterator.__anext__(), timeout=timeout))
except StopAsyncIteration:
pass
except asyncio.TimeoutError:
pass
finally:
close = getattr(iterator, "aclose", None)
if close is not None:
await close()
return collected
async def _collect_deploy_events(
events: Any,
*,
@@ -7,6 +7,7 @@ fixed source.
from __future__ import annotations
import asyncio
import json
import re
import time as _time_module_ref
@@ -296,6 +297,22 @@ class _AlarmFakeStream:
self.cancelled = True
class _IncrementalAlarmFakeStream(_AlarmFakeStream):
def __init__(self, messages: list[pb.AlarmFeedMessage], emitted: list[str]) -> None:
super().__init__(messages)
self._emitted = emitted
self._yielded = 0
async def __anext__(self) -> pb.AlarmFeedMessage:
if self._yielded > 0:
assert len(self._emitted) >= self._yielded, "previous alarm row was buffered"
if not self._messages:
await asyncio.Event().wait()
raise AssertionError("unreachable")
self._yielded += 1
return self._messages.pop(0)
class _BulkFakeUnary:
def __init__(self, replies: list[Any]) -> None:
self.replies = replies
@@ -667,8 +684,15 @@ def test_cli_write_bulk_happy_path(monkeypatch) -> None:
assert cmd.kind == pb.MX_COMMAND_KIND_WRITE_BULK
def test_cli_stream_alarms_happy_path(monkeypatch) -> None:
transitions = [
def test_cli_stream_alarms_default_streams_status_cached_and_live_rows_incrementally(
monkeypatch,
) -> None:
messages = [
pb.AlarmFeedMessage(provider_status=pb.AlarmProviderStatus(degraded=False)),
pb.AlarmFeedMessage(snapshot_status=pb.AlarmSnapshotStatus(truncated=False)),
pb.AlarmFeedMessage(
active_alarm=pb.ActiveAlarmSnapshot(alarm_full_reference="Tank01.Level.HiHi"),
),
pb.AlarmFeedMessage(
transition=pb.OnAlarmTransitionEvent(
alarm_full_reference="Tank01.Level.HiHi",
@@ -676,7 +700,15 @@ def test_cli_stream_alarms_happy_path(monkeypatch) -> None:
),
),
]
stream = _AlarmFakeStream(transitions)
emitted: list[str] = []
real_echo = cli_commands.click.echo
def recording_echo(message=None, **kwargs) -> None:
emitted.append(str(message))
real_echo(message, **kwargs)
monkeypatch.setattr(cli_commands.click, "echo", recording_echo)
stream = _IncrementalAlarmFakeStream(messages, emitted)
stub = _BulkFakeStub()
stub.set_alarm_stream(stream)
_install_fake_connect(monkeypatch, stub)
@@ -689,10 +721,8 @@ def test_cli_stream_alarms_happy_path(monkeypatch) -> None:
"--endpoint",
"localhost:5000",
"--plaintext",
"--max-messages",
"1",
"--timeout",
"5.0",
"0.01",
"--filter-prefix",
"Tank01.",
"--json",
@@ -700,8 +730,44 @@ def test_cli_stream_alarms_happy_path(monkeypatch) -> None:
)
assert result.exit_code == 0, result.output
payload = json.loads(result.output)
assert payload["messages"][0]["transition"]["alarmFullReference"] == "Tank01.Level.HiHi"
rows = [json.loads(line) for line in result.output.splitlines()]
assert [next(iter(row)) for row in rows] == [
"providerStatus",
"snapshotStatus",
"activeAlarm",
"transition",
]
assert stream.cancelled
def test_cli_stream_alarms_positive_max_messages_remains_finite(monkeypatch) -> None:
messages = [
pb.AlarmFeedMessage(provider_status=pb.AlarmProviderStatus()),
pb.AlarmFeedMessage(snapshot_status=pb.AlarmSnapshotStatus()),
pb.AlarmFeedMessage(active_alarm=pb.ActiveAlarmSnapshot()),
]
stream = _AlarmFakeStream(messages)
stub = _BulkFakeStub()
stub.set_alarm_stream(stream)
_install_fake_connect(monkeypatch, stub)
result = CliRunner().invoke(
main,
[
"stream-alarms",
"--endpoint",
"localhost:5000",
"--plaintext",
"--max-messages",
"2",
"--json",
],
)
assert result.exit_code == 0, result.output
rows = [json.loads(line) for line in result.output.splitlines()]
assert [next(iter(row)) for row in rows] == ["providerStatus", "snapshotStatus"]
assert stream.cancelled
def test_cli_acknowledge_alarm_happy_path(monkeypatch) -> None:
+6 -1
View File
@@ -77,7 +77,7 @@ cargo run -p mxgw-cli -- register --session-id <session-id> --client-name mxgw-r
cargo run -p mxgw-cli -- add-item --session-id <session-id> --server-handle 1 --item TestChildObject.TestInt --json
cargo run -p mxgw-cli -- advise --session-id <session-id> --server-handle 1 --item-handle 1 --json
cargo run -p mxgw-cli -- stream-events --session-id <session-id> --max-events 1 --json
cargo run -p mxgw-cli -- stream-alarms --session-id <session-id> --max-messages 1 --json
cargo run -p mxgw-cli -- stream-alarms --session-id <session-id> --max-messages 4 --json
cargo run -p mxgw-cli -- acknowledge-alarm --session-id <session-id> --alarm-reference "\\Galaxy\Area001.Pump001.PumpFault" --json
cargo run -p mxgw-cli -- write --session-id <session-id> --server-handle 1 --item-handle 1 --value-type int32 --value 123 --json
```
@@ -141,6 +141,11 @@ every verdict change — including the clearing frame sent when the gateway's
alarm monitor restarts and drops a truncated verdict. Track it if you need set
completeness on a live feed without polling `query_active_alarms`.
Unlike the protobuf-JSON formatters used by the other CLIs, the Rust CLI's
hand-built JSON status rows always include `"degraded": false` and
`"truncated": false`. This deliberate representational difference is
documented in the cross-language smoke matrix; both shapes mean the same thing.
The session helpers keep MXAccess handles visible:
```rust
+20 -4
View File
@@ -130,6 +130,7 @@ message MxCommand {
WriteSecuredBulkCommand write_secured_bulk = 41;
WriteSecured2BulkCommand write_secured2_bulk = 42;
ReadBulkCommand read_bulk = 43;
UpdateAlarmWatchListCommand update_alarm_watch_list = 44;
PingCommand ping = 100;
GetSessionStateCommand get_session_state = 101;
GetWorkerInfoCommand get_worker_info = 102;
@@ -174,6 +175,7 @@ enum MxCommandKind {
MX_COMMAND_KIND_WRITE_SECURED_BULK = 32;
MX_COMMAND_KIND_WRITE_SECURED2_BULK = 33;
MX_COMMAND_KIND_READ_BULK = 34;
MX_COMMAND_KIND_UPDATE_ALARM_WATCH_LIST = 35;
MX_COMMAND_KIND_PING = 100;
MX_COMMAND_KIND_GET_SESSION_STATE = 101;
MX_COMMAND_KIND_GET_WORKER_INFO = 102;
@@ -344,8 +346,8 @@ enum AlarmProviderMode {
// Subscription expression follows the canonical
// `\\<machine>\Galaxy!<area>` format (literal "Galaxy" provider). The
// worker spins up a wnwrapConsumer-backed subscription on its STA on
// first call; subsequent calls are an error (use UnsubscribeAlarms then
// SubscribeAlarms to reconfigure).
// first call; subsequent calls are an error. Use UpdateAlarmWatchList to
// replace the standby targets without restarting the primary subscription.
message SubscribeAlarmsCommand {
string subscription_expression = 1;
// UNSPECIFIED = auto-failover/failback. ALARMMGR/SUBTAG force one provider.
@@ -354,6 +356,17 @@ message SubscribeAlarmsCommand {
// alarmmgr mode; in subtag mode it bounds what the consumer can observe.
repeated AlarmSubtagTarget watch_list = 3;
AlarmFailoverConfig failover = 4;
// True when auto mode should retain subtag-fallback capability even if the
// gateway's initial discovery produced an empty watch list. False leaves an
// empty list as an intentional alarm-manager-only subscription.
bool fallback_enabled = 5;
}
// Replace the live alarm subscription's subtag standby targets. The worker
// constructs and attaches the replacement before swapping it into service, so
// a setup failure leaves the existing standby and active provider unchanged.
message UpdateAlarmWatchListCommand {
repeated AlarmSubtagTarget watch_list = 1;
}
// Tear down the worker's alarm consumer. No-op if no subscription is
@@ -537,11 +550,14 @@ message MxCommandReply {
MxValue return_value = 6;
// Correlated per-item outcome rows. For WRITE / WRITE2 / WRITE_SECURED /
// WRITE_SECURED2 replies the worker holds the reply for a bounded window
// (default 1.5 s, MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS) waiting for
// (default 1.5 s, MxGateway:Worker:WriteCompletionWaitMilliseconds) waiting for
// the matching MXAccess OnWriteComplete callback and copies its status rows
// here, so statuses[0] carries the real MXAccess commit outcome (success OR
// failure) while protocol_status/hresult still describe command acceptance
// only. Empty statuses on a write reply means the completion did not arrive
// only. Callback correlation is recency-based per item handle: after the
// bounded wait, a sequential write's callback supersedes a timed-out
// predecessor, and a later stale callback for that predecessor is ignored.
// Empty statuses on a write reply means the completion did not arrive
// within the window — the write is unconfirmed, not failed. Correlation is
// best-effort per (server_handle, item_handle): MXAccess's callback carries
// no transaction id, so concurrent writes to the same item within the
+9 -3
View File
@@ -13,7 +13,10 @@ pub mod mxaccess_gateway {
/// `mxaccess_gateway.v1` package — the v1 wire contract surfaced by the
/// gateway to language clients.
pub mod v1 {
#![allow(clippy::large_enum_variant)]
#![allow(clippy::large_enum_variant, clippy::result_large_err)]
// tonic's generated server traits use async_trait, whose expansion adds
// redundant must_use attributes to Future returns under Rust 1.99.
#![allow(clippy::double_must_use)]
tonic::include_proto!("mxaccess_gateway.v1");
}
@@ -24,7 +27,7 @@ pub mod mxaccess_worker {
/// `mxaccess_worker.v1` package — frame and envelope types used inside
/// the named-pipe transport between gateway and worker.
pub mod v1 {
#![allow(clippy::large_enum_variant)]
#![allow(clippy::large_enum_variant, clippy::result_large_err)]
tonic::include_proto!("mxaccess_worker.v1");
}
@@ -35,7 +38,10 @@ pub mod galaxy_repository {
/// `galaxy_repository.v1` package — types for the Galaxy hierarchy
/// discovery and deploy-event watch RPCs.
pub mod v1 {
#![allow(clippy::large_enum_variant)]
#![allow(clippy::large_enum_variant, clippy::result_large_err)]
// tonic's generated server traits use async_trait, whose expansion adds
// redundant must_use attributes to Future returns under Rust 1.99.
#![allow(clippy::double_must_use)]
tonic::include_proto!("galaxy_repository.v1");
}
+40 -13
View File
@@ -793,6 +793,28 @@ in the codebase for the forward-compat shape, but the gateway-side
`AcknowledgeAlarmByName` when the public RPC supplies a recognizable
`Provider!Group.Tag` reference.
**Opt-in managed GUID ack.** Setting `MXGATEWAY_USE_MANAGED_ALARM_ACK=1`
in the gateway's environment (workers inherit it) routes GUID-keyed acks
through `aaAlarmManagedClient.AlarmClient.AlarmAckByGUID` instead of the
wnwrap method. That managed method is implemented on this build and
carries operator domain and full name natively. The worker reaches it by
reflection (`AaManagedAlarmAckConsumer`) because the members are
non-public, and runs its own Initialize → Register → Subscribe lifecycle
against the alarm subscription expression. The path is deliberately
conservative:
- If the DLL cannot load or the lifecycle returns a non-zero status, the
worker drops back to the wnwrap ack path; alarm reads never fail
because of it.
- Subtag-mode acks (forced subtag, or failover currently on the standby)
stay on the subtag consumer: their GUIDs are synthetic and only it can
resolve them.
- By-name acks stay on wnwrap — `AlarmClient` has no by-name ack.
A zero return still means *accepted*, not *applied*; a secured
(`MxSecurityOperate`) alarm needs an authenticated operator. The skip-gated
`AlarmManagedGuidAckLiveSmokeTests` asserts the ack actually lands.
### 5. STA / threading — resolved
The wnwrap COM is `ThreadingModel=Apartment`. The consumer's
@@ -849,15 +871,20 @@ MXAccess item addresses for the `InAlarm`, `Acked`, `AckMsg`, and `Priority`
subtags (confirmed AVEVA `AlarmExtension` field names, verified against the live
ZB Galaxy `attribute_definition` rows).
**The list is fixed for the life of the subscription.** The worker builds its
subtag consumer from the targets `SubscribeAlarms` carried and keeps them until
the subscription ends; a second `SubscribeAlarms` on a subscribed session is
rejected, and the only way to replace the list — `UnsubscribeAlarms` then
`SubscribeAlarms` — rebuilds `FailoverAlarmConsumer`, restarting it on the
primary and discarding the warm standby state. Re-sending on a failover would
therefore flap the feed rather than repair it. A monitor restart (a worker fault,
or a gateway restart) re-resolves and re-sends, and that is the supported way to
pick up Galaxy objects added since the monitor started.
The gateway refreshes this list on the existing alarm reconciliation cadence
and immediately before publishing a failover-to-subtag status. It compares a
canonical fingerprint of all six target fields and sends
`UpdateAlarmWatchListCommand` only for a material change. The worker constructs,
subscribes, and attaches the replacement standby on its STA before swapping it
in; a setup failure leaves the previous standby and the current primary/subtag
mode unchanged. The gateway advances its acknowledged fingerprint only after an
OK reply, so a rejected update remains degraded and is retried.
`SubscribeAlarmsCommand.fallback_enabled` distinguishes an initially empty but
discovery-enabled fallback from an intentionally alarm-manager-only
subscription. In the former case the worker creates an empty standby that can be
hot-swapped after discovery recovers. `ForceAlarmManager` leaves the field false
and does not create a standby.
Because the list is fixed, the resolution is **graded** rather than merely
logged (GWC-37). `ResolveAsync` returns an `AlarmWatchListResolution` carrying
@@ -871,10 +898,10 @@ monitor folds the verdict into the alarm feed's `AlarmProviderStatus`:
`AlarmProviderReasons.SubtagDiscoveryFailed` and the failure message, so a
Galaxy Repository outage at startup is a standing status rather than one log
line emitted possibly weeks before the failover that needs the list.
- On the failover edge (`ApplyProviderModeChangeAsync` with `toMode == Subtag`)
the monitor re-resolves and compares against what it sent. A difference is
reported as `AlarmProviderReasons.SubtagWatchListStale` with both counts, on
the same status frame as the failover, because it cannot be applied.
- If the worker rejects a material update, the monitor retains the previous
acknowledged fingerprint and reports
`AlarmProviderReasons.SubtagWatchListStale` with the rejection diagnostic.
The next reconciliation or failover edge retries the same resolved targets.
`degraded` on the status is therefore the OR of "the subtag provider is the one
serving" and "the subtag watch-list cannot cover anything". In `Auto` the second
+3 -1
View File
@@ -138,7 +138,9 @@ would otherwise let any peer throttle a key it does not hold — and it joins th
partition only when the presented token is validly shaped
(`mxgw_<keyId>_<secret>`, key id at most 64 characters), with at most 32 key-id
partitions per address before the overflow collapses onto that address's fallback
partition. An over-limit state admits one probe per
partition. That peer-layer collapse does not exempt a validly shaped key id from
the cross-peer aggregate: its failures still feed and consult the key-id aggregate,
whose own cardinality is bounded by the configured LRU cap. An over-limit state admits one probe per
`ApiKeyFailureProbeIntervalSeconds` through to the real verifier and refuses
everything else with `ResourceExhausted` before the store read, so a legitimate
holder presenting the correct secret always reaches the constant-time compare and
+10 -7
View File
@@ -140,13 +140,16 @@ publish while the version constants still read the already-published
0.2.0/0.2.1 numbers above — see `clients/python/README.md` for the
user-facing warning. **0.2.0 is superseded by 0.2.1; do not install it.**
As of this remediation (CLI-46), the in-source version constants for the
.NET, Go, Python, and Rust clients (plus Contracts) read **0.2.1**, and the
Java client reads **0.2.2**. These are not yet published — the numbers
above (0.2.0 four clients + Contracts, 0.2.1 Java) remain the live registry
state until the next `pack-clients.ps1` / `tag-go-module.ps1` run. Once that
publish lands, update this section with the new coordinates and commit(s),
the same way the 2026-08-07 record above was written after the fact.
On 2026-08-25 the superseding release shipped from commit
`5776021970cbbe59813ba57579abe00d9fc61a46`. Published coordinates on
`gitea.dohertylan.com` are `nuget` `ZB.MOM.WW.MxGateway.Client` **0.2.1** and
`ZB.MOM.WW.MxGateway.Contracts` **0.2.1**, `pypi`
`zb-mom-ww-mxaccess-gateway-client` **0.2.1**, `cargo`
`zb-mom-ww-mxgateway-client` **0.2.1**, and `maven`
`com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client` **0.2.2**. The Go module
tag `clients/go/v0.2.1` points to that same commit. Every exact package
coordinate returned HTTP 200 through the Gitea package API after publication,
and `go list -m` resolved `v0.2.1` to the tagged commit.
## .NET
+25
View File
@@ -40,6 +40,15 @@ the corresponding MXAccess `AddItem`, `Advise`, `UnAdvise`, and `RemoveItem`
calls sequentially on the session STA and preserves input order in the result
list.
The shared alarm command model includes `SubscribeAlarms` and
`UpdateAlarmWatchList`. `SubscribeAlarmsCommand.fallback_enabled` makes an empty
initial watch list explicitly fallback-capable, allowing discovery recovery to
populate a standby without restarting the primary alarm-manager subscription.
`UpdateAlarmWatchListCommand.watch_list` is the complete replacement list. The
worker prepares the replacement on its STA before swapping it into the existing
failover consumer and returns a normal `MxCommandReply`; callers must treat only
`protocol_status = OK` as acknowledgement of the new list.
The command model also includes bulk write/read command kinds:
`WriteBulk`, `Write2Bulk`, `WriteSecuredBulk`, `WriteSecured2Bulk`, and
`ReadBulk`. They are unary `Invoke` payloads on the same `MxAccessGateway`
@@ -94,6 +103,15 @@ only and does not share types with `mxaccess_gateway.proto`. See
[Galaxy Repository Browse](./GalaxyRepository.md) for the RPC catalog and
behavior.
The retained proto remains the client-generation source of truth while the
gateway serves the wire-identical descriptor from the
`ZB.MOM.WW.GalaxyRepository` package. `GalaxyContractParityTests` converts both
loaded descriptors to `FileDescriptorProto`, removes source locations, and
normalizes the C# namespace and file name before comparing deterministic bytes.
Its mutation test also requires drift failures to identify the first differing
protobuf message. Any parity failure must be resolved before publishing either
Contracts or clients.
Generated C# output is written to `src/ZB.MOM.WW.MxGateway.Contracts/Generated/`. Do not
hand-edit generated files.
@@ -140,6 +158,13 @@ Run the focused contract tests after changing either `.proto` file:
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter ProtobufContractRoundTripTests
```
When changing `galaxy_repository.proto` or the served Galaxy Repository
package, also run the retained-vs-served parity guard:
```bash
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter GalaxyContractParityTests
```
The full solution build also regenerates the C# contracts before compiling
gateway and test projects:
+17
View File
@@ -66,6 +66,23 @@ declaration-ordered JSON; and the row sits alone on its own line for Go and for
Rust's `--jsonl`, but inside an `events` array for Python and for Rust's
aggregate `--json`.
Alarm-feed status rows deliberately preserve each CLI's established JSON
mapping while making false values explicit in every text formatter:
| CLI | Healthy provider text | Clearing snapshot text | JSON false-value shape |
|-----|-----------------------|------------------------|------------------------|
| `mxgw-dotnet` | `provider-status mode=Unspecified degraded=false reason=""` | `snapshot-status truncated=false` | `{"providerStatus": {}}` / `{"snapshotStatus": {}}` |
| `mxgw-go` | `provider-status mode=ALARM_PROVIDER_MODE_UNSPECIFIED degraded=false reason=""` | `snapshot-status truncated=false` | `{"providerStatus":{}}` / `{"snapshotStatus":{}}` |
| `mxgw-py` | same JSON row as `--json` | same JSON row as `--json` | `{"providerStatus": {}}` / `{"snapshotStatus": {}}`, emitted one row at a time |
| `mxgw-java` | `provider-status mode=ALARM_PROVIDER_MODE_UNSPECIFIED degraded=false reason=` | `snapshot-status truncated=false` | `{"providerStatus":{}}` / `{"snapshotStatus":{}}` |
| `mxgw-rs` | explicit `degraded=false` summary | explicit `truncated=false` summary | always includes `"degraded":false` / `"truncated":false` |
For the protobuf-JSON rows, an absent proto3 boolean means `false`; Rust's
always-explicit booleans are semantically equivalent. Python `stream-alarms`
defaults to an unbounded feed and emits each row immediately; a positive
`--max-messages` value provides a bounded sample and includes both status
preamble rows in the count.
## Integration Gate
Cross-language smoke execution is opt-in. Runners should skip the matrix unless
+3 -3
View File
@@ -90,9 +90,9 @@ detach-grace and replay-retention windows are on by default, the reconnect
surface is a trust boundary — a retained session outlives the stream that opened
it. The session records the opening key id (`GatewaySession.OwnerKeyId`) and
every `StreamEvents` attach/reattach is rejected with `PermissionDenied` unless
the caller's key id matches the owner. Gating on the `event` scope alone was
rejected: it would let any `event`-scoped key that learned a session id attach to
another key's retained session and receive its replayed and live data. The check
the caller's key id matches the owner. Gating on the `events:read` scope alone was
rejected: it would let any `events:read`-scoped key that learned a session id
attach to another key's retained session and receive its replayed and live data. The check
runs before any subscriber is attached, so a foreign key never touches the
replay ring. Admin-scope override is deferred.
+8 -1
View File
@@ -439,13 +439,20 @@ behavior.
| `MxGateway:Alarms:Enabled` | `false` | Gates the gateway's always-on central alarm monitor. When `true`, the gateway opens one gateway-owned worker session dedicated to alarms, caches the active-alarm set, and fans it out to every client through the `StreamAlarms` RPC — no client opens its own session to see alarms. |
| `MxGateway:Alarms:SubscriptionExpression` | _(empty)_ | AVEVA alarm-subscription expression the monitor subscribes on startup, in canonical `\\<machine>\Galaxy!<area>` form. The literal `Galaxy` provider is correct regardless of the Galaxy database name. When empty and `Enabled` is `true`, the gateway falls back to `\\<MachineName>\Galaxy!<DefaultArea>` if `DefaultArea` is set. |
| `MxGateway:Alarms:DefaultArea` | _(empty)_ | Area name used to compose a default subscription when `SubscriptionExpression` is empty. If both are empty while `Enabled` is `true`, the monitor faults with a configuration diagnostic. |
| `MxGateway:Alarms:ReconcileIntervalSeconds` | `30` | How often the monitor reconciles its in-process alarm cache against the worker's authoritative active-alarm snapshot, catching transitions the live poll-and-diff feed missed. Floored at 5 seconds. |
| `MxGateway:Alarms:ReconcileIntervalSeconds` | `30` | How often the monitor re-resolves and, on material change, hot-swaps the subtag fallback watch-list, then reconciles its in-process alarm cache against the worker's authoritative active-alarm snapshot. The acknowledged watch-list fingerprint advances only after an OK worker reply. Floored at 5 seconds. |
| `MxGateway:Alarms:PollIntervalMilliseconds` | `500` | Cadence at which the worker's STA polls the AVEVA alarm consumer (`GetXmlCurrentAlarms2`) for the active-alarm snapshot the live feed diffs. Must be between `100` and `3600000` (one hour): every poll is a COM call plus an XML parse on the same STA that serves reads and writes, so a tighter cadence starves the command path, while a value above an hour stops being a cadence and silently disables alarm polling. The gateway conveys the value to the worker via the `MXGATEWAY_ALARM_POLL_INTERVAL_MS` environment variable; a missing or out-of-range value leaves the worker on the 500 ms default rather than failing the session. |
| `MxGateway:Alarms:MaxAlarmsPerFetch` | `1024` | Cap the worker passes to `GetXmlCurrentAlarms2`'s `maxAlmCnt`. Must be between `64` and `65536` — the worker is a 32-bit process that materializes each reply as one BSTR plus a full `XmlDocument`, so an unbounded cap faults the STA with an out-of-memory rather than merely slowing it. It doubles as the **truncation threshold**: a fetch returning exactly this many records is treated as truncated, because the COM API caps its reply with no "more available" flag. On a truncated poll the worker retains the alarms the capped reply could not mention instead of letting their absence read as a clear, and logs a rate-limited `AlarmSnapshotTruncated` warning to its stderr (identifiers and counts only). **Remediation when you see that warning: raise this value** so the steady-state active-alarm count fits inside one fetch. A galaxy permanently above the cap holds stale entries in the snapshot until a sub-cap poll, and loses clear→re-raise cycles that happen entirely out of window (see `docs/DesignDecisions.md`). Conveyed to the worker via the `MXGATEWAY_ALARM_MAX_ALARMS_PER_FETCH` environment variable; a missing or out-of-range value leaves the worker on the 1024 default. |
The alarm monitor is independent of client sessions: `AcknowledgeAlarm` and
`StreamAlarms` are session-less RPCs served by the monitor.
GUID-keyed acknowledges can opt into AVEVA's managed alarm client by setting
the environment variable `MXGATEWAY_USE_MANAGED_ALARM_ACK=1` on the gateway
process (workers inherit it; under NSSM, add it to the service environment).
It is off by default and has no `MxGateway:*` option because it is still
awaiting live confirmation on the rig. See
[AlarmClientDiscovery](./AlarmClientDiscovery.md) ("Opt-in managed GUID ack").
### Alarm fallback options
The `Fallback` sub-section controls how the alarm feed selects between the
+7 -5
View File
@@ -650,11 +650,13 @@ runtime because the x86 Worker cannot build on Linux:
the tier is down, not the change**: an unreachable windev fails the job (never skips), so
if it goes red without a related code change, check the tier — `ssh windev`, read
`C:\build\mxaccessgw-ci\ci.log` — before assuming a real test failure.
- **`nightly-windev`** (Linux runner, scheduled `0 6 * * *`) — re-runs the full x86 build +
Worker.Tests on windev, then the live-MXAccess smoke (`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`)
and a full `slnx` build (which surfaces Windows-only analyzer diagnostics such as
`xUnit1030`). Gated `if: github.event_name == 'schedule'`, so it never gates a push. On
failure it opens a Gitea issue via the Actions token, since nobody watches the Actions page.
- **`nightly-windev`** (Linux runner, scheduled `0 6 * * *`) — runs, in order, the x86 Worker
build, `Worker.Tests`, a full `slnx` build (which surfaces Windows-only analyzer diagnostics
such as `xUnit1030`), the net10 gateway suite, and the live-MXAccess smoke
(`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`). The checkout and all five tiers share the windev
host lock, and the first nonzero native exit stops the sequence and propagates to the CI job.
Gated `if: github.event_name == 'schedule'`, so it never gates a push. On failure it opens a
Gitea issue via the Actions token, since nobody watches the Actions page.
### Runner capacity is shared and finite
+7 -1
View File
@@ -84,6 +84,12 @@ Carrying the enqueue timestamp into the worker layer is what lets queue-wait tim
An accepted gRPC command payload can still be too large for the worker pipe: the envelope built around it must fit `MxGateway:Worker:MaxMessageBytes`, which is validated at startup to sit at least a fixed envelope-overhead reserve above `MaxGrpcMessageBytes` (see the headroom rule in [Gateway Configuration](./GatewayConfiguration.md)) so this should not occur for a conformant payload, but if it does, `WorkerClient` raises `WorkerClientException(CommandTooLarge)` and `Invoke` reports `ResourceExhausted` for that command — the session is not faulted, so a client can retry with a smaller payload without reopening the session.
Selected control commands have additional public behavior beyond their protobuf shape:
| Command | Public behavior |
|---|---|
| `DrainEvents` | Returns a destructive diagnostic snapshot of the worker's outbound event queue. `max_events = 0` selects the default cap; positive values may not exceed `GatewayContractInfo.MaxDrainEventsPerCommand` (10,000). The worker also applies a serialized-byte budget derived from the negotiated frame limit, so a reply may contain fewer events than the requested count. When either cap leaves queued events, `MxCommandReply.diagnostic_message` reports the truncation and callers repeat `DrainEvents` until an empty reply. Events included in a drain are removed from the worker queue and therefore do not later appear on the normal session `StreamEvents` path; the background stream drain may concurrently consume other events first. |
### `StreamEvents`
`StreamEvents` is a server-streaming RPC. The handler delegates the full pipeline to `IEventStreamService` and just forwards each `MxEvent` onto the response stream. Keeping the channel and producer/consumer machinery out of the handler means cancellation, exception mapping, and metric bookkeeping live in one place.
@@ -100,7 +106,7 @@ An `OK` response means the alarm provider accepted the acknowledgement, not that
### `QueryActiveAlarms`
`QueryActiveAlarms` is a server-streaming, **session-less** RPC that returns a point-in-time snapshot of the currently-active alarm set. Unlike `StreamAlarms`, it does not stay attached for live transitions — it writes the snapshot and completes, which clients use after a reconnect to reseed alarm state or to reconcile alarms that may have been missed during a transport blip. It is served from the always-on `GatewayAlarmMonitor` cache (`IGatewayAlarmService.CurrentAlarms`), so no worker session is opened. The handler does not run through `MxAccessGrpcRequestValidator`; it only null-checks the request inline, then streams one `ActiveAlarmSnapshot` per cached alarm, honouring cancellation between writes. When `alarm_filter_prefix` is non-empty, only alarms whose `alarm_full_reference` starts with that prefix (ordinal `StartsWith`) are emitted; an empty prefix returns the full set. Exceptions are translated by the same `MapException` path as the other handlers. The RPC requires the `event` scope (`GatewayScopes.EventsRead`), the same scope as `StreamEvents` and `StreamAlarms`.
`QueryActiveAlarms` is a server-streaming, **session-less** RPC that returns a point-in-time snapshot of the currently-active alarm set. Unlike `StreamAlarms`, it does not stay attached for live transitions — it writes the snapshot and completes, which clients use after a reconnect to reseed alarm state or to reconcile alarms that may have been missed during a transport blip. It is served from the always-on `GatewayAlarmMonitor` cache (`IGatewayAlarmService.CurrentAlarms`), so no worker session is opened. The handler does not run through `MxAccessGrpcRequestValidator`; it only null-checks the request inline, then streams one `ActiveAlarmSnapshot` per cached alarm, honouring cancellation between writes. When `alarm_filter_prefix` is non-empty, only alarms whose `alarm_full_reference` starts with that prefix (ordinal `StartsWith`) are emitted; an empty prefix returns the full set. Exceptions are translated by the same `MapException` path as the other handlers. The RPC requires the `events:read` scope (`GatewayScopes.EventsRead`), the same scope as `StreamEvents` and `StreamAlarms`.
#### Provider status on the alarm feed
+25 -3
View File
@@ -90,13 +90,22 @@ Expected command-line arguments:
--protocol-version <version>
```
Expected protected environment values:
Expected launch environment values (the shared names are defined by
`WorkerLaunchEnvironment` in Contracts):
```text
MXGATEWAY_WORKER_NONCE=<random nonce>
MXGATEWAY_WORKER_LOG_CONTEXT=<optional context>
MXGATEWAY_WORKER_PIPE_CONNECT_ATTEMPT_TIMEOUT_MS=<milliseconds>
MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS=<milliseconds>
MXGATEWAY_EVENT_QUEUE_CAPACITY=<events>
MXGATEWAY_ALARM_POLL_INTERVAL_MS=<milliseconds>
MXGATEWAY_ALARM_MAX_ALARMS_PER_FETCH=<alarms>
```
The nonce is protected bootstrap material. The remaining values are derived
from the public `MxGateway:Worker:*` and `MxGateway:Alarms:*` configuration
keys documented in [Worker Process Launcher](./WorkerProcessLauncher.md).
Startup sequence:
1. Parse command-line arguments.
@@ -312,7 +321,8 @@ write wait still yields the empty-`statuses` unconfirmed reply, and an expired
per-tag `ReadBulk` wait still reports its own timeout.
The write wait's budget is `MxGateway:Worker:WriteCompletionWaitMilliseconds`
(default 1500). It is a bounded hold on the STA per unary write, so deployments
(default 1500). It applies to `Write`, `Write2`, `WriteSecured`, and
`WriteSecured2` and is a bounded hold on the STA per single-item write, so deployments
whose write workload is effectively fire-and-forget — no consumer reads the
reply's `statuses` — can lower it, or set `0` to skip the wait entirely and
reply on acceptance alone.
@@ -341,6 +351,18 @@ native client correlating by handle has the identical ambiguity). The exposure
is bounded: `statuses` is an enrichment field, and `Hresult`/`ProtocolStatus`
stay acceptance-only regardless of which completion (if any) was attributed.
## Live alarm watch-list replacement
An auto-mode `SubscribeAlarms` with `fallback_enabled = true` always builds a
`FailoverAlarmConsumer`, even when initial discovery supplied no targets. The
handler retains the subscription expression so `UpdateAlarmWatchList` can build
and subscribe a new subtag consumer entirely on the session STA. The failover
consumer attaches the replacement's transition callback before swapping the
standby reference, then detaches and disposes the old standby. If construction,
subscription, or attachment fails, it disposes the replacement and preserves
the old standby. The active-provider selector is never reset, so primary mode
stays primary and subtag mode immediately delegates to the new standby.
## COM Creation
The MXAccess analysis source at `C:\Users\dohertj2\Desktop\mxaccess` identifies
+1 -1
View File
@@ -247,7 +247,7 @@ A session that faults (for example, a slow single-subscriber client that overflo
A client that drops mid-stream reconnects by re-issuing `StreamEvents` with `StreamEventsRequest.after_worker_sequence` set to the last `worker_sequence` it observed. A non-zero `after_worker_sequence` means *resume*; `0` means *fresh stream* and behaves exactly as a first-time subscribe — no replay, no sentinel.
**Owner-scoped attach (security control).** `OpenSession` records the caller's API key id on the session as `GatewaySession.OwnerKeyId`. Every `StreamEvents` attach and reattach is owner-checked: `EventStreamService.StreamEventsAsync` compares the caller's key id (threaded from `GatewayGrpcAuthorizationInterceptor` via `MxAccessGatewayService.StreamEvents`) to the session owner and throws `SessionManagerException(PermissionDenied)` — surfaced as gRPC `PermissionDenied` — when they differ, before any subscriber is attached and before any replayed or live event is delivered. Possessing the `event` scope and knowing a session id is not enough. This is what makes the default-on detach-grace and replay-retention windows safe: without the check, any `event`-scoped key that learned a session id could attach to another key's retained session and receive its replayed and live data. A `null` owner (session opened with authentication disabled) matches only a `null` caller key.
**Owner-scoped attach (security control).** `OpenSession` records the caller's API key id on the session as `GatewaySession.OwnerKeyId`. Every `StreamEvents` attach and reattach is owner-checked: `EventStreamService.StreamEventsAsync` compares the caller's key id (threaded from `GatewayGrpcAuthorizationInterceptor` via `MxAccessGatewayService.StreamEvents`) to the session owner and throws `SessionManagerException(PermissionDenied)` — surfaced as gRPC `PermissionDenied` — when they differ, before any subscriber is attached and before any replayed or live event is delivered. Possessing the `events:read` scope and knowing a session id is not enough. This is what makes the default-on detach-grace and replay-retention windows safe: without the check, any `events:read`-scoped key that learned a session id could attach to another key's retained session and receive its replayed and live data. A `null` owner (session opened with authentication disabled) matches only a `null` caller key.
On a resume, `EventStreamService.StreamEventsAsync` attaches through `GatewaySession.AttachEventSubscriberWithReplay`, which calls `SessionEventDistributor.RegisterWithReplay`. That method snapshots the session's replay ring for events newer than `after_worker_sequence` **and** registers the live subscriber inside a single `_replayLock` critical section. This atomicity is what makes the replay→live handoff free of gaps and duplicates: the pump appends each event to the replay ring (under `_replayLock`) before fanning it to subscriber channels, so relative to that one critical section every event is either in the replay snapshot or fanned into the freshly-registered live channel — never both observably, never neither.
+19 -5
View File
@@ -26,6 +26,23 @@ The launcher sets the nonce through the `MXGATEWAY_WORKER_NONCE` environment
variable. The nonce is not included in `WorkerProcessCommandLine` so logs and
diagnostics can report the launch command without exposing the secret.
All launch-variable names live in the shared Contracts
`WorkerLaunchEnvironment` class and are consumed by both the launcher and the
worker. The launcher always exports all six values:
| Environment variable | Gateway source | Worker default when missing/invalid |
|---|---|---|
| `MXGATEWAY_WORKER_NONCE` | `WorkerProcessLaunchRequest.Nonce` | None; bootstrap fails because the nonce is required |
| `MXGATEWAY_WORKER_PIPE_CONNECT_ATTEMPT_TIMEOUT_MS` | `MxGateway:Worker:PipeConnectAttemptTimeoutMilliseconds` | `2000` ms |
| `MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS` | `MxGateway:Worker:WriteCompletionWaitMilliseconds` | `1500` ms |
| `MXGATEWAY_EVENT_QUEUE_CAPACITY` | `MxGateway:Worker:EventQueueCapacity` | `10000` events |
| `MXGATEWAY_ALARM_POLL_INTERVAL_MS` | `MxGateway:Alarms:PollIntervalMilliseconds` | `500` ms |
| `MXGATEWAY_ALARM_MAX_ALARMS_PER_FETCH` | `MxGateway:Alarms:MaxAlarmsPerFetch` | `1024` alarms |
The write-completion wait applies to the four single-item write variants:
`Write`, `Write2`, `WriteSecured`, and `WriteSecured2`. Bulk writes report COM
acceptance per entry and do not wait for `OnWriteComplete` rows.
## Validation And Cleanup
Before starting the process, the launcher validates that the configured worker
@@ -44,11 +61,8 @@ process remains alive. The policy is configured by
`WorkerOptions.StartupProbeRetryDelayMilliseconds`; the retry counter is
recorded as `mxgateway.retries.attempted` with `area=worker_startup`.
The launcher also passes
`MXGATEWAY_WORKER_PIPE_CONNECT_ATTEMPT_TIMEOUT_MS` to the worker process from
`WorkerOptions.PipeConnectAttemptTimeoutMilliseconds`. The worker uses that
value as the per-attempt named-pipe connect timeout inside its own bounded
Polly retry loop.
The worker uses the exported pipe-attempt value inside its own bounded Polly
retry loop.
If startup fails or exceeds `WorkerOptions.StartupTimeoutSeconds`, the launcher
kills the worker process tree, disposes the process handle, disposes the
@@ -0,0 +1,119 @@
# Cycle 3 P2 And Follow-Up Design
**Date:** 2026-08-25
**Source:** `archreview/2026-08-19/remediation/00-tracking.md`
**Scope:** The 17 findings recorded as deferred P2 plus the three follow-ups recorded by the cycle-3 integration gate. `SEC-37` and `TST-32` remain external actions and are not part of this repository change.
## Current-State Reconciliation
The 2026-08-25 audit refreshed `origin/main` and checked every finding against the current tree. Sixteen P2 findings remain open. `WRK-34` is already complete in commit `3679e59` and needs a tracker correction rather than another code change. All three integration-gate follow-ups remain open:
- a live worker command and gateway path for refreshing an alarm subscription's subtag watch-list;
- the full WRK-32 recency limitation in the public reply-field comment;
- publication of the already-bumped client release train.
## Design Principles
1. Preserve the existing worker STA and COM-affinity rules. Worker alarm-consumer construction, subscription changes, and disposal remain on the STA.
2. Preserve the primary alarm-manager subscription during watch-list repair. A refresh must never unsubscribe and recreate the whole alarm feed.
3. Make behavioral changes test-first and keep documentation in the same commit as the behavior it describes.
4. Perform one protobuf regeneration wave for the watch-list command, the WRK-32 reply-field clarification, and the IPC-36 operator-facing comment correction.
5. Publish clients only after every client-affecting change is integrated, all five client suites pass, the release commit is on `main`, and registry collision checks pass.
6. Keep the canonical archreview tracker current at every batch boundary, including commit IDs and verification counts.
## Workstreams
### 1. Tracker And Low-Risk Currency
Correct WRK-34 to Done, repair the stale replay-watermark documentation (GWC-36), document the public `DrainEvents` command (IPC-38), reconcile `stillpending.md` (TST-33), wire the code-review README check into portable CI (TST-34), and replace the nonexistent `event` scope spelling with `events:read` (TST-36).
These edits form the first batch because they have a small conflict surface and establish an accurate tracker before code changes begin.
### 2. Gateway Correctness And Security
- **GWC-33:** stop faulted or closed worker clients from contributing frozen queue depth while retaining fault records for the configured grace period.
- **GWC-35:** make snapshot reconciliation emit a Raise when a cached acknowledged alarm is replaced by an active unacknowledged instance.
- **SEC-38:** consult and feed the cross-peer aggregate for every validly-shaped key ID, even when the per-peer key-partition cap routes the request through the fallback bucket. Aggregate cardinality remains bounded by the aggregate cache's existing eviction policy.
- **SEC-39:** never let a null effective key ID clear the shared fallback partition.
Each change gets a focused regression test that first reproduces the filed failure mode.
### 3. Secondary Contracts
- **IPC-36:** move all six worker-launch environment-variable names into one net48-safe Contracts class, consume those constants on both sides, enumerate the launch contract in the launcher and worker design documentation, and test that the launcher sets every declared variable.
- **IPC-37:** compare the retained gateway Galaxy descriptor with the descriptor served by the shared `ZB.MOM.WW.GalaxyRepository` package after normalizing source-only metadata.
This work makes the two non-protobuf peer contracts executable invariants rather than comments.
### 4. Client Alarm-Feed Parity
- **CLI-47:** explicitly render false-valued provider and snapshot status fields in .NET text output; document the deliberate protobuf-JSON omission behavior and Rust's explicit JSON shape.
- **CLI-48:** add provider-status and clearing snapshot-status coverage in .NET, Go, and Java, including Go's missing `stream-alarms` test scaffold.
- **CLI-49:** expose the two Go status payload aliases and update the alarm-feed alias documentation.
- **CLI-50:** change Python `stream-alarms` from a one-message buffered sample to incremental streaming with an unbounded-by-default limit, while retaining timeout and explicit finite-limit behavior.
The Python change uses the same arrival-order streaming model as the other four CLIs. Documentation examples will request a finite bound when a bounded demonstration is desirable.
### 5. Live Alarm Watch-List Refresh
The complete periodic hot-swap design is selected over failover-edge-only refresh and unsubscribe/resubscribe.
#### Contract
Add an explicit worker command for updating an active alarm subscription's watch-list. Add an explicit subscription flag that distinguishes these cases when the initial list is empty:
- alarm-manager-only deployment with no fallback requested;
- auto fallback configured, but discovery currently produced no targets.
The command is gateway-owned even though it travels through the shared `MxCommand` envelope. It returns a normal `MxCommandReply`; invalid mode/state or replacement-subscription failure returns a structured failure without destroying the existing subscription.
#### Worker
In auto mode, retain a failover composite even when fallback is enabled with an initially empty list. The composite retains its subscription expression and supports replacing its standby consumer on the STA:
1. construct the replacement standby;
2. subscribe and warm it before altering the current child;
3. attach transition forwarding;
4. atomically replace the standby reference;
5. dispose the previous standby;
6. if standby is currently active, keep the mode on subtag and expose the replacement snapshot immediately.
If any pre-swap step fails, dispose the replacement and keep the old standby untouched. Forced alarm-manager mode rejects refresh as inapplicable. Forced subtag mode may replace its sole consumer through the same build-first/swap-second rule.
#### Gateway
Reuse the existing alarm reconciliation cadence; do not add another timer or configuration option. Resolve only when fallback can use subtags. Compare a canonical target fingerprint with the list last acknowledged by the worker, and send an update only after material change or recovery from an empty/failed resolution.
The failover edge still performs an immediate refresh attempt before publishing its coverage verdict. A successful update changes the recorded sent fingerprint, clears stale-coverage status, and triggers reconciliation. A failed update retains the previous fingerprint and publishes/logs the degraded reason.
This design repairs the original startup-outage case before failover whenever Galaxy Repository recovers during normal monitor operation, while preserving the warm primary and current provider mode.
### 6. Windows Gateway Coverage
**TST-35** extends windev `live` mode to run the net10 gateway suite after the serialized Worker tier and full solution build. The existing host lock remains the load-sensitivity control. Logs record the observed count rather than hard-coding the historical 879 or 1183 count as an invariant.
### 7. Protobuf Regeneration And Release
The watch-list command lands in the same regeneration wave as:
- WRK-32's explicit sequential stale-callback/recency limitation;
- IPC-36's replacement of the internal environment-variable name with the public configuration key.
Regenerate Contracts and all five client trees according to `docs/Contracts.md` and `docs/ClientPackaging.md`, then run the codegen cleanliness checks and every affected suite.
After integration to `main`, publish .NET/Contracts, Python, Rust, and Go as `0.2.1`, and Java as `0.2.2`. Query each exact coordinate before publishing, never overwrite, create and push the Go module tag at the exact release commit, then query every registry coordinate again and update the packaging record and tracker with immutable release evidence.
## Verification Strategy
- macOS: clean `NonWindows.slnx` build and full gateway suite.
- Focused gateway tests after each behavioral batch.
- All five client build/test commands from `docs/ClientPackaging.md`.
- `scripts/check-codegen.ps1`, client-version guard, and code-review README check.
- windev: x86 Worker build plus Worker.Tests for worker/proto changes; `live` mode for TST-35 and the final integration gate.
- Release: exact registry-coordinate checks and Go module resolution from the pushed tag.
## Completion Criteria
The continuation is complete only when all 16 open P2 findings and all three follow-ups are Done or explicitly blocked with evidence, WRK-34 is corrected in the tracker, every required verification is green, client releases are confirmed at their registries, and `archreview/2026-08-19/remediation/00-tracking.md` contains the commit and test evidence for every batch.
@@ -0,0 +1,405 @@
# Cycle 3 P2 And Follow-Up Implementation Plan
> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.
**Goal:** Close the 16 still-open cycle-3 P2 findings, correct WRK-34's tracker status, implement the three integration-gate follow-ups, and publish the completed client release train.
**Architecture:** Work proceeds in seven reviewable batches. Small tracker and documentation corrections land first; focused gateway, security, and secondary-contract fixes follow; client parity is completed before a single protobuf regeneration wave adds live alarm watch-list refresh and the remaining public comments; Windows CI is extended before the final windev gate; packages are published only from the integrated `main` release commit.
**Tech Stack:** .NET 10/C#, .NET Framework 4.8 x86, protobuf/Grpc.Tools, PowerShell 5.1/7, Go, Python/pytest, Rust/Cargo, Java/Gradle, Gitea Actions and package registries.
---
## Preconditions
- Work only in `.worktrees/c3-p2-followups` on `codex/c3-p2-followups`.
- Preserve the clean baseline recorded on 2026-08-20/25: `NonWindows.slnx` builds with zero warnings and the gateway suite passes 1183/1183.
- Follow red-green-refactor for every behavior change. Documentation-only changes use an explicit failing grep/check as their red gate.
- After every batch, update `archreview/2026-08-19/remediation/00-tracking.md` with status, commits, and exact verification output before committing.
- Worker/protobuf changes require windev x86 verification before Done.
### Task 1: Batch H — Tracker And Documentation Currency
**Findings:** WRK-34 tracker correction, GWC-36, IPC-38, TST-33, TST-34, TST-36
**Files:**
- Modify: `archreview/2026-08-19/remediation/00-tracking.md`
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Sessions/EventSubscriberReplayAttachment.cs`
- Modify: `docs/Grpc.md`
- Modify: `stillpending.md`
- Modify: `.gitea/workflows/ci.yml`
- Modify: `REVIEW-PROCESS.md`
- Modify: `CLAUDE.md`
- Modify: `docs/Sessions.md`
- Modify: `docs/DesignDecisions.md`
**Step 1: Capture the failing currency checks**
Run:
```bash
git grep -n '`event` scope' -- CLAUDE.md docs/
rg -n 'not safe to rely|Phase 5, not started|6 \[LiveMxAccessFact\]|wire behavior is unverified|still scheduled.*Phase 4' stillpending.md
rg -n 'regen-readme.py|check-code-reviews-readme' .gitea/workflows/ci.yml
rg -n 'DrainEvents' docs/Grpc.md
```
Expected: stale scope and pending claims are found; CI and `DrainEvents` checks find no intended entry.
**Step 2: Correct GWC-36 and TST-36**
Change `OldestAvailableSequence` to document retained-oldest or `highest observed + 1` when an empty ring has a gap, with `0` only when `Gap` is false. Replace prose references to the nonexistent ``event`` scope with ``events:read`` without renaming identifiers such as `GatewayScopes.EventsRead`.
**Step 3: Correct IPC-38 and TST-33**
Add a `DrainEvents` command row to `docs/Grpc.md` covering the 10,000-event count cap, byte cap, truncation diagnostic, drain-until-empty contract, and removal from the normal session event stream. Reconcile the five stale `stillpending.md` claims and stamp the file as a historical inventory whose live status comes from archreview trackers.
**Step 4: Wire TST-34**
Add this portable-job step next to the existing repository consistency checks:
```yaml
- name: Code-reviews README consistency
run: python code-reviews/regen-readme.py --check
```
Qualify the interpreter note in `REVIEW-PROCESS.md`: Windows may require `python`; macOS/Linux normally use `python3`, while CI uses its configured `python`.
**Step 5: Re-run the checks**
Run:
```bash
python3 code-reviews/regen-readme.py --check
git grep -n '`event` scope' -- CLAUDE.md docs/ || true
rg -n 'not safe to rely|Phase 5, not started|6 \[LiveMxAccessFact\]|wire behavior is unverified|still scheduled.*Phase 4' stillpending.md || true
rg -n 'DrainEvents' docs/Grpc.md
dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx
```
Expected: README check passes; stale greps return nothing; DrainEvents is documented; build has zero warnings/errors.
**Step 6: Update tracker and commit**
Mark the five P2 findings Done and WRK-34 Done (`3679e59`), record verification, then commit:
```bash
git add .gitea/workflows/ci.yml REVIEW-PROCESS.md CLAUDE.md stillpending.md docs/Grpc.md docs/Sessions.md docs/DesignDecisions.md src/ZB.MOM.WW.MxGateway.Server/Sessions/EventSubscriberReplayAttachment.cs archreview/2026-08-19/remediation/00-tracking.md
git commit -m "chore(GWC-36,IPC-38,TST-33,TST-34,TST-36): close cycle-3 currency gaps"
```
### Task 2: Batch I — Gateway Correctness And Security
**Findings:** GWC-33, GWC-35, SEC-38, SEC-39
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerClient.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/WorkerClientTests.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Alarms/GatewayAlarmMonitor.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Alarms/GatewayAlarmMonitorAttachOrderTests.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Security/Authorization/ApiKeyFailureLimiter.cs`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Security/Authorization/ApiKeyFailureLimiterTests.cs`
- Modify: `docs/Authentication.md`
- Modify: `archreview/2026-08-19/remediation/00-tracking.md`
**Step 1: RED for GWC-33**
Add a worker-client test that stages an event, faults the client without disposing it, and asserts the aggregate queue-depth observable stops counting that client while the faulted object remains alive. Run:
```bash
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter FullyQualifiedName~WorkerClientTests
```
Expected: the new assertion fails because the registration is disposed only by `DisposeAsync`.
**Step 2: GREEN for GWC-33**
Make queue-depth registration disposal idempotent and call it on both terminal transitions and disposal. Do not clear the faulted client's diagnostic state or shorten `FaultedGraceSeconds` retention. Re-run the WorkerClient filter and commit the focused change.
**Step 3: RED/GREEN for GWC-35**
Add a monitor test whose first reconcile contains `ActiveAcked` and whose next reconcile carries a newer `Active` instance. Assert one `Raise` repair carrying the incoming instance identity. Verify red, then generalize the both-present state delta:
```csharp
AlarmTransitionKind kind = incoming.Value.CurrentState == AlarmConditionState.ActiveAcked
? AlarmTransitionKind.Acknowledge
: AlarmTransitionKind.Raise;
```
Keep the existing duplicate suppression keyed by original raise timestamp. Re-run `GatewayAlarmMonitorAttachOrderTests`.
**Step 4: RED/GREEN for SEC-38**
Add a limiter test that fills all 32 peer key-ID partitions, then attacks one valid victim key ID through the fallback from multiple peers. Assert the victim aggregate trips. Change resolution/recording so every validly-shaped presented key ID consults and feeds its aggregate even if its peer partition fell back; continue using the resolved partition key for the peer layer and the existing capped aggregate eviction for cardinality. Update class remarks and `docs/Authentication.md`.
**Step 5: RED/GREEN for SEC-39**
Add a test that records fallback failures, calls `Reset` with a null effective key ID, and asserts the fallback remains tracked. Tighten ownership:
```csharp
if (effectiveKeyId is not null
&& string.Equals(effectiveKeyId, partition.KeyId, StringComparison.Ordinal))
{
RemovePartition(partitionKey);
}
```
**Step 6: Verify, track, and commit**
Run:
```bash
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~WorkerClientTests|FullyQualifiedName~GatewayAlarmMonitorAttachOrderTests|FullyQualifiedName~ApiKeyFailureLimiterTests"
dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx
```
Expected: all filtered tests pass; zero warnings/errors. Update tracker with per-finding commits and counts.
### Task 3: Batch J — Secondary Contract Guards
**Findings:** IPC-36, IPC-37
**Files:**
- Create: `src/ZB.MOM.WW.MxGateway.Contracts/WorkerLaunchEnvironment.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Server/Workers/WorkerProcessLauncher.cs`
- Modify: worker consumers under `src/ZB.MOM.WW.MxGateway.Worker/Bootstrap`, `Ipc`, and `MxAccess`
- Test: `src/ZB.MOM.WW.MxGateway.Tests/Gateway/Workers/WorkerProcessLauncherTests.cs`
- Create: `src/ZB.MOM.WW.MxGateway.Tests/Contracts/GalaxyContractParityTests.cs`
- Modify: `src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj`
- Modify: `docs/WorkerProcessLauncher.md`
- Modify: `docs/MxAccessWorkerInstanceDesign.md`
- Modify: `docs/Contracts.md`
- Modify: `archreview/2026-08-19/remediation/00-tracking.md`
**Step 1: RED for IPC-36**
Change the launcher test to enumerate the six constants expected on `WorkerLaunchEnvironment` and assert each is present in `ProcessStartInfo.Environment`. Run the test and confirm compilation fails because the shared class does not exist.
**Step 2: GREEN for IPC-36**
Create a net48-safe static class containing six `public const string` names. Replace both gateway and worker literals with these constants; remove the duplicate public constants from `WorkerProcessLauncher` unless compatibility requires obsolete aliases. Add a launch-environment table with variable, source configuration key, and worker default. Refresh Write/Write2 wording.
**Step 3: RED for IPC-37**
Add `GalaxyContractParityTests` that converts both loaded descriptors with `ToProto()`, clears `SourceCodeInfo`, normalizes `Options.CsharpNamespace` and file `Name`, then compares deterministic bytes. Before the assertion, mutate a clone in a local test helper to prove the comparison reports the differing symbol/message clearly; the real descriptor assertion must pass at HEAD.
**Step 4: Document and verify**
Record the parity guard in the Contracts project comment and `docs/Contracts.md`. Run:
```bash
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~WorkerProcessLauncherTests|FullyQualifiedName~GalaxyContractParityTests"
dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx
```
Then update tracker and commit IPC-36/37.
### Task 4: Batch K — Client Alarm-Feed Parity
**Findings:** CLI-47, CLI-48, CLI-49, CLI-50
**Files:**
- Modify: `clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli/MxGatewayClientCli.cs`
- Test: `clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/MxGatewayClientCliTests.cs`
- Test/modify: `clients/go/cmd/mxgw-go/main.go`, `main_test.go`
- Modify: `clients/go/mxgateway/types.go`
- Test/modify: Java CLI and `MxGatewayCliTests.java`
- Modify/test: `clients/python/src/zb_mom_ww_mxgateway_cli/commands.py`, `clients/python/tests/`
- Modify: all affected client READMEs and `docs/CrossLanguageSmokeMatrix.md`
- Modify: `archreview/2026-08-19/remediation/00-tracking.md`
**Step 1: RED/GREEN for CLI-47 and CLI-48**
In .NET, Go, and Java tests, feed `provider_status { degraded: false }` and `snapshot_status { truncated: false }`; assert exact text rows and protobuf-JSON behavior. Confirm .NET text fails as `{}` and Go lacks a stream-alarm harness. Hand-format .NET text to match the explicit rows; add the missing Go and Java coverage without changing their correct output.
**Step 2: RED/GREEN for CLI-49**
Add a Go compile-time consumer test that declares variables of `mxgateway.AlarmProviderStatus` and `mxgateway.AlarmSnapshotStatus`. Confirm red, then add aliases and update the five-arm `AlarmFeedMessage` comment.
**Step 3: RED/GREEN for CLI-50**
Add Python CLI tests proving default `stream-alarms` emits provider status, snapshot status, cached alarms, and later live messages incrementally until timeout/cancellation; add a finite `--max-messages` test. Change the default to `0` (unbounded) and print rows as consumed instead of collecting first. Keep the maximum bound for positive finite values and ensure iterator cleanup occurs in `finally`.
**Step 4: Verify all clients**
Run the authoritative commands from `docs/ClientPackaging.md`:
```bash
dotnet test clients/dotnet/ZB.MOM.WW.MxGateway.Client.slnx
(cd clients/go && go test ./...)
(cd clients/python && python -m pytest)
(cd clients/rust && cargo test --workspace)
(cd clients/java && JAVA_HOME=/opt/homebrew/opt/openjdk@17 gradle test)
```
Revert only proven generator churn; do not revert source changes. Update tracker and commit CLI-47–50.
### Task 5: Batch L — Live Watch-List Refresh And Protobuf Wave
**Follow-ups:** complete GWC-37 application path, WRK-32 public comment
**Files:**
- Modify: `src/ZB.MOM.WW.MxGateway.Contracts/Protos/mxaccess_gateway.proto`
- Regenerate: Contracts and every client generated tree required by `docs/Contracts.md`
- Modify/test: worker alarm command executor, handler, failover/subtag consumers, and Worker.Tests
- Modify/test: `src/ZB.MOM.WW.MxGateway.Server/Alarms/GatewayAlarmMonitor.cs` and alarm monitor tests
- Modify: alarm, worker protocol, configuration, and client contract documentation
- Modify: `archreview/2026-08-19/remediation/00-tracking.md`
**Step 1: RED — contract round trip**
Add a protobuf round-trip test for:
```proto
message UpdateAlarmWatchListCommand {
repeated AlarmSubtagTarget watch_list = 1;
}
```
Add a new non-conflicting `MxCommand` oneof field and `MxCommandKind`, plus an explicit `fallback_enabled` field on `SubscribeAlarmsCommand`. Confirm the test cannot compile before editing the proto.
**Step 2: RED — worker routing and failure preservation**
Add Worker.Tests covering command routing, rejection without an active/fallback-capable subscription, replacement setup failure preserving the previous standby, primary-mode replacement keeping primary active, and standby-mode replacement keeping subtag active.
**Step 3: GREEN — worker hot-swap**
Implement `IAlarmCommandHandler.UpdateWatchList`. Retain the subscription expression. Refactor `FailoverAlarmConsumer` so standby replacement performs build/subscribe/attach before swap and disposes the old standby after swap. No COM operation may leave the STA. In auto mode, build the composite with an empty standby when `fallback_enabled` is true; leave an intentionally alarm-manager-only subscription unchanged.
**Step 4: RED — gateway periodic update**
Extend `GatewayAlarmMonitorProviderModeTests`/`AlarmFailoverEndToEndTests` with these scenarios:
- startup discovery fails/empty, later reconcile resolves targets, exactly one update command is sent;
- unchanged targets send no update;
- changed targets update the sent fingerprint only after an OK reply;
- worker rejection retains the prior fingerprint and degraded coverage reason;
- failover edge performs an immediate refresh before publishing status.
**Step 5: GREEN — gateway refresh path**
Reuse `ReconcileIntervalSeconds`. Resolve only when mode can use subtags, compare canonical full-target fingerprints, invoke the new command on material change/recovery, and reconcile after success. Do not add a timer or option.
**Step 6: Complete the shared proto comments**
In the same proto edit:
- replace the IPC-36 internal environment-variable name with `MxGateway:Worker:WriteCompletionWaitMilliseconds`;
- describe WRK-32 correlation as recency-based per handle and name the sequential timed-out-predecessor stale-callback case, not only concurrent writes.
**Step 7: Regenerate once and run portable checks**
Follow `docs/Contracts.md` exactly, then run:
```bash
pwsh scripts/check-codegen.ps1
dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx
dotnet test src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj --filter "FullyQualifiedName~ProtobufContractRoundTripTests|FullyQualifiedName~AlarmFailoverEndToEndTests|FullyQualifiedName~GatewayAlarmMonitorProviderModeTests|FullyQualifiedName~ClientProtoInputTests"
```
Run all five client suites again because generated trees changed.
**Step 8: Verify worker on windev**
Commit the candidate, then run the repository's windev test mode at that SHA. Required evidence: x86 Worker build success and Worker.Tests with zero failures. Only then mark the watch-list/WRK-32 follow-ups Done in the tracker and commit any evidence-only tracker update.
### Task 6: Batch M — Windows Gateway Suite In Nightly CI
**Finding:** TST-35
**Files:**
- Modify: `scripts/ci/windev-worker-ci.ps1`
- Modify: `.gitea/workflows/ci.yml` comments if needed
- Modify: `docs/GatewayTesting.md`
- Modify: `scripts/ci/README.md`
- Modify: `archreview/2026-08-19/remediation/00-tracking.md`
**Step 1: Add an acceptance check before behavior**
Document the expected `live` order and add a script-level/static assertion or dry-run seam showing gateway tests are absent before the change.
**Step 2: Extend live mode**
After Worker.Tests and full-slnx build, run:
```powershell
Invoke-Native 'gateway tests on Windows' {
dotnet test 'src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj' -c Release --no-build
}
```
Keep it before the opt-in live-MXAccess smoke, under the existing host lock. Log the actual result; do not enforce a historical count.
**Step 3: Verify on windev**
Run `live` mode at the committed SHA. Require Worker.Tests, gateway tests, full solution build, and live smoke to complete successfully. Update docs/tracker with date, SHA, counts, and duration; commit TST-35.
### Task 7: Batch N — Full Integration And Release
**Follow-up:** publish the five clients and Contracts
**Files:**
- Modify after publication: `docs/ClientPackaging.md`
- Modify: `archreview/2026-08-19/remediation/00-tracking.md`
**Step 1: Full local integration gate**
Run:
```bash
git diff --check
pwsh scripts/check-codegen.ps1
pwsh scripts/check-client-versions.ps1
python3 code-reviews/regen-readme.py --check
dotnet build src/ZB.MOM.WW.MxGateway.NonWindows.slnx
dotnet test src/ZB.MOM.WW.MxGateway.NonWindows.slnx --no-build --no-restore
```
Run all five client suites. Require zero unexpected working-tree changes.
**Step 2: Final windev gate**
Run `windev-worker-ci.ps1 -Mode live` at the exact candidate SHA. Record Worker.Tests, gateway tests, live smoke, and full solution build evidence.
**Step 3: Integrate before release**
Use `superpowers:finishing-a-development-branch`. Merge/push the verified branch to `main`, confirm CI green, and resolve the exact release commit SHA. Do not publish artifacts from an unmerged feature commit.
**Step 4: Preflight exact coordinates**
Confirm these are absent (HTTP 404) immediately before publishing:
- NuGet `ZB.MOM.WW.MxGateway.Client` 0.2.1
- NuGet `ZB.MOM.WW.MxGateway.Contracts` 0.2.1
- PyPI `zb-mom-ww-mxaccess-gateway-client` 0.2.1
- Cargo `zb-mom-ww-mxgateway-client` 0.2.1
- Maven `com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client` 0.2.2
- Git tag `clients/go/v0.2.1`
Abort rather than overwrite if any coordinate exists unexpectedly.
**Step 5: Publish from clean `main`**
Run the documented `scripts/pack-clients.ps1 -Publish` commands for .NET, Python, Rust, and Java with required credentials, then:
```powershell
pwsh scripts/tag-go-module.ps1 -Version v0.2.1 -Push
```
The Go tag must point to the exact integrated release commit.
**Step 6: Verify release and close tracker**
Query every exact registry coordinate for HTTP 200 and resolve the Go module/tag remotely. Update `docs/ClientPackaging.md` and the tracker with release SHA, coordinates, and verification date. Run the doc checks, commit the release record, and push it.
## Final Tasklist Exit Gate
- [x] All 16 open P2 findings are Done with commit/test evidence.
- [x] WRK-34 is corrected to Done (`3679e59`).
- [x] Live watch-list refresh passes portable and windev tests.
- [x] WRK-32 public proto comment is regenerated through every client surface.
- [x] TST-35 nightly Windows gateway suite is proven on windev.
- [x] Full NonWindows gateway suite and all five client suites pass.
- [x] Worker.Tests and Windows gateway suite pass at the integrated release SHA.
- [x] Five client releases plus Contracts are confirmed in registries; Go tag resolves.
- [x] `archreview/2026-08-19/remediation/00-tracking.md` contains no stale Deferred row for completed work.
+6 -2
View File
@@ -11,11 +11,15 @@ code back so a regression turns the job red.
| File | Runs on | Role |
|---|---|---|
| `run-windev-ci.sh <build\|test\|live>` | Linux CI job | Writes the SSH key/known-hosts, base64-encodes a bootstrap, SSHes to windev, returns the remote exit code. |
| `windev-worker-ci.ps1 -Sha -Mode` | windev | Locks the CI clone, checks out the SHA, runs the x86 build / `Worker.Tests` / live smoke. |
| `windev-worker-ci.ps1 -Sha -Mode` | windev | Locks the CI clone, checks out the SHA, and runs the requested cumulative Windows tiers. |
| `windev.known_hosts` | committed | Pinned windev host keys so `StrictHostKeyChecking=yes` has no TOFU prompt. Host keys are public; committing is intentional. |
Modes are cumulative: `build` = x86 Worker build; `test` = build + `Worker.Tests`;
`live` = test + full-`slnx` build + live-MXAccess smoke (`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`).
`live` = test + full-`slnx` build + the net10 gateway suite + live-MXAccess smoke
(`MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1`). The `live` order is x86 Worker build,
`Worker.Tests`, full solution build, gateway tests, then live smoke. One host lock covers
checkout and every tier, so concurrent runs cannot overlap; each native command is checked
immediately and the first nonzero exit aborts the remaining tiers and propagates to CI.
CI jobs (`.gitea/workflows/ci.yml`): `windows-x86` runs `test` per push/PR; `nightly-windev`
runs `live` on the `0 6 * * *` schedule and opens a Gitea issue on failure.
+7 -2
View File
@@ -16,8 +16,8 @@
Modes (cumulative):
build x86 Worker build only (net48/CS0246/x86 compile guard — always fast).
test build + Worker.Tests (x86). The per-push default.
live test + the live-MXAccess smoke (MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1) and a full
slnx build (surfaces Windows-only analyzer diagnostics, e.g. xUnit1030). Nightly.
live test + full slnx build + the net10 gateway suite + the live-MXAccess smoke
(MXGATEWAY_RUN_LIVE_MXACCESS_TESTS=1). Nightly.
Runs under Windows PowerShell 5.1 (`powershell.exe`), so it avoids PS7-only syntax and
checks $LASTEXITCODE after every native command rather than relying on stderr-as-error
@@ -112,6 +112,7 @@ try {
$workerProj = 'src/ZB.MOM.WW.MxGateway.Worker/ZB.MOM.WW.MxGateway.Worker.csproj'
$workerTests = 'src/ZB.MOM.WW.MxGateway.Worker.Tests/ZB.MOM.WW.MxGateway.Worker.Tests.csproj'
$gatewayTests = 'src/ZB.MOM.WW.MxGateway.Tests/ZB.MOM.WW.MxGateway.Tests.csproj'
$integrationTests = 'src/ZB.MOM.WW.MxGateway.IntegrationTests/ZB.MOM.WW.MxGateway.IntegrationTests.csproj'
# build: always. The x86/net48 compile is the floor guard (catches CS0246 on
@@ -127,6 +128,10 @@ try {
# that never fire on the macOS/Linux NonWindows build.
Invoke-Native 'full slnx build' { dotnet build src/ZB.MOM.WW.MxGateway.slnx -c Release }
Invoke-Native 'gateway tests on Windows' {
dotnet test $gatewayTests -c Release --no-build
}
$env:MXGATEWAY_RUN_LIVE_MXACCESS_TESTS = '1'
try {
Invoke-Native 'live-MXAccess smoke' {
@@ -25,7 +25,7 @@ public static class GatewayContractInfo
/// (<c>WorkerPipeSession.CreateDrainEventsReply</c>, the backstop that also interprets
/// <c>max_events = 0</c>) cannot drift apart. A count cap alone is necessary but not
/// sufficient: the worker additionally caps the reply by serialized bytes against the
/// negotiated frame maximum (WRK-21), so a reply may carry fewer events than this ceiling
/// negotiated frame maximum, so a reply may carry fewer events than this ceiling
/// and fewer than are queued. Callers drain iteratively until an empty reply.
/// This is a documented behavioral bound, not wire schema — it is deliberately a C#
/// constant and not a <c>.proto</c> field.
File diff suppressed because it is too large Load Diff
@@ -130,6 +130,7 @@ message MxCommand {
WriteSecuredBulkCommand write_secured_bulk = 41;
WriteSecured2BulkCommand write_secured2_bulk = 42;
ReadBulkCommand read_bulk = 43;
UpdateAlarmWatchListCommand update_alarm_watch_list = 44;
PingCommand ping = 100;
GetSessionStateCommand get_session_state = 101;
GetWorkerInfoCommand get_worker_info = 102;
@@ -174,6 +175,7 @@ enum MxCommandKind {
MX_COMMAND_KIND_WRITE_SECURED_BULK = 32;
MX_COMMAND_KIND_WRITE_SECURED2_BULK = 33;
MX_COMMAND_KIND_READ_BULK = 34;
MX_COMMAND_KIND_UPDATE_ALARM_WATCH_LIST = 35;
MX_COMMAND_KIND_PING = 100;
MX_COMMAND_KIND_GET_SESSION_STATE = 101;
MX_COMMAND_KIND_GET_WORKER_INFO = 102;
@@ -344,8 +346,8 @@ enum AlarmProviderMode {
// Subscription expression follows the canonical
// `\\<machine>\Galaxy!<area>` format (literal "Galaxy" provider). The
// worker spins up a wnwrapConsumer-backed subscription on its STA on
// first call; subsequent calls are an error (use UnsubscribeAlarms then
// SubscribeAlarms to reconfigure).
// first call; subsequent calls are an error. Use UpdateAlarmWatchList to
// replace the standby targets without restarting the primary subscription.
message SubscribeAlarmsCommand {
string subscription_expression = 1;
// UNSPECIFIED = auto-failover/failback. ALARMMGR/SUBTAG force one provider.
@@ -354,6 +356,17 @@ message SubscribeAlarmsCommand {
// alarmmgr mode; in subtag mode it bounds what the consumer can observe.
repeated AlarmSubtagTarget watch_list = 3;
AlarmFailoverConfig failover = 4;
// True when auto mode should retain subtag-fallback capability even if the
// gateway's initial discovery produced an empty watch list. False leaves an
// empty list as an intentional alarm-manager-only subscription.
bool fallback_enabled = 5;
}
// Replace the live alarm subscription's subtag standby targets. The worker
// constructs and attaches the replacement before swapping it into service, so
// a setup failure leaves the existing standby and active provider unchanged.
message UpdateAlarmWatchListCommand {
repeated AlarmSubtagTarget watch_list = 1;
}
// Tear down the worker's alarm consumer. No-op if no subscription is
@@ -537,11 +550,14 @@ message MxCommandReply {
MxValue return_value = 6;
// Correlated per-item outcome rows. For WRITE / WRITE2 / WRITE_SECURED /
// WRITE_SECURED2 replies the worker holds the reply for a bounded window
// (default 1.5 s, MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS) waiting for
// (default 1.5 s, MxGateway:Worker:WriteCompletionWaitMilliseconds) waiting for
// the matching MXAccess OnWriteComplete callback and copies its status rows
// here, so statuses[0] carries the real MXAccess commit outcome (success OR
// failure) while protocol_status/hresult still describe command acceptance
// only. Empty statuses on a write reply means the completion did not arrive
// only. Callback correlation is recency-based per item handle: after the
// bounded wait, a sequential write's callback supersedes a timed-out
// predecessor, and a later stale callback for that predecessor is ignored.
// Empty statuses on a write reply means the completion did not arrive
// within the window — the write is unconfirmed, not failed. Correlation is
// best-effort per (server_handle, item_handle): MXAccess's callback carries
// no transaction id, so concurrent writes to the same item within the
@@ -0,0 +1,25 @@
namespace ZB.MOM.WW.MxGateway.Contracts;
/// <summary>Names of environment variables shared by the gateway worker launcher and worker process.</summary>
public static class WorkerLaunchEnvironment
{
/// <summary>Bootstrap nonce used to authenticate the worker handshake.</summary>
public const string WorkerNonce = "MXGATEWAY_WORKER_NONCE";
/// <summary>Per-attempt named-pipe connection timeout in milliseconds.</summary>
public const string PipeConnectAttemptTimeoutMilliseconds =
"MXGATEWAY_WORKER_PIPE_CONNECT_ATTEMPT_TIMEOUT_MS";
/// <summary>Bounded wait for a Write/Write2/WriteSecured/WriteSecured2 completion callback in milliseconds.</summary>
public const string WriteCompletionWaitMilliseconds =
"MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS";
/// <summary>Capacity of the worker's outbound MXAccess event queue.</summary>
public const string EventQueueCapacity = "MXGATEWAY_EVENT_QUEUE_CAPACITY";
/// <summary>Worker alarm polling cadence in milliseconds.</summary>
public const string AlarmPollIntervalMilliseconds = "MXGATEWAY_ALARM_POLL_INTERVAL_MS";
/// <summary>Maximum alarm records requested in one current-alarm fetch.</summary>
public const string MaxAlarmsPerFetch = "MXGATEWAY_ALARM_MAX_ALARMS_PER_FETCH";
}
@@ -37,7 +37,9 @@
the wire-identical Galaxy types from the ZB.MOM.WW.GalaxyRepository package. This proto is
still the generation source of truth for the language clients (Go/Rust/Java/Python compile it
by path; the .NET client consumes Contracts.Proto.Galaxy.* via this project reference) and for
clients/proto/proto-inputs.json. Removing it breaks all five clients — do not delete. -->
clients/proto/proto-inputs.json. GalaxyContractParityTests normalizes its descriptor against
the descriptor served by the package and fails on wire drift. Removing it breaks all five
clients — do not delete. -->
<Protobuf Include="Protos\galaxy_repository.proto" ProtoRoot="Protos" OutputDir="Generated" GrpcOutputDir="Generated" GrpcServices="Both" />
</ItemGroup>
@@ -27,7 +27,7 @@ public sealed class DashboardLdapLiveTests
private const string SharedDirectoryPassword = "password";
/// <summary>
/// Dashboard visibility tags (SEC-25) used by the ACL scenarios below. They are operator-chosen
/// Dashboard visibility tags used by the ACL scenarios below. They are operator-chosen
/// labels that exist only in this fixture's configuration and in the fake sessions' owner-tag
/// list — nothing in the shared directory carries them.
/// </summary>
@@ -168,7 +168,7 @@ public sealed class DashboardLdapLiveTests
}
/// <summary>
/// Verifies the SEC-25 tag grant end-to-end from a real LDAP bind: <c>gw-viewer</c>'s only
/// Verifies the tag grant end-to-end from a real LDAP bind: <c>gw-viewer</c>'s only
/// group (GwReader) is mapped to the <c>team-a</c> visibility tag by <c>Dashboard:GroupToTag</c>,
/// and the principal that bind produces is admitted by <see cref="IDashboardSessionAcl"/> for a
/// <c>team-a</c>-tagged session but refused for a <c>team-b</c>-tagged one.
@@ -275,7 +275,7 @@ public sealed class DashboardLdapLiveTests
};
/// <summary>
/// The SEC-25 fixture map: GwReader is admitted as a Viewer and granted <c>team-a</c>. Both keys
/// The tagged fixture map: GwReader is admitted as a Viewer and granted <c>team-a</c>. Both keys
/// name groups that already exist in the shared directory — the tag layer is config-only.
/// </summary>
private static DashboardOptions TaggedDashboardOptions() => new()
@@ -526,7 +526,7 @@ public sealed class WorkerLiveMxAccessSmokeTests(ITestOutputHelper output)
// Assert. But a silent fall-through to "operator only" (currentUserId 0)
// quietly demotes the rest of this test from an authenticated round-trip
// to a bare failure-reply-shape check, which a stale rig credential (e.g.
// TST-31) would otherwise hide indefinitely in nightly runs. Log the
// one that has expired) would otherwise hide indefinitely in nightly runs. Log the
// outcome loudly — never the password (verifyUser only) — so that
// degradation is visible in the test output even though it never fails
// the test by itself.
@@ -22,8 +22,8 @@
(IntegrationTests-028).
-->
<ItemGroup>
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.2.1" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.2.1" />
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.3.0" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.3.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="10.0.7" />
<PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="10.0.7" />
</ItemGroup>
@@ -19,7 +19,7 @@ public static class AlarmProviderReasons
/// <summary>
/// Reason prefix set when the effective mode can need the subtag watch-list
/// (<c>Auto</c> or <c>ForceSubtag</c>) but the list resolved to zero targets, so the
/// subtag provider covers no alarms (GWC-37). In <c>Auto</c> this is a warning about the
/// subtag provider covers no alarms. In <c>Auto</c> this is a warning about the
/// standby, not about the currently-serving alarm manager, which is why the dashboard
/// renders it against the <c>ALARMMGR</c> mode as a coverage badge rather than a failover.
/// </summary>
@@ -29,7 +29,7 @@ public static class AlarmProviderReasons
/// <summary>
/// Reason prefix set when Galaxy Repository alarm discovery was enabled but failed, so
/// the watch-list carries only what configuration contributed and may be incomplete
/// (GWC-37). Distinct from <see cref="SubtagWatchListEmpty"/>: discovery failing with a
/// . Distinct from <see cref="SubtagWatchListEmpty"/>: discovery failing with a
/// usable config-only list is a completeness warning, not a total loss of coverage.
/// </summary>
public const string SubtagDiscoveryFailed =
@@ -37,7 +37,7 @@ public static class AlarmProviderReasons
/// <summary>
/// Reason prefix set when a re-resolution at the failover edge produced a different
/// watch-list than the one sent at <c>SubscribeAlarms</c> (GWC-37). The worker keeps the
/// watch-list than the one sent at <c>SubscribeAlarms</c>. The worker keeps the
/// list it was given for the life of the subscription, so the difference is reported
/// rather than applied; a monitor restart re-resolves and re-sends.
/// </summary>
@@ -5,8 +5,8 @@ namespace ZB.MOM.WW.MxGateway.Server.Alarms;
/// <summary>
/// The outcome of one subtag watch-list resolution: the composed targets plus enough
/// provenance for the caller to tell an empty list that means "nothing was configured"
/// from one that means "discovery was configured and silently produced nothing"
/// (GWC-37). <see cref="IAlarmWatchListResolver.ResolveAsync"/> deliberately never throws
/// from one that means "discovery was configured and silently produced nothing".
/// <see cref="IAlarmWatchListResolver.ResolveAsync"/> deliberately never throws
/// on a discovery failure, so without this the two are indistinguishable at the call site
/// and a Galaxy Repository outage at startup can only be found in a log line.
/// </summary>
@@ -39,7 +39,7 @@ public sealed class AlarmWatchListResolver : IAlarmWatchListResolver
AlarmDiscoveryOptions discovery = options.Fallback.Discovery;
// GWC-37: the caller cannot grade an empty watch-list without knowing whether discovery
// The caller cannot grade an empty watch-list without knowing whether discovery
// was asked to contribute and whether it failed while doing so, because the failure is
// deliberately swallowed below.
string? discoveryFailureMessage = null;
@@ -41,7 +41,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
// every path that writes _alarms must null this under _sync, or readers keep a stale set.
private ActiveAlarmSnapshot[]? _currentAlarmsProjection;
// NEXT-03 dedup tombstones, guarded by _sync: alarm instances whose Clear was synthesized by
// Dedup tombstones, guarded by _sync: alarm instances whose Clear was synthesized by
// the most recent reconcile pass, keyed by reference with the instance's original raise
// timestamp as the identity marker. A buffered live Clear for the same instance is a duplicate
// of the repair and is suppressed. One generation deep: each reconcile pass replaces the map,
@@ -57,7 +57,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private string _providerReason = string.Empty;
private DateTimeOffset _providerSince = DateTimeOffset.UtcNow;
// GWC-37 subtag-coverage verdict, guarded by _sync: whether the watch-list the worker is
// Subtag-coverage verdict, guarded by _sync: whether the watch-list the worker is
// using can actually cover anything, and why not. Independent of _providerDegraded (which
// means "the subtag provider is the one serving"): in Auto mode the alarm manager can be
// serving perfectly while the standby behind it covers zero alarms, and that is invisible
@@ -66,9 +66,10 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
private bool _subtagCoverageMissing;
private string _subtagCoverageReason = string.Empty;
// The watch-list references actually sent to the worker in SubscribeAlarms, guarded by _sync.
// The worker holds that list for the life of the subscription, so a later re-resolution is
// compared against this to report drift rather than silently diverging.
// The full canonical watch-list fingerprint acknowledged by the worker and its alarm
// references, guarded by _sync. The fingerprint advances only after an OK subscribe/update
// reply, so a rejected replacement remains material and is retried on the next refresh edge.
private string _acknowledgedWatchListFingerprint = string.Empty;
private IReadOnlyCollection<string> _sentWatchListReferences = [];
// Whether the worker's most recent reconcile fetch was capped, guarded by _sync.
@@ -257,7 +258,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
try
{
// Attach the internal distributor subscriber BEFORE subscribing (GWC-26). The pump
// Attach the internal distributor subscriber BEFORE subscribing. The pump
// has been running since MarkReady started the dashboard mirror, and the distributor
// only fans to subscribers registered at the time of the fan-out, so a subscriber
// taken after SubscribeAlarms + the first reconcile would silently lose every
@@ -350,15 +351,12 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
AlarmProviderMode forcedMode = MapForcedMode(_options.Fallback.Mode);
// GWC-37: grade the coverage before the command goes out, so a Galaxy Repository outage at
// Grade the coverage before the command goes out, so a Galaxy Repository outage at
// startup is a standing status on the feed instead of one log line an operator reads weeks
// later — when the failover that needed the watch-list finds it empty. Broadcast because
// feed subscribers outlive the monitor's worker session and may already be attached.
lock (_sync)
{
_sentWatchListReferences = watchList
.Select(target => target.AlarmFullReference)
.ToHashSet(StringComparer.OrdinalIgnoreCase);
SetSubtagCoverage(GradeSubtagCoverage(forcedMode, resolution, staleAgainstSent: false), broadcast: true);
}
@@ -374,6 +372,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
{
SubscriptionExpression = subscription,
ForcedMode = forcedMode,
FallbackEnabled = CanUseSubtagFallback(forcedMode),
Failover = new AlarmFailoverConfig
{
ConsecutiveFailureThreshold = _options.Fallback.ConsecutiveFailureThreshold,
@@ -404,6 +403,11 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
?? $"status {code}";
throw new InvalidOperationException($"Worker rejected SubscribeAlarms: {diagnostic}");
}
lock (_sync)
{
AcknowledgeWatchList(watchList);
}
}
private async Task ReconcileLoopAsync(string sessionId, CancellationToken cancellationToken)
@@ -416,6 +420,8 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
{
try
{
await RefreshAlarmWatchListAsync(sessionId, broadcast: true, cancellationToken)
.ConfigureAwait(false);
await ReconcileAsync(sessionId, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException)
@@ -501,7 +507,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
}
}
// NEXT-03: best-effort dedup of the reconcile/live race. A reconcile that already synthesized
// Best-effort dedup of the reconcile/live race. A reconcile that already synthesized
// this transition as a feed repair left the cache carrying the worker's transition timestamp
// and resulting state — both derived from the same worker-side value the live transition
// carries — so an exact (timestamp, state) match means this live transition's outcome has
@@ -515,7 +521,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
&& existing.CurrentState == incoming.CurrentState;
}
// NEXT-03, the Clear leg. A reconcile Clear repair removes the cache entry before the buffered
// The Clear leg of the dedup. A reconcile Clear repair removes the cache entry before the buffered
// live Clear drains, so there is no cached state to compare against; the tombstone recorded by
// ApplyReconcile identifies the cleared instance by its original raise timestamp instead. The
// match consumes the tombstone, so a genuinely new raise/clear cycle (which carries a newer
@@ -545,18 +551,14 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
AlarmProviderMode toMode = change.Mode;
string reason = change.Reason ?? string.Empty;
// GWC-37: this is the moment the subtag watch-list stops being theoretical — the provider
// that is about to serve is the one built from it. Re-resolve BEFORE the status is
// published so the frame the operator sees on the failover carries the coverage verdict
// rather than a second frame later, and so a watch-list that has drifted since
// SubscribeAlarms is reported instead of silently diverging. The re-resolution is NOT
// re-sent: the worker rejects a second SubscribeAlarms while subscribed, and the only way
// to replace the list — Unsubscribe then Subscribe — rebuilds the FailoverAlarmConsumer,
// which restarts on the primary that just failed and discards the warm standby state, so
// it would flap the feed rather than repair it. A monitor restart re-resolves and re-sends.
// The failover edge is the moment the subtag watch-list stops being theoretical. Refresh
// it before publishing the status frame so recovered discovery is applied to the already
// active standby and the operator sees the acknowledged coverage verdict atomically with
// the provider switch.
if (toMode == AlarmProviderMode.Subtag)
{
await RefreshSubtagCoverageAsync(cancellationToken).ConfigureAwait(false);
await RefreshAlarmWatchListAsync(sessionId, broadcast: false, cancellationToken)
.ConfigureAwait(false);
}
AlarmProviderStatus status;
@@ -606,14 +608,17 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
}
}
// GWC-37: re-resolves the subtag watch-list and re-grades the coverage verdict without
// touching the worker's subscription. Best-effort: a resolver that throws (only cancellation
// and programming errors do — discovery failure is reported in the resolution) leaves the
// verdict from the last subscribe in place, because a stale verdict is better than none.
// Does not broadcast: the callers publish a status frame of their own immediately after.
private async Task RefreshSubtagCoverageAsync(CancellationToken cancellationToken)
private async Task RefreshAlarmWatchListAsync(
string sessionId,
bool broadcast,
CancellationToken cancellationToken)
{
AlarmProviderMode forcedMode = MapForcedMode(_options.Fallback.Mode);
if (!CanUseSubtagFallback(forcedMode))
{
return;
}
AlarmWatchListResolution resolution;
try
{
@@ -627,18 +632,96 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
{
_logger.LogWarning(
exception,
"Re-resolving the alarm subtag watch-list failed; keeping the coverage verdict from the last subscribe.");
"Re-resolving the alarm subtag watch-list failed; keeping the last acknowledged watch list.");
return;
}
string resolvedFingerprint = CanonicalWatchListFingerprint(resolution.Targets);
string acknowledgedFingerprint;
lock (_sync)
{
bool stale = !resolution.Targets
.Select(target => target.AlarmFullReference)
.ToHashSet(StringComparer.OrdinalIgnoreCase)
.SetEquals(_sentWatchListReferences);
SetSubtagCoverage(GradeSubtagCoverage(forcedMode, resolution, stale), broadcast: false);
acknowledgedFingerprint = _acknowledgedWatchListFingerprint;
}
if (string.Equals(resolvedFingerprint, acknowledgedFingerprint, StringComparison.Ordinal))
{
lock (_sync)
{
SetSubtagCoverage(
GradeSubtagCoverage(forcedMode, resolution, staleAgainstSent: false),
broadcast);
}
return;
}
UpdateAlarmWatchListCommand update = new();
update.WatchList.AddRange(resolution.Targets);
WorkerCommandReply reply = await _sessionManager.InvokeAsync(
sessionId,
new WorkerCommand
{
Command = new MxCommand
{
Kind = MxCommandKind.UpdateAlarmWatchList,
UpdateAlarmWatchList = update,
},
},
cancellationToken)
.ConfigureAwait(false);
ProtocolStatusCode? code = reply.Reply?.ProtocolStatus?.Code;
lock (_sync)
{
if (code == ProtocolStatusCode.Ok)
{
AcknowledgeWatchList(resolution.Targets);
SetSubtagCoverage(
GradeSubtagCoverage(forcedMode, resolution, staleAgainstSent: false),
broadcast);
return;
}
string diagnostic = reply.Reply?.DiagnosticMessage
?? reply.Reply?.ProtocolStatus?.Message
?? $"status {code}";
string reason = GradeSubtagCoverage(forcedMode, resolution, staleAgainstSent: true);
SetSubtagCoverage($"{reason}; worker rejected watch-list update ({diagnostic})", broadcast);
}
}
private bool CanUseSubtagFallback(AlarmProviderMode forcedMode) =>
forcedMode != AlarmProviderMode.Alarmmgr
&& (forcedMode == AlarmProviderMode.Subtag
|| _options.Fallback.Discovery.UseGalaxyRepository
|| _options.Fallback.Discovery.IncludeAttributes.Length > 0);
// Caller holds _sync.
private void AcknowledgeWatchList(IReadOnlyList<AlarmSubtagTarget> watchList)
{
_acknowledgedWatchListFingerprint = CanonicalWatchListFingerprint(watchList);
_sentWatchListReferences = watchList
.Select(target => target.AlarmFullReference)
.ToHashSet(StringComparer.OrdinalIgnoreCase);
}
private static string CanonicalWatchListFingerprint(IEnumerable<AlarmSubtagTarget> watchList) =>
string.Join(
"\n",
watchList
.Select(target => string.Join(
"|",
FingerprintPart(target.AlarmFullReference),
FingerprintPart(target.SourceObjectReference),
FingerprintPart(target.ActiveSubtag),
FingerprintPart(target.AckedSubtag),
FingerprintPart(target.AckCommentSubtag),
FingerprintPart(target.PrioritySubtag)))
.Order(StringComparer.Ordinal));
private static string FingerprintPart(string? value)
{
string normalized = value ?? string.Empty;
return $"{normalized.Length}:{normalized}";
}
// Grades one resolution against the effective mode. Returns the reason the subtag coverage is
@@ -684,8 +767,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
{
reasons.Add(
$"{AlarmProviderReasons.SubtagWatchListStale} ({resolution.Targets.Count} target(s) now "
+ $"resolve against {_sentWatchListReferences.Count} sent at subscribe); restart the alarm "
+ "monitor to re-send it");
+ $"resolve against {_sentWatchListReferences.Count} acknowledged by the worker)");
}
return string.Join("; ", reasons);
@@ -721,7 +803,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
// Caller holds _sync. Builds an AlarmProviderStatus snapshot of the current state.
//
// GWC-37: degraded is the OR of "the subtag provider is the one serving" and "the subtag
// Degraded is the OR of "the subtag provider is the one serving" and "the subtag
// watch-list cannot cover anything". The second is what makes an empty watch-list visible in
// Auto mode, where the alarm manager is serving happily and the standby behind it is empty.
// The reasons are concatenated when both apply, which is the worst case worth naming loudly:
@@ -775,7 +857,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
// Replaces the cache with the worker's authoritative snapshot, broadcasting
// a synthetic transition for any alarm the live stream missed.
//
// Repair scope (GWC-26): presence deltas (Clear/Raise) plus acked-state deltas. These are
// Repair scope: presence deltas (Clear/Raise) plus acked-state deltas. These are
// ALARM FEED transitions (AlarmFeedMessage on the StreamAlarms/dashboard surface), rebuilt
// from the worker's own authoritative snapshot to repair what the live feed missed. They are
// not MxEvents and never reach the gRPC StreamEvents path, so this feed-level repair does not
@@ -785,7 +867,7 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
// reads the worker's current state while the corresponding live transition may still be
// buffered in the alarm lease's channel; both would then broadcast, and the two are
// indistinguishable on the feed, since nothing serializes a reconcile against the in-flight
// live stream. ApplyTransition narrows that window with a best-effort dedup (NEXT-03): a live
// live stream. ApplyTransition narrows that window with a best-effort dedup: a live
// transition whose worker timestamp and resulting state the cache already carries — or whose
// Clear matches a tombstone recorded below — was already broadcast as a repair and is
// suppressed. The dedup fires only on a positive marker match, so the contract stays
@@ -839,15 +921,17 @@ public sealed class GatewayAlarmMonitor : BackgroundService, IGatewayAlarmServic
new AlarmFeedMessage { Transition = TransitionFromSnapshot(incoming.Value, AlarmTransitionKind.Raise) },
incoming.Key);
}
else if (existing.CurrentState != incoming.Value.CurrentState
&& incoming.Value.CurrentState == AlarmConditionState.ActiveAcked)
else if (existing.CurrentState != incoming.Value.CurrentState)
{
// The alarm was already known but the worker now reports it acknowledged: the
// live Acknowledge transition never reached the feed. Without this the acked
// state is absorbed silently by the snapshot replace below and subscribers show
// the alarm unacked until it clears.
// The alarm was already known but the worker now reports a different state. An
// ActiveAcked snapshot repairs a missed Acknowledge. Any other incoming active
// state repairs a missed Raise for the incoming instance (including an
// acknowledged alarm reference that was cleared and re-raised between passes).
AlarmTransitionKind kind = incoming.Value.CurrentState == AlarmConditionState.ActiveAcked
? AlarmTransitionKind.Acknowledge
: AlarmTransitionKind.Raise;
Broadcast(
new AlarmFeedMessage { Transition = TransitionFromSnapshot(incoming.Value, AlarmTransitionKind.Acknowledge) },
new AlarmFeedMessage { Transition = TransitionFromSnapshot(incoming.Value, kind) },
incoming.Key);
}
}
@@ -8,7 +8,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Configuration;
/// <c>ZB.MOM.WW.GalaxyRepository</c> package binds the options but deliberately ships no validator
/// (see <c>A2-galaxyrepository-adoption-handoff.md</c>); the gateway owns the rule because it is the
/// process that writes the snapshot. When persistence is on, the snapshot path must be a valid,
/// rooted path on the running host for the same reason the auth DB path must be (SEC-33): a
/// rooted path on the running host for the same reason the auth DB path must be: a
/// non-rooted value silently resolves against the launch working directory.
/// </summary>
public sealed class GalaxyRepositoryOptionsValidator : OptionsValidatorBase<GalaxyRepositoryOptions>
@@ -18,7 +18,7 @@ internal static class GatewayConfigPathRules
/// against the launch working directory, so the store moves with the CWD and can leak into the
/// source tree. Rooting is checked with <see cref="Path.IsPathRooted(string)"/> — the current
/// OS — so a Windows drive/UNC literal on a Unix host fails fast at startup rather than being
/// blessed and then written as a junk-named relative file (the SEC-01/SEC-33 mechanism). Reject
/// blessed and then written as a junk-named relative file. Reject
/// rather than auto-root; silent relocation of a credential store is worse than a boot error.
/// Blank is handled by the caller's required-field check and is not treated as non-rooted here.
/// </summary>
@@ -113,7 +113,7 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
"MxGateway:Security:AuditRetentionDays must be greater than zero (at least one day of audit history is retained).",
builder);
// The two-layer limiter knobs (SEC-31) accept 0 as "disable this layer": a zero aggregate
// The two-layer limiter knobs accept 0 as "disable this layer": a zero aggregate
// limit turns off cross-peer counting, and a zero probe interval restores absolute blocking.
// Negatives express no intent.
AddIfNegative(
@@ -341,7 +341,7 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
}
// QueueCapacity flows into WorkerClient as EventChannelCapacity, where the staging channel is
// sized checked(2 * EventChannelCapacity) (GWC-24). Cap it at int.MaxValue/2 so that doubling
// sized checked(2 * EventChannelCapacity). Cap it at int.MaxValue/2 so that doubling
// cannot overflow and throw OverflowException at session creation; mirrors the MaxSparseArrayLength
// upper-bound pattern.
private const int MaximumEventQueueCapacity = int.MaxValue / 2;
@@ -607,7 +607,7 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
// at startup rather than mid-traffic. Only checked when both knobs are themselves in range so the
// message is not doubled up with the individual range errors.
//
// IPC-33: the coupling also has an upper half. The worker budgets a DrainEvents reply against the
// The coupling also has an upper half. The worker budgets a DrainEvents reply against the
// negotiated frame maximum less two reserves, and that same MxCommandReply is then returned
// verbatim through Invoke, where the gateway's gRPC MaxSendMessageSize (= MaxGrpcMessageBytes)
// applies. Letting the pipe maximum run arbitrarily far above the gRPC cap lets the worker pack a
@@ -671,7 +671,7 @@ public sealed class GatewayOptionsValidator : OptionsValidatorBase<GatewayOption
// Path rooting/validity rules are shared with GalaxyRepositoryOptionsValidator (both write a
// host file) so the two validators cannot drift; see GatewayConfigPathRules. Rooting is checked
// against the running OS via Path.IsPathRooted — a Windows drive/UNC literal on a Unix host now
// fails fast instead of being blessed and written as a junk-named relative file (SEC-33).
// fails fast instead of being blessed and written as a junk-named relative file.
private static void AddIfNotRooted(string? value, string message, ValidationBuilder builder)
=> GatewayConfigPathRules.AddIfNotRooted(value, message, builder);
@@ -61,7 +61,7 @@ public sealed class WorkerOptions
/// still fits one worker frame, and at most <em>twice</em> that reserve above it so a
/// <c>DrainEvents</c> reply budgeted against this maximum (frame max less two reserves) still
/// fits the gRPC send cap it is returned through — otherwise the worker can pack a reply the
/// pipe carries and the gRPC layer refuses, losing the already-drained events (IPC-33). The
/// pipe carries and the gRPC layer refuses, losing the already-drained events. The
/// gateway conveys this value to the worker in the handshake
/// (<c>GatewayHello.max_frame_bytes</c>). Default is the 16 MB public gRPC cap plus one reserve.
/// </summary>
@@ -5,7 +5,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// <summary>
/// Handler for <see cref="AuthenticatedDashboardRoleRequirement"/>: succeeds only when the caller
/// is authenticated AND holds one of the requirement's roles. It reads no options at all, which
/// is the point — there is no configuration value that can turn this into a bypass (SEC-40).
/// is the point — there is no configuration value that can turn this into a bypass.
/// </summary>
public sealed class AuthenticatedDashboardRoleHandler
: AuthorizationHandler<AuthenticatedDashboardRoleRequirement>
@@ -13,9 +13,9 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// into this one by accident.
/// </summary>
/// <remarks>
/// Introduced for SEC-40 to gate the alarm feed (<c>AlarmsHub</c> and the <c>/alarms</c> page),
/// Introduced to gate the alarm feed (<c>AlarmsHub</c> and the <c>/alarms</c> page),
/// which is gateway-central and session-less and therefore carries no per-session provenance the
/// SEC-25 tag ACL could scope it by. Requiring a real dashboard role claim is the narrowing that
/// tag ACL could scope it by. Requiring a real dashboard role claim is the narrowing that
/// is actually available: it keeps galaxy-wide alarm metadata — alarm reference (a full tag path),
/// state, severity, category, area, acknowledging operator — away from the read-only environment
/// bypasses, which exist for local development and unauthenticated deployments rather than as a
@@ -32,7 +32,7 @@ public sealed record DashboardAlarmProviderStatus(
/// <summary>
/// Badge label shown when the alarm manager is serving normally but the subtag standby
/// behind it covers nothing — an empty or undiscoverable watch-list (GWC-37). The feed is
/// behind it covers nothing — an empty or undiscoverable watch-list. The feed is
/// healthy today; what is degraded is the fallback that would take over tomorrow, so the
/// badge names the standby rather than claiming the feed has failed over.
/// </summary>
@@ -77,7 +77,7 @@ public sealed record DashboardAlarmProviderStatus(
&& status.Mode == AlarmProviderMode.Subtag
&& string.Equals(reason, AlarmProviderReasons.ForcedSubtag, StringComparison.Ordinal);
// GWC-37: degraded WITHOUT subtag mode means the alarm manager is still serving and the
// Degraded WITHOUT subtag mode means the alarm manager is still serving and the
// standby is the unsound half. "Subtag monitoring (degraded)" would misreport which
// provider is live, so this state gets its own label; the amber badge is shared because
// the operator action — fix discovery — is equally overdue in both.
@@ -35,7 +35,7 @@ public static class DashboardAuthenticationDefaults
/// <summary>
/// Policy guarding the alarm feed — the <c>AlarmsHub</c> SignalR surface and the
/// <c>/alarms</c> page that reads the same central monitor in process (SEC-40). Accepts the
/// <c>/alarms</c> page that reads the same central monitor in process. Accepts the
/// same two schemes as <see cref="HubClientsPolicy"/> but carries
/// <see cref="AuthenticatedDashboardRoleRequirement"/>, a requirement type
/// <see cref="DashboardAuthorizationHandler"/> does not handle, so the environment bypasses
@@ -49,10 +49,10 @@ public static class DashboardAuthenticationDefaults
public const string KeyPrefixClaimType = "mxgateway:key_prefix";
/// <summary>
/// Claim carrying one dashboard event-visibility tag the caller is granted (SEC-25). Stamped
/// Claim carrying one dashboard event-visibility tag the caller is granted. Stamped
/// at cookie login by <see cref="DashboardAuthenticator"/> and at hub-token mint by
/// <see cref="HubTokenService"/>, both resolving the caller's LDAP groups through
/// <c>MxGateway:Dashboard:GroupToTag</c>. Since SEC-41 <see cref="IDashboardSessionAcl"/> reads
/// <c>MxGateway:Dashboard:GroupToTag</c>. <see cref="IDashboardSessionAcl"/> reads
/// this claim only as a FALLBACK, for a principal that carries no
/// <see cref="LdapGroupClaimType"/> claims to re-resolve from (the hub-token principal, whose
/// payload carries tags but not groups); a cookie principal is re-resolved against current
@@ -21,7 +21,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// <param name="roleMapper">Maps LDAP groups to dashboard roles.</param>
/// <param name="options">
/// Gateway options supplying <c>MxGateway:Dashboard:GroupToTag</c>, the map that turns the user's
/// LDAP groups into the dashboard visibility tags stamped on the cookie principal (SEC-25).
/// LDAP groups into the dashboard visibility tags stamped on the cookie principal.
/// </param>
/// <param name="logger">Logger for diagnostic, credential-free login outcomes.</param>
public sealed class DashboardAuthenticator(
@@ -142,7 +142,7 @@ public sealed class DashboardAuthenticator(
claims.AddRange(groupNames.Select(group => new Claim(
DashboardAuthenticationDefaults.LdapGroupClaimType,
group)));
// Dashboard event-visibility tags (SEC-25). Visibility only — never a data-access grant —
// Dashboard event-visibility tags. Visibility only — never a data-access grant —
// and never logged: only the decision, never the tag values, reaches diagnostics.
claims.AddRange(DashboardGroupTagMapping
.MapGroupsToTags(groupNames, groupToTag)
@@ -107,7 +107,7 @@ public static class DashboardEndpointRouteBuilderExtensions
// SignalR hubs. Authorization is enforced on the hub class via [Authorize], so both
// policies below accept either the dashboard cookie or a HubToken bearer. Snapshot and
// events use HubClientsPolicy; AlarmsHub uses AlarmsPolicy, which additionally requires a
// real authenticated dashboard role (SEC-40) — see AlarmsHub for why the alarm feed is
// real authenticated dashboard role — see AlarmsHub for why the alarm feed is
// gated harder than the session-scoped ones.
endpoints.MapHub<DashboardSnapshotHub>("/hubs/snapshot");
endpoints.MapHub<AlarmsHub>("/hubs/alarms");
@@ -177,7 +177,7 @@ public static class DashboardServiceCollectionExtensions
DashboardAuthenticationDefaults.AuthenticationScheme,
DashboardAuthenticationDefaults.HubAuthenticationScheme)
.AddRequirements(DashboardAuthorizationRequirement.AnyDashboardRole));
// SEC-40. Same two schemes as HubClientsPolicy (the alarm feed has an in-page reader
// Same two schemes as HubClientsPolicy (the alarm feed has an in-page reader
// and an external hub reader), but a requirement type DashboardAuthorizationHandler
// does not handle, so neither Authentication:Mode=Disabled nor anonymous localhost
// can satisfy it. Galaxy-wide alarm metadata therefore needs a real dashboard role.
@@ -26,7 +26,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// (ordinal-ignore-case).</description></item>
/// </list>
/// <para>
/// <strong>Where the grant comes from (SEC-41).</strong> The grant is re-resolved on every call
/// <strong>Where the grant comes from.</strong> The grant is re-resolved on every call
/// from the caller's <see cref="DashboardAuthenticationDefaults.LdapGroupClaimType"/> claims
/// through <c>MxGateway:Dashboard:GroupToTag</c> as it stands in the currently bound options —
/// the same single <see cref="DashboardGroupTagMapping.MapGroupsToTags"/> call
@@ -120,7 +120,7 @@ public sealed class DashboardSessionAcl(
if (groups is not null)
{
// SEC-41: current options, not the login-time snapshot. MapGroupsToTags returns an
// Current options, not the login-time snapshot. MapGroupsToTags returns an
// OrdinalIgnoreCase set, so probing it with the session's tags carries the
// case-insensitive comparison in the same direction the claim scan below does.
IReadOnlySet<string> granted = DashboardGroupTagMapping.MapGroupsToTags(
@@ -39,7 +39,7 @@ public sealed class HubTokenService
// revocable. A short lifetime bounds the exposure window of a token captured from a proxy
// or log after logout (the cookie is cleared on logout, but outstanding tokens are not), and
// bounds how long a stale role set survives a role change. It now bounds a stale *tag* grant
// the same way (SEC-25): the token carries the tags resolved from the caller's LDAP groups at
// the same way: the token carries the tags resolved from the caller's LDAP groups at
// mint time, so revoking a GroupToTag entry takes effect for token-authenticated hub
// connections within one lifetime — the natural place the deferred "tokens gain session
// binding" note landed. Five minutes is transparent to clients that re-fetch from /hubs/token
@@ -11,9 +11,9 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard.Hubs;
/// </summary>
/// <remarks>
/// The feed is galaxy-wide: the alarm monitor is gateway-central and its messages carry no
/// session provenance, so the SEC-25 per-session tag ACL that scopes <c>EventsHub</c> has nothing
/// session provenance, so the per-session tag ACL that scopes <c>EventsHub</c> has nothing
/// to intersect here and is deliberately not applied — every authenticated Viewer sees the whole
/// subscription expression's alarm metadata. What SEC-40 tightened instead is the entry gate:
/// subscription expression's alarm metadata. What is tightened instead is the entry gate:
/// <see cref="DashboardAuthenticationDefaults.AlarmsPolicy"/> rather than the
/// <see cref="DashboardAuthenticationDefaults.HubClientsPolicy"/> the other hubs use, so the
/// read-only environment bypasses (<c>Authentication:Mode=Disabled</c>, anonymous localhost) no
@@ -23,7 +23,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard.Hubs;
/// <c>StreamAlarms</c> subscribers and the alarms page as well, and none of
/// those audiences is subject to this dashboard-display flag.
/// <para>
/// The broadcast stays deliberately unfiltered (SEC-40). Audience is decided once, at hub
/// The broadcast stays deliberately unfiltered. Audience is decided once, at hub
/// admission — <see cref="AlarmsHub"/> carries
/// <see cref="DashboardAuthenticationDefaults.AlarmsPolicy"/>, so only an authenticated caller
/// with a real dashboard role is ever in <see cref="AlarmsHub.AllAlarmsGroup"/> — and the alarm
@@ -39,8 +39,8 @@ public sealed class EventsHub(
/// The hub-level <see cref="AuthorizeAttribute"/> (<c>HubClientsPolicy</c>)
/// only checks that the caller carries one of the dashboard roles, which by
/// itself would let any Viewer subscribe to any session id they name. The
/// per-session decision is <see cref="IDashboardSessionAcl"/>'s
/// (SEC-25 / TST-15). The admin bypass is evaluated first, so an
/// per-session decision is <see cref="IDashboardSessionAcl"/>'s.
/// The admin bypass is evaluated first, so an
/// Administrator joins any session id they name; every check below it
/// applies to non-Admin callers only. For those: a Viewer sees a session
/// only when its tags intersect their granted tags, an untagged session
@@ -95,16 +95,12 @@ public sealed class EventsHub(
viewerRegistry.RemoveViewer(Context.ConnectionId, sessionId);
}
/// <summary>
/// Releases every session subscription the dropped connection held. A browser
/// tab that closes never calls <see cref="UnsubscribeSession"/>, so without
/// this the session would look watched forever and the mirror would keep
/// cloning and sending events to an empty group.
/// </summary>
/// <param name="exception">The exception that terminated the connection, if any.</param>
/// <returns>A task representing the disconnect handling.</returns>
/// <inheritdoc />
public override Task OnDisconnectedAsync(Exception? exception)
{
// Release every session subscription the dropped connection held. A browser tab that
// closes never calls UnsubscribeSession, so without this the session would look watched
// forever and the mirror would keep cloning and sending events to an empty group.
viewerRegistry.ReleaseConnection(Context.ConnectionId);
return base.OnDisconnectedAsync(exception);
@@ -4,7 +4,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Dashboard;
/// <summary>
/// Decides whether a dashboard principal may observe one session's mirrored
/// event stream (SEC-25 / TST-15). Consulted at every subscribe seam: the
/// event stream. Consulted at every subscribe seam: the
/// SignalR <c>EventsHub.SubscribeSession</c> join and the in-process
/// <c>IDashboardSessionEventSubscriber.Subscribe</c> used by the
/// session-details page.
@@ -154,8 +154,8 @@ public static class GatewayApplication
// having pushed the ambient identity before the library's galaxy service runs. That
// ordering is still required, but it is no longer trusted silently: with
// Authentication:Mode=ApiKey the provider denies with PermissionDenied and logs the broken
// invariant when the identity is absent, instead of resolving to the full hierarchy
// (GWC-38). Under Mode=Disabled identity-absent is expected and stays unscoped.
// invariant when the identity is absent, instead of resolving to the full hierarchy.
// Under Mode=Disabled identity-absent is expected and stays unscoped.
builder.Services.AddSingleton<ZB.MOM.WW.GalaxyRepository.Grpc.IGalaxyBrowseScopeProvider,
Security.Authorization.GatewayBrowseScopeProvider>();
@@ -164,7 +164,7 @@ public static class GatewayApplication
// snapshot. GalaxyRepositoryOptions.SnapshotCachePath is init-only, so the default cannot be
// applied via PostConfigure — supply it as a configuration value (before the bind) when the
// shipped config leaves it blank. It resolves to the per-OS CommonApplicationData location,
// byte-identical to the removed appsettings literal on Windows (SEC-33).
// byte-identical to the removed appsettings literal on Windows.
if (string.IsNullOrWhiteSpace(builder.Configuration["MxGateway:Galaxy:SnapshotCachePath"]))
{
builder.Configuration["MxGateway:Galaxy:SnapshotCachePath"] = Path.Combine(
@@ -175,7 +175,7 @@ public static class GatewayApplication
builder.Services.AddZbGalaxyRepository(builder.Configuration, "MxGateway:Galaxy");
// Validate that persistence has a valid, host-rooted snapshot path (SEC-33).
// Validate that persistence has a valid, host-rooted snapshot path.
builder.Services.AddSingleton<
Microsoft.Extensions.Options.IValidateOptions<ZB.MOM.WW.GalaxyRepository.GalaxyRepositoryOptions>,
Configuration.GalaxyRepositoryOptionsValidator>();
@@ -224,7 +224,7 @@ public static class GatewayApplication
/// </para>
/// <para>
/// The location mirrors <c>AuthenticationOptions.SqlitePath</c> so both gateway stores sit
/// together, and the mechanism is the one SEC-33 already used for
/// together, and the mechanism is the same mechanism already used for
/// <c>MxGateway:Galaxy:SnapshotCachePath</c> below — same problem, same fix, same file. It also
/// matches what <c>docs/GatewayConfiguration.md</c> already tells operators to
/// pass to the <c>secret</c> CLI — an absolute default also removes the CLI/gateway divergence
@@ -116,7 +116,7 @@ public sealed class EventStreamService(
// its `yield return` is what feeds the gRPC writer.
ChannelReader<MxEvent> reader = subscriber.Reader;
// GWC-15: register this subscriber's channel as a live backlog source instead of
// Register this subscriber's channel as a live backlog source instead of
// reconciling the queue-depth gauge on every event. The gauge previously read the
// bounded channel's Count (which takes the channel's internal lock) and adjusted the
// metric under its own lock on every streamed event. Now the metric reads Count only
@@ -197,7 +197,7 @@ public sealed class EventStreamService(
}
// The queue-depth gauge is maintained lazily via the backlog registration
// above (GWC-15): the metric reads this subscriber's channel Count only when
// above: the metric reads this subscriber's channel Count only when
// scraped, so there is no per-event gauge bookkeeping on this hot path.
yield return mxEvent;
}
@@ -35,7 +35,7 @@ public sealed class MxAccessGatewayService(
// The session's owner id and its dashboard-visibility tags both come from the resolved
// API key identity, never from the request: the key is the tenant principal, so a
// client cannot label its own session with another tenant's tag (SEC-25).
// client cannot label its own session with another tenant's tag.
ApiKeyIdentity? owner = identityAccessor.Current;
GatewaySession session = await sessionManager
.OpenSessionAsync(
@@ -130,7 +130,7 @@ public sealed class MxAccessGatewayService(
// Map from the command alone: cloning the whole request only to overwrite its command with
// commandToInvoke deep-cloned the (potentially large) original payload for nothing, since
// MapCommand reads nothing but the command (GWC-29). The one clone that matters still
// MapCommand reads nothing but the command. The one clone that matters still
// happens inside MapCommand, which is what keeps the worker-bound graph unaliased from
// commandToInvoke — the caller still reads it below via TrackCommandReply.
WorkerCommand workerCommand = mapper.MapCommand(commandToInvoke);
@@ -167,7 +167,7 @@ public sealed class MxAccessGatewayService(
.WithCancellation(context.CancellationToken)
.ConfigureAwait(false))
{
// GWC-06: measure send latency with the allocation-free timestamp API rather
// Measure send latency with the allocation-free timestamp API rather
// than allocating a Stopwatch object per event per subscriber on the highest-
// volume gateway path. Stopwatch.GetTimestamp/GetElapsedTime measure the exact
// same wall-clock span the former Stopwatch.StartNew()/.Elapsed did.
@@ -185,15 +185,6 @@ public sealed class MxAccessGatewayService(
}
/// <inheritdoc />
/// <remarks>
/// Surfaces the public AcknowledgeAlarm RPC. Acknowledgement is
/// session-less: the gateway routes it through the always-on
/// <see cref="IGatewayAlarmService"/> monitor session. An
/// <c>alarm_full_reference</c> that parses as a canonical GUID forwards
/// to <c>AcknowledgeAlarmCommand</c>; a <c>Provider!Group.Tag</c>
/// reference forwards to <c>AcknowledgeAlarmByNameCommand</c>; anything
/// else returns an <c>InvalidRequest</c> diagnostic in the reply.
/// </remarks>
public override async Task<AcknowledgeAlarmReply> AcknowledgeAlarm(
AcknowledgeAlarmRequest request,
ServerCallContext context)
@@ -35,7 +35,7 @@ public sealed class MxAccessGrpcMapper
/// <summary>
/// Maps a gRPC MX command to a worker command. Callers that already hold the command — including
/// the constraint pipeline, whose rewritten command is not the one on the request — use this
/// overload rather than cloning a whole request to carry a single field (GWC-29); nothing outside
/// overload rather than cloning a whole request to carry a single field; nothing outside
/// the command is read here.
/// </summary>
/// <param name="command">Command payload.</param>
@@ -71,7 +71,7 @@ public sealed class MxAccessGrpcMapper
};
}
// GWC-07 / IPC-05: ownership transfer, not a deep clone — the same rule MapEvent follows,
// ownership transfer, not a deep clone — the same rule MapEvent follows,
// applied to the other (and larger, on bulk reads) hot-path message. The enclosing
// WorkerCommandReply is parsed fresh from a single pipe frame in WorkerClient's read loop
// and is single-consumer by construction: CompleteCommand's TryRemove hands it to exactly
@@ -93,10 +93,10 @@ public sealed class MxAccessGrpcMapper
{
ArgumentNullException.ThrowIfNull(workerEvent);
// GWC-07 / IPC-05: ownership transfer, not a deep clone. The enclosing WorkerEvent is
// ownership transfer, not a deep clone. The enclosing WorkerEvent is
// parsed fresh from a single pipe frame in WorkerClient's read loop and is discarded
// immediately after this mapping — the SessionEventDistributor pump is its single
// consumer (GWC-01 claims the worker event channel as single-reader), so nothing else
// consumer (it claims the worker event channel as single-reader), so nothing else
// aliases or mutates workerEvent.Event. We therefore move the inner MxEvent into the
// outbound graph instead of cloning it. Downstream the pump fans this one MxEvent to
// every subscriber and retains it in the replay ring, but that sharing is READ-ONLY
@@ -35,7 +35,7 @@ public sealed class GatewayMetrics : IDisposable
private readonly ConcurrentDictionary<string, long> _eventsBySession = new(StringComparer.Ordinal);
private readonly Dictionary<string, long> _retryAttemptsByArea = new(StringComparer.OrdinalIgnoreCase);
// GWC-15: live backlog sources for the gRPC event-stream queue-depth gauge. Each active
// live backlog sources for the gRPC event-stream queue-depth gauge. Each active
// StreamEvents subscriber registers a delegate returning its channel's current backlog; the
// gauge sums them ONLY when scraped or projected (GetGrpcEventStreamQueueDepth), so nothing is
// read on the per-event streaming hot path. Concurrent because subscribers register/unregister
@@ -43,7 +43,7 @@ public sealed class GatewayMetrics : IDisposable
private readonly ConcurrentDictionary<long, Func<int>> _eventStreamBacklogSources = new();
private long _nextEventStreamBacklogSourceId;
// PERF-02: the same pull model for the worker event queue depth. It replaces a pushed scalar that
// the same pull model for the worker event queue depth. It replaces a pushed scalar that
// every WorkerClient wrote twice per event (staged, consumed) under _syncRoot — a process-wide
// lock on the hottest path, and last-writer-wins across sessions, so the gauge reported one
// arbitrary session's backlog instead of the gateway's. Each client registers a source returning
@@ -210,7 +210,7 @@ public sealed class GatewayMetrics : IDisposable
/// <param name="method">Name of the command method.</param>
public void CommandStarted(string method)
{
// PERF-02: the three command counters run two-to-three times per gRPC call, so they use
// the three command counters run two-to-three times per gRPC call, so they use
// Interlocked rather than _syncRoot — the same idiom as EventReceived. Nothing here needs a
// consistent multi-field view; GetSnapshot reads each with Interlocked.Read.
Interlocked.Increment(ref _commandsStarted);
@@ -281,7 +281,7 @@ public sealed class GatewayMetrics : IDisposable
/// Registers a live depth source for the worker event queue-depth gauge and returns a handle
/// that removes it when disposed. Each <c>WorkerClient</c> registers once and reports its own
/// undelivered (staged + queued) event count, so the gauge is the gateway-wide sum instead of
/// the last value any one session happened to push (PERF-02).
/// the last value any one session happened to push.
/// </summary>
/// <param name="depth">
/// Returns this worker client's current undelivered event count. Invoked only at collection
@@ -303,7 +303,7 @@ public sealed class GatewayMetrics : IDisposable
/// returns a handle that removes it when disposed. The gauge sums every registered
/// source's current value on demand (when scraped or via <see cref="GetSnapshot"/>),
/// rather than tracking a pushed running total, so the streaming hot path does no
/// per-event gauge bookkeeping (GWC-15).
/// per-event gauge bookkeeping.
/// </summary>
/// <param name="backlog">
/// Returns this subscriber's current channel backlog. Invoked only at collection time;
@@ -351,7 +351,7 @@ public sealed class GatewayMetrics : IDisposable
{
// Tagged by stage only. Neither the key id nor the peer address may become a tag: key ids
// are credential material and peer addresses are unbounded cardinality, and /metrics is not
// authenticated (open SEC-14), so anything tagged here is world-readable.
// authenticated, so anything tagged here is world-readable.
_authThrottledCounter.Add(1, new KeyValuePair<string, object?>("stage", stage));
}
@@ -460,7 +460,7 @@ public sealed class GatewayMetrics : IDisposable
{
// Compute the live queue depths outside _syncRoot: the sources are the subscriber channels'
// Count (their own locks) and the worker clients' interlocked counters, neither of which may
// run under this lock. GWC-15, PERF-02.
// run under this lock.
int workerEventQueueDepth = GetWorkerEventQueueDepth();
int grpcEventStreamQueueDepth = GetGrpcEventStreamQueueDepth();
lock (_syncRoot)
@@ -521,7 +521,7 @@ public sealed class GatewayMetrics : IDisposable
}
}
// Sums the undelivered event backlog across every live worker client (PERF-02).
// Sums the undelivered event backlog across every live worker client.
private int GetWorkerEventQueueDepth() => SumSources(_workerEventQueueDepthSources);
// Sums the live backlog across every registered event-stream subscriber.
@@ -571,6 +571,7 @@ public sealed class GatewayMetrics : IDisposable
{
private int _disposed;
/// <summary>Removes the source from its gauge; subsequent calls are no-ops.</summary>
public void Dispose()
{
if (Interlocked.Exchange(ref _disposed, 1) == 0)
@@ -39,12 +39,7 @@ public sealed class AuditDrainService(
/// <summary>Upper bound on how long shutdown waits for the remaining buffered events.</summary>
private static readonly TimeSpan ShutdownDrainCap = TimeSpan.FromSeconds(2);
/// <summary>
/// Bootstraps the audit table, runs one retention sweep, then attaches the drain so the
/// writer switches from synchronous write-through to enqueueing.
/// </summary>
/// <param name="cancellationToken">Token to observe for cancellation.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
/// <inheritdoc />
public override async Task StartAsync(CancellationToken cancellationToken)
{
try
@@ -64,12 +59,7 @@ public sealed class AuditDrainService(
await base.StartAsync(cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Detaches the drain (so late writes go straight to the sink) and gives the buffered
/// events a bounded window to reach the store.
/// </summary>
/// <param name="cancellationToken">Token to observe for cancellation.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
/// <inheritdoc />
public override async Task StopAsync(CancellationToken cancellationToken)
{
writer.DetachDrain();
@@ -87,11 +87,9 @@ public sealed class SqliteCanonicalAuditStore(
}
/// <inheritdoc />
/// <remarks>
/// One connection, one transaction and one prepared command for the whole batch: the drain
/// pays a single commit for up to <see cref="AuditDrainService.MaxBatchSize"/> events rather
/// than one round-trip per event.
/// </remarks>
// One connection, one transaction and one prepared command for the whole batch: the drain
// pays a single commit for up to AuditDrainService.MaxBatchSize events rather
// than one round-trip per event.
public async Task InsertBatchAsync(IReadOnlyList<AuditEvent> auditEvents, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(auditEvents);
@@ -148,17 +146,15 @@ public sealed class SqliteCanonicalAuditStore(
}
/// <inheritdoc />
/// <remarks>
/// The comparison goes through SQLite's <c>datetime()</c> rather than comparing the stored
/// ISO-8601 text directly. Text comparison is only correct while every row is UTC-normalized
/// ISO-8601 — which <see cref="AuditEvent.OccurredAtUtc"/> guarantees for rows written through
/// this store, but not for rows that entered the table any other way (a repair script, an
/// older schema, a future producer). On a mixed-format column a text comparison silently
/// deletes live audit: <c>2026-05-17T09:00:00-05:00</c> is two hours AFTER a
/// <c>2026-05-17T12:00:00+00:00</c> cutoff yet sorts before it. Comparing instants is correct
/// regardless of how the text got there, and anything <c>datetime()</c> cannot parse yields
/// NULL and is therefore never deleted — audit that cannot be dated is kept, not swept.
/// </remarks>
// The comparison goes through SQLite's datetime() rather than comparing the stored
// ISO-8601 text directly. Text comparison is only correct while every row is UTC-normalized
// ISO-8601 — which AuditEvent.OccurredAtUtc guarantees for rows written through
// this store, but not for rows that entered the table any other way (a repair script, an
// older schema, a future producer). On a mixed-format column a text comparison silently
// deletes live audit: 2026-05-17T09:00:00-05:00 is two hours AFTER a
// 2026-05-17T12:00:00+00:00 cutoff yet sorts before it. Comparing instants is correct
// regardless of how the text got there, and anything datetime() cannot parse yields
// NULL and is therefore never deleted — audit that cannot be dated is kept, not swept.
public async Task<int> DeleteOlderThanAsync(DateTimeOffset cutoffUtc, CancellationToken cancellationToken)
{
await using SqliteConnection connection =
@@ -13,7 +13,7 @@ public sealed record ApiKeyConstraints(
private readonly IReadOnlyList<string> _dashboardTags = Array.Empty<string>();
/// <summary>
/// Gets the dashboard event-visibility tags granted to this key (SEC-25).
/// Gets the dashboard event-visibility tags granted to this key.
/// </summary>
/// <remarks>
/// <para>
@@ -44,7 +44,7 @@ public interface IApiKeyCacheInvalidator
/// evicting so an in-flight verification that started under the old generation discards its own
/// repopulation (set-then-recheck in <c>VerifyAsync</c>) — making the "gateway-initiated mutations
/// take effect immediately" contract true even against a verify that was already in the inner library
/// when the revoke landed (SEC-34 window 3).
/// when the revoke landed.
/// </para>
/// <para>
/// The short TTL is the backstop for two remaining bounded-staleness windows. (1) Out-of-band
@@ -141,7 +141,7 @@ public sealed class CachingApiKeyVerifier : IApiKeyVerifier, IApiKeyCacheInvalid
}
// Caches a successful verification unless the key was invalidated while the inner verify was in
// flight (SEC-34 window 3, detected via the generation snapshot). The entry lifetime stays the
// flight (detected via the generation snapshot). The entry lifetime stays the
// TTL: the library verification identity carries no expiry, so the cache cannot cap at the key's
// ExpiresUtc — that window remains TTL-bounded and documented in the class remarks.
private void TryCacheSuccess(
@@ -202,11 +202,14 @@ public sealed class CachingApiKeyVerifier : IApiKeyVerifier, IApiKeyCacheInvalid
? generation
: 0;
// Parses the key id out of a "Bearer mxgw_<keyId>_<secret>" header without any store access —
// the same split the authorization interceptor does. Returns null for a header this cache cannot
// attribute to a key id (in which case the generation race-guard is simply not applied).
//
// Correctness of the SEC-34 generation guard rests on parts[1] being the FULL key id: the '_'
/// <summary>
/// Parses the key id out of a <c>Bearer mxgw_&lt;keyId&gt;_&lt;secret&gt;</c> header without any store access,
/// using the same split the authorization interceptor does.
/// </summary>
/// <param name="authorizationHeader">The raw authorization header value, or <see langword="null"/>.</param>
/// <returns>The full key id, or <see langword="null"/> for a header this cache cannot attribute to a key id (the generation race-guard is then not applied).</returns>
// Implementation note:
// Correctness of the generation guard rests on parts[1] being the FULL key id: the '_'
// split would truncate a key id that itself contained '_', silently disarming the guard for that
// key. This is safe because '_' is the token's field delimiter and both — and the only — key
// creation paths in the gateway forbid it: ApiKeyAdminCommandLineParser.IsValidKeyId and
@@ -19,7 +19,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Security.Authorization;
/// source-rotating sprayer that never trips any single partition.
/// </para>
/// <para>
/// An over-limit state is a valve, not a wall (SEC-31): at most one request per
/// An over-limit state is a valve, not a wall: at most one request per
/// <see cref="SecurityOptions.ApiKeyFailureProbeIntervalSeconds"/> is admitted through to the real
/// verifier, and a successful verification resets both layers. The reset path therefore stays
/// reachable while throttled — the earlier design keyed solely on the attacker-supplied key id and
@@ -28,8 +28,11 @@ namespace ZB.MOM.WW.MxGateway.Server.Security.Authorization;
/// <para>
/// The tracked partition set is a bounded LRU (<see cref="SecurityOptions.ApiKeyFailureTrackedPeers"/>).
/// Because only a validly shaped token mints a key-id partition and each transport peer may mint at
/// most <see cref="MaxKeyIdPartitionsPerPeer"/> of them (the overflow collapses onto that address's
/// fallback partition), the map bounds memory <em>and</em> resists a junk-token flush (SEC-32):
/// most <see cref="MaxKeyIdPartitionsPerPeer"/> of them (the peer-layer overflow collapses onto that
/// address's fallback partition), the map bounds memory <em>and</em> resists a junk-token flush.
/// A validly shaped presented key id still feeds its cross-peer aggregate when its peer
/// layer uses the fallback, so exhausting the per-peer cap cannot bypass distributed-spray
/// protection; the aggregate map remains bounded by the same capped eviction policy:
/// eviction prefers fully expired windows, never removes an over-limit partition below a transient
/// overshoot ceiling of twice the cap, and only then falls back to least-recently-active. Successful
/// peers are removed on reset, so in steady state the map holds only partitions with recent failures
@@ -155,9 +158,9 @@ public sealed class ApiKeyFailureLimiter
WindowState? peerState = _partitions.TryGetValue(partitionKey, out WindowState? tracked) ? tracked : null;
WindowState? aggregateState = null;
if (effectiveKeyId is not null
if (partition.KeyId is not null
&& _aggregateLimit > 0
&& _aggregates.TryGetValue(effectiveKeyId, out WindowState? aggregate))
&& _aggregates.TryGetValue(partition.KeyId, out WindowState? aggregate))
{
aggregateState = aggregate;
}
@@ -222,11 +225,13 @@ public sealed class ApiKeyFailureLimiter
(Peer: peer, KeyId: effectiveKeyId));
RecordInto(state, now, _limit);
// Only a key id that earned its own partition feeds the aggregate: an id squeezed out by the
// per-peer cap is spray, and letting it through would make aggregate cardinality unbounded.
if (effectiveKeyId is not null && _aggregateLimit > 0)
// The aggregate follows the validly shaped presented key id independently of the peer
// layer's storage choice. Otherwise an attacker could exhaust each peer's dedicated-key cap
// and route a victim id through the fallback to bypass cross-peer throttling. Aggregate
// cardinality is still bounded by EvictIfOverCapacity below.
if (partition.KeyId is not null && _aggregateLimit > 0)
{
WindowState aggregate = _aggregates.GetOrAdd(effectiveKeyId, static _ => new WindowState(null, null));
WindowState aggregate = _aggregates.GetOrAdd(partition.KeyId, static _ => new WindowState(null, null));
RecordInto(aggregate, now, _aggregateLimit);
}
@@ -261,7 +266,8 @@ public sealed class ApiKeyFailureLimiter
// contributed by other key ids — and by junk-shaped tokens — from the same address, so
// removing it would let one successful authentication wipe an in-progress spray. The shared
// bucket decays by window expiry instead; the caller still recovers through probe admission.
if (string.Equals(effectiveKeyId, partition.KeyId, StringComparison.Ordinal))
if (effectiveKeyId is not null
&& string.Equals(effectiveKeyId, partition.KeyId, StringComparison.Ordinal))
{
RemovePartition(partitionKey);
}
@@ -8,7 +8,7 @@ namespace ZB.MOM.WW.MxGateway.Server.Security.Authorization;
/// <summary>Scopes Galaxy browse results to the calling API key's BrowseSubtrees constraint.</summary>
/// <remarks>
/// Distinguishes two states the pre-GWC-38 code collapsed into one: "authenticated, no
/// Distinguishes two states the earlier code collapsed into one: "authenticated, no
/// BrowseSubtrees constraint" (legitimately unscoped — the common case) and "no ambient identity
/// at all" (the interceptor-ordering invariant below is broken). The second used to resolve to the
/// full hierarchy silently, which made a scoping bypass invisible; it now denies loudly.
@@ -140,12 +140,17 @@ public sealed class GatewayGrpcAuthorizationInterceptor(
// Token shape mxgw_<keyId>_<secret>: the key id is the second underscore-delimited segment (the
// secret may itself contain underscores, which does not affect the key id). The shape is checked
// before a key-id partition is minted so a spray of invented tokens cannot mint one tracked
// partition each and flush the limiter's bounded map (SEC-32). Anything that fails the check
// partition each and flush the limiter's bounded map. Anything that fails the check
// falls back to the sender's transport-peer partition.
//
// Internal rather than private so the parse rules can be pinned directly by test; the shape
// check is a security boundary (SEC-32) and is worth asserting without routing every case
// check is a security boundary and is worth asserting without routing every case
// through a full RPC.
/// <summary>
/// Extracts the API key id from a bearer authorization header.
/// </summary>
/// <param name="authorizationHeader">The raw <c>authorization</c> header value, or <see langword="null"/>.</param>
/// <returns>The key id when the header has the <c>mxgw_&lt;keyId&gt;_&lt;secret&gt;</c> shape; otherwise <see langword="null"/>.</returns>
internal static string? TryResolveKeyId(string? authorizationHeader)
{
if (string.IsNullOrWhiteSpace(authorizationHeader))
@@ -24,9 +24,11 @@ namespace ZB.MOM.WW.MxGateway.Server.Sessions;
/// replay batch.
/// </param>
/// <param name="OldestAvailableSequence">
/// The oldest worker sequence still retained and replayable; <c>0</c> when nothing is
/// retained. Populates the <c>ReplayGap.oldest_available_sequence</c> field. Meaningful
/// only when <paramref name="Gap"/> is <see langword="true"/>.
/// Populates the <c>ReplayGap.oldest_available_sequence</c> field. When
/// <paramref name="Gap"/> is <see langword="true"/>, this is the oldest worker sequence
/// still retained and replayable, or the highest observed sequence plus one when the ring
/// is empty. It is <c>0</c> only when <paramref name="Gap"/> is
/// <see langword="false"/>, where the value is not emitted.
/// </param>
/// <param name="LiveResumeSequence">
/// The worker sequence the live channel must resume strictly after: the highest replayed
@@ -39,7 +39,7 @@ public sealed class GatewaySession
private bool _everHadEventSubscriber;
private SessionEventDistributor? _eventDistributor;
private bool _eventDistributorStarted;
// GWC-34: set under _syncRoot when DisposeAsync nulls _eventDistributor, making the
// Set under _syncRoot when DisposeAsync nulls _eventDistributor, making the
// "one distributor per session, never reset" lifecycle symmetric at both ends. Without it
// EnsureDistributorCreated reads the null as "not created yet" and builds a REPLACEMENT
// distributor after teardown — one whose pump can never start (_eventDistributorStarted is a
@@ -163,7 +163,7 @@ public sealed class GatewaySession
/// is unit-testable.
/// </param>
/// <param name="ownerDashboardTags">
/// Dashboard event-visibility tags inherited from the owning API key (SEC-25). Copied into
/// Dashboard event-visibility tags inherited from the owning API key. Copied into
/// the immutable <see cref="Tags"/> set; <see langword="null"/> or empty means untagged.
/// The tags come from the owner key, never from the client's wire request, so a client
/// cannot label its own session with another tenant's tag.
@@ -265,8 +265,8 @@ public sealed class GatewaySession
public string? OwnerKeyId { get; }
/// <summary>
/// Gets the dashboard event-visibility tags this session inherited from its owning API key
/// (SEC-25). An empty set means untagged.
/// Gets the dashboard event-visibility tags this session inherited from its owning API key.
/// An empty set means untagged.
/// </summary>
/// <remarks>
/// Immutable for the session's life — assigned once at construction from the owner key's
@@ -541,7 +541,7 @@ public sealed class GatewaySession
// concurrent callers (e.g. MarkReady's dashboard mirror and a racing first
// AttachEventSubscriber) agree on a single distributor and a single start.
//
// GWC-34: after DisposeAsync the null distributor means "gone", not "not built yet", so this
// After DisposeAsync the null distributor means "gone", not "not built yet", so this
// throws instead of constructing a replacement nothing will ever pump. The attach paths already
// roll back and surface the exception, which is the same SessionNotReady the readiness gates
// raise — a caller that loses the race to teardown now fails immediately instead of hanging.
@@ -610,7 +610,7 @@ public sealed class GatewaySession
/// </exception>
public IEventSubscriberLease AttachInternalEventSubscriber()
{
// Readiness gate, mirroring AttachEventSubscriber (GWC-27). It must run BEFORE
// Readiness gate, mirroring AttachEventSubscriber. It must run BEFORE
// EnsureDistributorCreated: a premature attach would construct the distributor and start
// its pump against a not-yet-Ready worker, the pump source would throw SessionNotReady,
// PumpAsync would complete every subscriber with that error and latch _completed, and
@@ -694,7 +694,7 @@ public sealed class GatewaySession
// These three calls are OUTSIDE _syncRoot to avoid holding it across
// EnsureDistributorCreated's own lock and StartAsync's Task.Run.
//
// GWC-34: the state check above can be overtaken by a close+dispose before we get here, and
// The state check above can be overtaken by a close+dispose before we get here, and
// EnsureDistributorCreated now says so instead of building an orphan distributor. The
// dashboard mirror is a best-effort observability attachment on a session that is already
// gone, so it is abandoned here rather than propagated out of MarkReady — the mirror-started
@@ -1918,7 +1918,7 @@ public sealed class GatewaySession
{
distributor = _eventDistributor;
_eventDistributor = null;
// GWC-34: latch the null-out so EnsureDistributorCreated cannot read it as
// Latch the null-out so EnsureDistributorCreated cannot read it as
// "not created yet" and hand a racing attach a distributor that will never pump.
_eventDistributorDisposed = true;
}
@@ -19,7 +19,7 @@ public interface ISessionManager
/// <summary>
/// Opens a new gateway session, stamping the owning API key's dashboard event-visibility
/// tags onto it (SEC-25).
/// tags onto it.
/// </summary>
/// <param name="request">Request payload.</param>
/// <param name="clientIdentity">Client identity string.</param>
@@ -520,7 +520,7 @@ public sealed class SessionEventDistributor : IAsyncDisposable
{
gap = _anyEventSeen && afterSequence < _highestSequenceSeen;
// GWC-25: nothing is retained, but a gapped client still needs a usable resume
// Nothing is retained, but a gapped client still needs a usable resume
// anchor. The documented client formula is
// after_worker_sequence = oldest_available_sequence - 1, so reporting 0 here made
// an unsigned client compute ulong.MaxValue: the follow-up resume then replayed
@@ -532,7 +532,7 @@ public sealed class SessionManager : ISessionManager
// The short prefix and bare guid keep the pipe's Unix-domain-socket path
// (TMPDIR + "CoreFxPipe_" + name) inside the 104-byte sun_path limit on
// macOS, whose default per-user TMPDIR is ~49 chars; the gateway PID keeps
// the name collision-free across gateway restarts (NEXT-01).
// the name collision-free across gateway restarts.
string pipeName = $"mxgw-{Environment.ProcessId}-{sessionUid}";
string nonce = CreateNonce();
DateTimeOffset openedAt = _timeProvider.GetUtcNow();
@@ -28,7 +28,7 @@ public sealed class WorkerClient : IWorkerClient
// Staging hand-off between the read loop and the dedicated event writer. The read loop writes
// here with a non-blocking TryWrite so a full consumer channel (_events) can never stall the read
// loop behind an event — replies and heartbeats keep flowing. Bounded at 2 × EventChannelCapacity
// (GWC-24): an unbounded staging channel let a consumer that drains slower than the worker
// because an unbounded staging channel let a consumer that drains slower than the worker
// produces grow gateway memory without limit and without a fault, because each individual timed
// write into _events still completed inside EventChannelFullModeTimeout. A rejected TryWrite here
// is the sustained-slow-drain signal and faults the client immediately (ProtocolViolation), which
@@ -41,7 +41,7 @@ public sealed class WorkerClient : IWorkerClient
private readonly SemaphoreSlim _pendingCommandSlots;
private readonly CancellationTokenSource _stopCts = new();
// PERF-02: this client's contribution to the gateway-wide worker queue-depth gauge. Registered
// this client's contribution to the gateway-wide worker queue-depth gauge. Registered
// once here and read only when the gauge is scraped, so staging and consuming an event cost an
// Interlocked on _eventQueueDepth and nothing else — previously each of those two hot-path steps
// called into GatewayMetrics and took its process-wide lock. Null when metrics are disabled.
@@ -233,13 +233,13 @@ public sealed class WorkerClient : IWorkerClient
// size the caller controls; checking here keeps a MessageTooLarge in the write loop a
// genuine desync signal.
//
// PERF-07: this size cannot be handed to WorkerFrameWriter to spare its own
// this size cannot be handed to WorkerFrameWriter to spare its own
// CalculateSize. WriteLoopAsync stamps envelope.Sequence immediately before the write
// (GWC-28), and a non-zero varint field grows the encoding — so the number computed here
// and a non-zero varint field grows the encoding — so the number computed here
// is a lower bound on the frame the writer actually emits, never the frame length. Passing
// it as a knownSize would under-length the prefix and desync the worker's framing.
//
// IPC-39: being a lower bound is conservative for the knownSize question but the wrong
// being a lower bound is conservative for the knownSize question but the wrong
// direction for this pre-check, which must not admit an envelope the writer will then
// reject with a session-fatal MessageTooLarge. Compare against the frame maximum less the
// MaxSequenceEncodingBytes the stamp can add, so every envelope this check accepts still
@@ -257,7 +257,7 @@ public sealed class WorkerClient : IWorkerClient
await EnqueueAsync(commandEnvelope, cancellationToken).ConfigureAwait(false);
// PERF-07: one pooled timer instead of a linked CTS + Task.Delay + WhenAny per command.
// one pooled timer instead of a linked CTS + Task.Delay + WhenAny per command.
// Task.WaitAsync arms a TimerQueueTimer on the shared timer queue and cancels it when the
// reply lands, so the steady-state cost of a command that replies in time is a single
// continuation — the old shape allocated a linked CancellationTokenSource, its
@@ -354,7 +354,7 @@ public sealed class WorkerClient : IWorkerClient
{
await foreach (WorkerEvent workerEvent in _events.Reader.ReadAllAsync(cancellationToken).ConfigureAwait(false))
{
// No metrics call on the hot path: the gauge pulls _eventQueueDepth when scraped (PERF-02).
// No metrics call on the hot path: the gauge pulls _eventQueueDepth when scraped.
Interlocked.Decrement(ref _eventQueueDepth);
yield return workerEvent;
}
@@ -420,7 +420,7 @@ public sealed class WorkerClient : IWorkerClient
/// <returns>A task that represents the asynchronous operation.</returns>
public async ValueTask DisposeAsync()
{
// GWC-32: disposal is the third channel-completing path, and it must enter a terminal state
// disposal is the third channel-completing path, and it must enter a terminal state
// like the other two (MarkClosed, SetFaulted) BEFORE the channels are completed. Otherwise a
// client disposed while still Ready keeps reporting IsTerminalState() == false, and a read
// loop already inside DispatchEnvelope (the _stopCts.Cancel below only interrupts the NEXT
@@ -444,11 +444,10 @@ public sealed class WorkerClient : IWorkerClient
{
_state = WorkerClientState.Closed;
}
UnregisterEventQueueDepthSource();
}
// Drop out of the worker queue-depth gauge before teardown: whatever this client still holds
// is about to be discarded, and the sum must not keep counting a client that is going away.
_eventQueueDepthRegistration?.Dispose();
KillOwnedProcess("Dispose");
_stopCts.Cancel();
_outboundEnvelopes.Writer.TryComplete();
@@ -491,7 +490,7 @@ public sealed class WorkerClient : IWorkerClient
/// Worst-case number of bytes <see cref="WriteLoopAsync"/>'s sequence stamp adds to an
/// envelope that was sized before it: one tag byte plus a ten-byte varint, the maximum
/// encoding of a 64-bit protobuf field. The <c>CommandTooLarge</c> pre-check subtracts this
/// from the negotiated frame maximum (IPC-39) because it measures the envelope while
/// from the negotiated frame maximum because it measures the envelope while
/// <c>Sequence</c> is still 0 — an unset field costs nothing to encode, so the measured size
/// is a lower bound on the frame the writer emits, and without the allowance an envelope in
/// the last eleven bytes of the range would pass the pre-check and then fault the session on
@@ -506,12 +505,12 @@ public sealed class WorkerClient : IWorkerClient
{
await foreach (WorkerEnvelope envelope in _outboundEnvelopes.Reader.ReadAllAsync(_stopCts.Token).ConfigureAwait(false))
{
// GWC-28: stamp the sequence at the point of writing, not when the envelope is built.
// stamp the sequence at the point of writing, not when the envelope is built.
// Stamping at construction let two concurrent InvokeAsync callers take 1 and 2 and then
// enqueue in the order 2, 1 — non-monotonic on the wire, breaking gateway.md's
// "monotonic per sender" contract. This loop is the channel's single consumer
// (SingleReader = true), so wire order and stamp order are the same thing here and
// _nextSequence needs no interlocking. Mirrors the worker's WRK-04 fix.
// _nextSequence needs no interlocking. Mirrors the worker's equivalent fix.
envelope.Sequence = unchecked(++_nextSequence);
await _writer.WriteAsync(envelope, _stopCts.Token).ConfigureAwait(false);
}
@@ -679,11 +678,11 @@ public sealed class WorkerClient : IWorkerClient
/// faults regardless of how backed up the event path is. It returns <c>false</c> in two cases:
/// the staging channel has been completed by one of the three completing paths —
/// <see cref="MarkClosed"/>, <see cref="SetFaulted"/> or <see cref="DisposeAsync"/>, each of
/// which enters a terminal state before completing the channels (GWC-32), so the event is
/// which enters a terminal state before completing the channels, so the event is
/// safely dropped because the client is already terminal — or the channel is full at its
/// 2 × <see cref="WorkerClientOptions.EventChannelCapacity"/> bound. The latter means the
/// consumer has been draining slower than the worker produces for the whole time it took the
/// worker to emit that many further events, so the client is faulted immediately (GWC-24) —
/// worker to emit that many further events, so the client is faulted immediately —
/// <see cref="SetFaulted"/> is non-blocking, so the read loop still never awaits here. The
/// complementary full-stall case (a consumer that stops entirely) is caught earlier by the
/// timed write in <see cref="EnqueueWorkerEventAsync"/>.
@@ -754,7 +753,7 @@ public sealed class WorkerClient : IWorkerClient
/// the actionable fix (attach <c>StreamEvents</c> or raise
/// <c>MxGateway:Events:QueueCapacity</c>). This is the full-stall half of
/// the backpressure policy; a consumer that merely drains too slowly is
/// caught by the staging bound in <see cref="StageWorkerEvent"/> (GWC-24).
/// caught by the staging bound in <see cref="StageWorkerEvent"/>.
/// Queue depth is not counted here — the event was already counted when it
/// was staged, so moving it between the two channels changes nothing.
/// </summary>
@@ -852,7 +851,7 @@ public sealed class WorkerClient : IWorkerClient
/// <summary>
/// Fails the pending command for a caller whose own token fired — a client deadline shorter
/// than the gateway's command timeout, or a disconnect that cancelled the RPC's token — and
/// forwards a <c>WorkerCancel</c> for the correlation the caller has abandoned (IPC-34).
/// forwards a <c>WorkerCancel</c> for the correlation the caller has abandoned.
/// <para>
/// Caller cancellation is the common abandonment path, not the rare one, and it leaves the
/// worker in exactly the state the timeout arm's cancel exists for: the correlation is still
@@ -983,7 +982,7 @@ public sealed class WorkerClient : IWorkerClient
{
WorkerEnvelope envelope = await _reader.ReadAsync(cancellationToken).ConfigureAwait(false);
// IPC-35: a fault frame during the handshake is the worker reporting a structured startup
// a fault frame during the handshake is the worker reporting a structured startup
// failure — most often MXAccess COM creation (not installed, not registered, access denied)
// — not a protocol problem. Flattening it to ProtocolViolation discarded the category and
// diagnostic exactly where they matter most: first session open on a freshly deployed host,
@@ -1092,6 +1091,7 @@ public sealed class WorkerClient : IWorkerClient
}
_state = WorkerClientState.Closed;
UnregisterEventQueueDepthSource();
}
_stopCts.Cancel();
@@ -1126,6 +1126,7 @@ public sealed class WorkerClient : IWorkerClient
}
_state = WorkerClientState.Faulted;
UnregisterEventQueueDepthSource();
}
_stopCts.Cancel();
@@ -1139,6 +1140,16 @@ public sealed class WorkerClient : IWorkerClient
_logger.LogWarning(exception, "Worker client faulted for session {SessionId}: {Message}", SessionId, message);
}
/// <summary>
/// Removes this terminal client's abandoned backlog from the gateway-wide queue-depth
/// gauge. The registration handle is idempotent, so terminal transitions and later disposal
/// may all call this safely while the faulted client remains retained for diagnostics.
/// </summary>
private void UnregisterEventQueueDepthSource()
{
_eventQueueDepthRegistration?.Dispose();
}
private void KillOwnedProcess(string reason)
{
WorkerProcessHandle? processHandle = _connection.ProcessHandle;
@@ -1292,7 +1303,7 @@ public sealed class WorkerClient : IWorkerClient
string correlationId,
WorkerCommand command)
{
// IPC-05: no second clone. MxAccessGrpcMapper.MapCommand already deep-cloned the command
// no second clone. MxAccessGrpcMapper.MapCommand already deep-cloned the command
// out of the caller-owned gRPC MxCommandRequest, so this WorkerCommand is a fresh graph
// owned by the invoke pipeline and referenced by no other consumer. The envelope is built
// and owned entirely inside WorkerClient and the command is not touched again after this
@@ -1329,7 +1340,7 @@ public sealed class WorkerClient : IWorkerClient
{
// Sequence is deliberately left unset here: WriteLoopAsync stamps it immediately before the
// frame goes out, so the numbers are monotonic in wire order however the callers interleave
// between construction and enqueue (GWC-28, mirroring the worker's WRK-04 fix).
// between construction and enqueue (mirroring the worker's fix).
WorkerEnvelope envelope = new()
{
ProtocolVersion = _connection.FrameOptions.ProtocolVersion,
@@ -19,7 +19,7 @@ public sealed class WorkerFrameReader
private readonly WorkerFrameProtocolOptions _options;
private readonly Stream _stream;
// Reused across frames rather than allocated per read (GWC-30). Safe because ReadAsync is
// Reused across frames rather than allocated per read. Safe because ReadAsync is
// single-consumer by construction; the prefix is fully overwritten by every read.
private readonly byte[] _lengthPrefix = new byte[sizeof(uint)];
@@ -61,8 +61,8 @@ public sealed class WorkerFrameReader
}
// Rent the payload buffer from the shared pool rather than allocating a fresh byte[] per
// frame; large event frames near the cap would otherwise allocate an LOH buffer each time
// (IPC-14). ParseFrom copies whatever it needs into the parsed message, so the rented buffer
// frame; large event frames near the cap would otherwise allocate an LOH buffer each time.
// ParseFrom copies whatever it needs into the parsed message, so the rented buffer
// can be returned as soon as parsing completes without the envelope aliasing it. The rented
// buffer may be larger than requested, so the read and parse are bounded to length.
int length = checked((int)payloadLength);
@@ -57,7 +57,7 @@ public sealed class WorkerFrameWriter
// Serialize once into a single pooled buffer that carries the 4-byte little-endian length
// prefix followed by the payload, then issue one stream write. This avoids a second
// serialization pass (ToByteArray re-runs CalculateSize), a separate prefix array and its
// own write, and any per-frame heap allocation (GWC-08, IPC-13). The rented buffer may be
// own write, and any per-frame heap allocation. The rented buffer may be
// larger than requested, so only the first frameLength bytes are ever written.
int frameLength = sizeof(uint) + payloadLength;
byte[] frame = ArrayPool<byte>.Shared.Rent(frameLength);
@@ -2,6 +2,7 @@ using System.Diagnostics;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.MxGateway.Contracts;
using ZB.MOM.WW.MxGateway.Server.Configuration;
using ZB.MOM.WW.MxGateway.Server.Metrics;
using Polly;
@@ -15,41 +16,47 @@ namespace ZB.MOM.WW.MxGateway.Server.Workers;
public sealed class WorkerProcessLauncher : IWorkerProcessLauncher
{
/// <summary>Environment variable for worker nonce.</summary>
public const string WorkerNonceEnvironmentVariableName = "MXGATEWAY_WORKER_NONCE";
[Obsolete("Use WorkerLaunchEnvironment.WorkerNonce.")]
public const string WorkerNonceEnvironmentVariableName = WorkerLaunchEnvironment.WorkerNonce;
/// <summary>Environment variable for worker pipe connect attempt timeout.</summary>
[Obsolete("Use WorkerLaunchEnvironment.PipeConnectAttemptTimeoutMilliseconds.")]
public const string WorkerPipeConnectAttemptTimeoutEnvironmentVariableName =
"MXGATEWAY_WORKER_PIPE_CONNECT_ATTEMPT_TIMEOUT_MS";
WorkerLaunchEnvironment.PipeConnectAttemptTimeoutMilliseconds;
/// <summary>
/// Conveys MxGateway:Worker:WriteCompletionWaitMilliseconds to the worker:
/// the bounded wait for the OnWriteComplete callback on
/// WriteSecured/WriteSecured2 replies. 0 disables the wait.
/// the bounded wait for the OnWriteComplete callback on Write, Write2,
/// WriteSecured, and WriteSecured2 replies. 0 disables the wait.
/// </summary>
[Obsolete("Use WorkerLaunchEnvironment.WriteCompletionWaitMilliseconds.")]
public const string WorkerWriteCompletionWaitEnvironmentVariableName =
"MXGATEWAY_WORKER_WRITE_COMPLETION_WAIT_MS";
WorkerLaunchEnvironment.WriteCompletionWaitMilliseconds;
/// <summary>
/// Conveys MxGateway:Alarms:PollIntervalMilliseconds to the worker: the
/// cadence at which the worker's STA polls the AVEVA alarm consumer.
/// </summary>
[Obsolete("Use WorkerLaunchEnvironment.AlarmPollIntervalMilliseconds.")]
public const string WorkerAlarmPollIntervalEnvironmentVariableName =
"MXGATEWAY_ALARM_POLL_INTERVAL_MS";
WorkerLaunchEnvironment.AlarmPollIntervalMilliseconds;
/// <summary>
/// Conveys MxGateway:Alarms:MaxAlarmsPerFetch to the worker: the cap
/// passed to GetXmlCurrentAlarms2, which is also the record count at
/// which the worker treats a fetch as truncated.
/// </summary>
[Obsolete("Use WorkerLaunchEnvironment.MaxAlarmsPerFetch.")]
public const string WorkerMaxAlarmsPerFetchEnvironmentVariableName =
"MXGATEWAY_ALARM_MAX_ALARMS_PER_FETCH";
WorkerLaunchEnvironment.MaxAlarmsPerFetch;
/// <summary>
/// Conveys MxGateway:Worker:EventQueueCapacity to the worker: the capacity
/// of the outbound MXAccess event queue, whose overflow faults the session.
/// </summary>
[Obsolete("Use WorkerLaunchEnvironment.EventQueueCapacity.")]
public const string WorkerEventQueueCapacityEnvironmentVariableName =
"MXGATEWAY_EVENT_QUEUE_CAPACITY";
WorkerLaunchEnvironment.EventQueueCapacity;
private readonly IWorkerProcessFactory _processFactory;
private readonly IWorkerStartupProbe _startupProbe;
@@ -204,16 +211,16 @@ public sealed class WorkerProcessLauncher : IWorkerProcessLauncher
startInfo.ArgumentList.Add(argument);
}
startInfo.Environment[WorkerNonceEnvironmentVariableName] = request.Nonce;
startInfo.Environment[WorkerPipeConnectAttemptTimeoutEnvironmentVariableName] =
startInfo.Environment[WorkerLaunchEnvironment.WorkerNonce] = request.Nonce;
startInfo.Environment[WorkerLaunchEnvironment.PipeConnectAttemptTimeoutMilliseconds] =
_workerOptions.PipeConnectAttemptTimeoutMilliseconds.ToString(System.Globalization.CultureInfo.InvariantCulture);
startInfo.Environment[WorkerWriteCompletionWaitEnvironmentVariableName] =
startInfo.Environment[WorkerLaunchEnvironment.WriteCompletionWaitMilliseconds] =
_workerOptions.WriteCompletionWaitMilliseconds.ToString(System.Globalization.CultureInfo.InvariantCulture);
startInfo.Environment[WorkerEventQueueCapacityEnvironmentVariableName] =
startInfo.Environment[WorkerLaunchEnvironment.EventQueueCapacity] =
_workerOptions.EventQueueCapacity.ToString(System.Globalization.CultureInfo.InvariantCulture);
startInfo.Environment[WorkerAlarmPollIntervalEnvironmentVariableName] =
startInfo.Environment[WorkerLaunchEnvironment.AlarmPollIntervalMilliseconds] =
_alarmsOptions.PollIntervalMilliseconds.ToString(System.Globalization.CultureInfo.InvariantCulture);
startInfo.Environment[WorkerMaxAlarmsPerFetchEnvironmentVariableName] =
startInfo.Environment[WorkerLaunchEnvironment.MaxAlarmsPerFetch] =
_alarmsOptions.MaxAlarmsPerFetch.ToString(System.Globalization.CultureInfo.InvariantCulture);
commandLine = new WorkerProcessCommandLine(executablePath, arguments);
@@ -10,10 +10,10 @@
<ItemGroup>
<PackageReference Include="Grpc.AspNetCore" Version="2.76.0" />
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.2.1" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.2.1" />
<PackageReference Include="ZB.MOM.WW.Auth.ApiKeys" Version="0.2.1" />
<PackageReference Include="ZB.MOM.WW.Auth.AspNetCore" Version="0.2.1" />
<PackageReference Include="ZB.MOM.WW.Auth.Abstractions" Version="0.3.0" />
<PackageReference Include="ZB.MOM.WW.Auth.Ldap" Version="0.3.0" />
<PackageReference Include="ZB.MOM.WW.Auth.ApiKeys" Version="0.3.0" />
<PackageReference Include="ZB.MOM.WW.Auth.AspNetCore" Version="0.3.0" />
<PackageReference Include="ZB.MOM.WW.Audit" Version="0.1.0" />
<PackageReference Include="ZB.MOM.WW.Theme" Version="0.4.1" />
<PackageReference Include="ZB.MOM.WW.Configuration" Version="0.1.0" />
@@ -19,7 +19,7 @@ public sealed class AlarmWatchListResolverTests
/// <summary>
/// Resolves and projects to the target list. These cases assert composition rules, which
/// live in <see cref="AlarmWatchListResolution.Targets"/>; the discovery-provenance fields
/// the resolution also carries (GWC-37) are asserted by their own cases below.
/// the resolution also carries are asserted by their own cases below.
/// </summary>
/// <param name="resolver">The resolver under test.</param>
/// <param name="options">Alarm options to resolve against.</param>
@@ -259,7 +259,7 @@ public sealed class AlarmWatchListResolverTests
}
/// <summary>
/// GWC-37: swallowing the discovery failure is right for the resolver but leaves the
/// Swallowing the discovery failure is right for the resolver but leaves the
/// caller unable to tell "discovery produced nothing" from "nothing was configured", which
/// is why the empty watch-list could sit unnoticed for a monitor's whole life. The
/// resolution therefore reports whether discovery was attempted, whether it failed, and
@@ -284,7 +284,7 @@ public sealed class AlarmWatchListResolverTests
/// <summary>
/// The complement of the case above: discovery disabled is not discovery failed. An empty
/// watch-list here means the deployment did not ask for subtag coverage, which the monitor
/// must not report as a degraded standby (GWC-37).
/// must not report as a degraded standby.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
@@ -13,7 +13,7 @@ using ZB.MOM.WW.MxGateway.Tests.TestSupport;
namespace ZB.MOM.WW.MxGateway.Tests.Alarms;
/// <summary>
/// GWC-26 regression tests for the alarm monitor's startup ordering. Unlike the
/// Regression tests for the alarm monitor's startup ordering. Unlike the
/// sibling alarm-monitor tests, the session manager here hands the monitor a REAL
/// <see cref="GatewaySession"/> that is driven to Ready with a dashboard mirror, so the
/// distributor pump is already running when the monitor attaches — the production
@@ -156,7 +156,67 @@ public sealed class GatewayAlarmMonitorAttachOrderTests
}
/// <summary>
/// NEXT-03. A reconcile Raise repair applied while the matching live Raise is still
/// An acknowledged cached alarm followed by an active snapshot with a newer
/// original-raise timestamp is a new alarm instance, not an acknowledgement reversal. The
/// reconcile repair must broadcast a Raise carrying the incoming instance identity.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task ApplyReconcile_WhenAcknowledgedAlarmHasNewActiveInstance_BroadcastsRaise()
{
using GatewayMetrics metrics = new();
await using FakeSessionManager sessions = new();
using GatewayAlarmMonitor monitor = CreateMonitor(sessions, metrics);
using CancellationTokenSource cts = new();
await monitor.StartAsync(cts.Token);
await sessions.WaitForSubscribeStartAsync(WaitTimeout);
Timestamp oldRaise = Timestamp.FromDateTimeOffset(
new DateTimeOffset(2026, 8, 10, 13, 0, 0, TimeSpan.Zero));
sessions.SetReconcileSnapshot(SnapshotAt(AlarmConditionState.ActiveAcked, oldRaise, oldRaise));
sessions.EmitEvent(ProviderModeProbe(1));
await WaitUntilAsync(
() => monitor.CurrentAlarms.Any(alarm => alarm.AlarmFullReference == AlarmReference
&& alarm.CurrentState == AlarmConditionState.ActiveAcked),
WaitTimeout);
List<AlarmFeedMessage> received = [];
TaskCompletionSource snapshotComplete = new(TaskCreationOptions.RunContinuationsAsynchronously);
using CancellationTokenSource streamCts = new();
Task reader = ReadFeedAsync(monitor, received, snapshotComplete, streamCts.Token, untilSnapshotComplete: true);
await snapshotComplete.Task.WaitAsync(WaitTimeout);
Timestamp newRaise = Timestamp.FromDateTimeOffset(
new DateTimeOffset(2026, 8, 10, 13, 0, 20, TimeSpan.Zero));
sessions.SetReconcileSnapshot(SnapshotAt(AlarmConditionState.Active, newRaise, newRaise));
sessions.EmitEvent(ProviderModeProbe(2));
AlarmFeedMessage raise = await WaitForAsync(
received,
m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition
&& m.Transition.TransitionKind == AlarmTransitionKind.Raise,
WaitTimeout);
Assert.Equal(AlarmReference, raise.Transition.AlarmFullReference);
Assert.Equal(newRaise, raise.Transition.OriginalRaiseTimestamp);
Assert.Equal(newRaise, raise.Transition.TransitionTimestamp);
lock (received)
{
AlarmFeedMessage[] transitions = received
.Where(m => m.PayloadCase == AlarmFeedMessage.PayloadOneofCase.Transition)
.ToArray();
Assert.Single(transitions);
}
await streamCts.CancelAsync();
await reader;
await cts.CancelAsync();
await monitor.StopAsync(CancellationToken.None);
}
/// <summary>
/// A reconcile Raise repair applied while the matching live Raise is still
/// buffered must not double-broadcast: the live transition carrying the same worker
/// timestamp and resulting state the cache already holds is a duplicate and is suppressed.
/// </summary>
@@ -215,7 +275,7 @@ public sealed class GatewayAlarmMonitorAttachOrderTests
}
/// <summary>
/// NEXT-03, the Clear leg. A reconcile Clear repair removes the cache entry, so the
/// The Clear leg of the dedup. A reconcile Clear repair removes the cache entry, so the
/// buffered live Clear is deduped through the tombstone keyed on the instance's original
/// raise timestamp — and a genuinely new raise/clear cycle is never swallowed.
/// </summary>
@@ -430,7 +490,7 @@ public sealed class GatewayAlarmMonitorAttachOrderTests
SourceProvider = AlarmProviderMode.Alarmmgr,
};
// Snapshot carrying the worker-side identity markers the NEXT-03 dedup compares on.
// Snapshot carrying the worker-side identity markers the dedup compares on.
private static ActiveAlarmSnapshot SnapshotAt(
AlarmConditionState state,
Timestamp lastTransition,
@@ -442,7 +502,7 @@ public sealed class GatewayAlarmMonitorAttachOrderTests
return snapshot;
}
// Live transition with explicit worker timestamps, for driving the NEXT-03 dedup.
// Live transition with explicit worker timestamps, for driving the dedup.
private static MxEvent TransitionAt(
ulong sequence,
AlarmTransitionKind kind,

Some files were not shown because too many files have changed in this diff Show More